News
Please click here for latest announcements.
Teaser PV
Technology Overview
Image generation using AI, which has become a hot topic in recent years, generally takes input commands called “prompt”. It is a one-to-one process between human and AI, or one could say that the human is giving orders to the AI. Mitsui-chan is based on the concept of "art created with AI and everyone" and will gradually create artworks by taking direction from viewers' comments on YouTube. AI Vtuber Mitsua-chan aims to create an unprecedented participatory art experience where people create art together through VTuber!
An ethical issue is pointed out as to recent AI image generation that copyrighted works have been incorporated into training data sets without permission, even though there are no legal problems in collection of data in such ways. Mitsua-chan doesn’t use the existing trained Stable Diffusion models as they are, but is trained from scratch with public domain or properly licensed images to challenge to improve the ethical issues for artists by giving back to the providers of the images on which the model is trained! At the moment, we are focusing on resolving "ethical issues regarding representation of generated images". Learn more here.
How Training Works
Mitsua-chan's AI is based on the "latent diffusion model" mechanism, and there are three stages in the training process.
-
STEP 1
In the first phase, Mitsua-chan studied the basic concepts of art in an ethical manner by training U-Net and VAE of the model from scratch with over 11 million images of public domain/CC0 or proper license for permission to use. This trained model is published as “Mitsua DiffusionOne”. Please note that this model focuses on solving ethical problems regarding representation of generated images and does not have sophisticated artistic expression yet. At the moment, the text encoder module utilizes pretrained OpenCLIP. Click here for a detailed explanation.
-
STEP 2
In the second phase, images collected from X(Twitter) accounts that have been approved to participate in the training via Opt-in Application and images posted on the Discord server will be included to the training data set, which will be additionally trained together with the images used in phase 1. We will not collect images from accounts other than those approved to participate in the learning process. Please note that we will review your account and may not approve your application if there are any infringements of rights or other concerns.
There are two ways to participate:
1. Participate by tweeting with the hashtag "#みつあ勉強用" on your X(Twitter) account
2. Participate with all of the images posted on your X(Twitter) account from the past.
Please refer to the Opt-in Application page for details on how to participate in each of the above.
There are rules that we ask you to follow for these images submission, so please refer to the Training Image Guidelines for details.
Once your account has been approved, you will be able to join the dedicated Discord server, and you may also post images for training in batches within Discord.
It is also possible to opt-out (not participate in the training) even after you participate in the training. The training process from the second stage onward will be executed from scratch each time, and the effect of the excluded data will not remain in the trained model. All participants in the training will be rewarded according to the number of images and their quality in return for their contribution to Elan Mitsua's painting ability.
->Opt-in Application -
STEP 3
Finally, in the third phase, Elan Mitsua's AI will be fine-tuned with a dataset of images from "#みつあ勉強用" tweets. Images added to this dataset will have a significant impact on Elan Mitsua's painting style and artistic concept development. We are also pleased to list the credits of all the participants for the training on the Contributor Credits page.
Please note that the model trained using those submitted images will be used only for Elan Mitsua's social channels and research and development relating to the Elan Mitsua Project, as described in the Training Image Guidelines, and will not be used for any other purpose. We are also preparing a trial function of the image generation for non-commercial use within a Discord server exclusively for training participants. The trained model itself will not be released to the public because it will be impossible to properly manage the model and its right holders. Please check here for our policy on releasing models in the future.
Elan Mitsua’s learning process
How to participate in Elan Mitsua’s learning
How YouTube streaming works
Elan Mitsua's painting is performed interactively through YouTube streaming.
While discussing the theme of the work or deciding the composition style in real time together with the audience through YouTube comments, the picture will be painted little by little.
Completed works will be released under the CC0 1.0 Universal (CC0 1.0) Public Domain Dedication license. However, if the work contains any derivative elements, the license of the published work will be subject to the respective official derivative guidelines. Details of the streaming and the first streaming schedule will be announced on our SNS account.