This Stable Diffusion mod will turn you into a Pokémon
An open-source text-to-image AI has been modified to generate Pokémon-like characters from simple text prompts — meaning you can now create a pocket monster based on your likeness.
The background: Text-to-image AIs are computer programs that generate images based on short text prompts, and they’ve exploded in popularity this year, led by the unveiling of the uber-impressive DALL-E 2 image generator in April 2022.
While there’s currently a waitlist to try DALL-E 2, other text-to-image AIs are free to use right now, including Midjourney, NightCafe Creator, and Stable Diffusion, a program released by London- and Los Altos-based startup Stability AI.
Training the Text-to-Pokémon AI took just 6 hours and $10 using a cloud-based GPU.
Text-to-Pokémon: Unlike those other text-to-image AIs, Stable Diffusion is open source, meaning the program’s code and training data is freely available online. That gives users the ability not only to generate images with the AI, but to modify the model itself.
Now, machine learning researcher Justin Pinkney has trained the Stable Diffusion AI on a captioned dataset of nearly 1,000 images of Pokémon, creating a version that will create a new Pokémon-like creature based on whatever text prompt you give it.
He also released a how-to guide explaining how anyone could fine-tune Stable Diffusion using their own dataset, noting that training the AI took just 6 hours and $10 using a cloud-based GPU.
How it works: Using Pinkey’s Text-to-Pokémon generator is easy: just log in to GitHub and write a brief description of your imagined Pokémon in the “prompt” field. You can opt to receive up to four images for each prompt by changing the “num_outputs” field accordingly.
You can include the names of celebrities in your prompt, or even type in your own name or a short description of yourself to see what you might look like as a Pokémon. (When you’re done having fun there, check out these “old” Pokémon cards generated by DALL-E 2).
The big picture: Had Stability AI decided to keep its source code closed, the fun that Text-to-Pokémon is providing the internet right now likely wouldn’t be possible. However, open sourcing the code for advanced text-to-image AIs could have serious downsides.
People have already figured out how to modify Stable Diffusion’s code to remove its NSFW filter and the watermark that identifies generated images as being made by the AI. Using those mods, they can create illegal NSFW content or deepfakes that aren’t easily identifiable as such.
Ultimately, the landscape around text-to-image AIs is just starting to take shape, but it seems if we want to be able to enjoy harmless diversions like Text-to-Pokémon, we’re going to need to accept some not-so-innocuous mods of Stable Diffusion, too.
We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].
