This Stable Diffusion mod will turn you into a Pokémon 

The open-source text-to-image AI has a new use.

An open-source text-to-image AI has been modified to generate Pokémon-like characters from simple text prompts — meaning you can now create a pocket monster based on your likeness.

The background: Text-to-image AIs are computer programs that generate images based on short text prompts, and they’ve exploded in popularity this year, led by the unveiling of the uber-impressive DALL-E 2 image generator in April 2022.

While there’s currently a waitlist to try DALL-E 2, other text-to-image AIs are free to use right now, including Midjourney, NightCafe Creator, and Stable Diffusion, a program released by London- and Los Altos-based startup Stability AI.

Training the Text-to-Pokémon AI took just 6 hours and $10 using a cloud-based GPU.

Text-to-Pokémon: Unlike those other text-to-image AIs, Stable Diffusion is open source, meaning the program’s code and training data is freely available online. That gives users the ability not only to generate images with the AI, but to modify the model itself.

Now, machine learning researcher Justin Pinkney has trained the Stable Diffusion AI on a captioned dataset of nearly 1,000 images of Pokémon, creating a version that will create a new Pokémon-like creature based on whatever text prompt you give it.

He also released a how-to guide explaining how anyone could fine-tune Stable Diffusion using their own dataset, noting that training the AI took just 6 hours and $10 using a cloud-based GPU.

How it works: Using Pinkey’s Text-to-Pokémon generator is easy: just log in to GitHub and write a brief description of your imagined Pokémon in the “prompt” field. You can opt to receive up to four images for each prompt by changing the “num_outputs” field accordingly.

You can include the names of celebrities in your prompt, or even type in your own name or a short description of yourself to see what you might look like as a Pokémon. (When you’re done having fun there, check out these “old” Pokémon cards generated by DALL-E 2).

The big picture: Had Stability AI decided to keep its source code closed, the fun that Text-to-Pokémon is providing the internet right now likely wouldn’t be possible. However, open sourcing the code for advanced text-to-image AIs could have serious downsides.

People have already figured out how to modify Stable Diffusion’s code to remove its NSFW filter and the watermark that identifies generated images as being made by the AI. Using those mods, they can create illegal NSFW content or deepfakes that aren’t easily identifiable as such. 

Ultimately, the landscape around text-to-image AIs is just starting to take shape, but it seems if we want to be able to enjoy harmless diversions like Text-to-Pokémon, we’re going to need to accept some not-so-innocuous mods of Stable Diffusion, too.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Large language models are biased. Can logic help save them?
MIT researchers trained logic-aware language models to reduce harmful stereotypes like gender and racial biases.
New voice cloning AI lets “you” speak multiple languages
Voice cloning AIs are gaining more abilities, while the amount of audio needed to replicate a person’s voice is shrinking.
New AI can “reimagine” your pictures in infinite ways
Stability AI has launched Stable Diffusion Reimagine, an AI that uses uploaded images as inspiration for new creations.
A search engine researcher explains the promise and peril of letting AI search the web for you
AI responds to questions but doesn’t actually know anything and is prone to making things up. Should you trust is with your internet search?
How “InstructGPT” lobotomizes the insanity of raw GPT
OpenAI has created a version of GPT, called “InstructGPT,” that cuts down on the AI’s bizarre and irrational responses.
Up Next
Subscribe to Freethink for more great stories