This Stable Diffusion mod will turn you into a Pokémon 

The open-source text-to-image AI has a new use.

An open-source text-to-image AI has been modified to generate Pokémon-like characters from simple text prompts — meaning you can now create a pocket monster based on your likeness.

The background: Text-to-image AIs are computer programs that generate images based on short text prompts, and they’ve exploded in popularity this year, led by the unveiling of the uber-impressive DALL-E 2 image generator in April 2022.

While there’s currently a waitlist to try DALL-E 2, other text-to-image AIs are free to use right now, including Midjourney, NightCafe Creator, and Stable Diffusion, a program released by London- and Los Altos-based startup Stability AI.

Training the Text-to-Pokémon AI took just 6 hours and $10 using a cloud-based GPU.

Text-to-Pokémon: Unlike those other text-to-image AIs, Stable Diffusion is open source, meaning the program’s code and training data is freely available online. That gives users the ability not only to generate images with the AI, but to modify the model itself.

Now, machine learning researcher Justin Pinkney has trained the Stable Diffusion AI on a captioned dataset of nearly 1,000 images of Pokémon, creating a version that will create a new Pokémon-like creature based on whatever text prompt you give it.

He also released a how-to guide explaining how anyone could fine-tune Stable Diffusion using their own dataset, noting that training the AI took just 6 hours and $10 using a cloud-based GPU.

How it works: Using Pinkey’s Text-to-Pokémon generator is easy: just log in to GitHub and write a brief description of your imagined Pokémon in the “prompt” field. You can opt to receive up to four images for each prompt by changing the “num_outputs” field accordingly.

You can include the names of celebrities in your prompt, or even type in your own name or a short description of yourself to see what you might look like as a Pokémon. (When you’re done having fun there, check out these “old” Pokémon cards generated by DALL-E 2).

The big picture: Had Stability AI decided to keep its source code closed, the fun that Text-to-Pokémon is providing the internet right now likely wouldn’t be possible. However, open sourcing the code for advanced text-to-image AIs could have serious downsides.

People have already figured out how to modify Stable Diffusion’s code to remove its NSFW filter and the watermark that identifies generated images as being made by the AI. Using those mods, they can create illegal NSFW content or deepfakes that aren’t easily identifiable as such. 

Ultimately, the landscape around text-to-image AIs is just starting to take shape, but it seems if we want to be able to enjoy harmless diversions like Text-to-Pokémon, we’re going to need to accept some not-so-innocuous mods of Stable Diffusion, too.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Shining a light on oil fields to make them more sustainable
Sensors and analytics give oil well operators real-time alerts when things go wrong, so they can respond before they become disasters.
OpenAI’s GPT-4 outperforms doctors in another new study
OpenAI’s most powerful AI model, GPT-4, outperformed junior doctors in deciding how to treat patients with eye problems.
Watch the first AI vs. human dogfight using military jets
An AI fighter pilot faced off against a human pilot in a “dogfight” using actual planes — a huge milestone in military automation.
AI can help predict whether a patient will respond to specific tuberculosis treatments
Instead of a one-size-fits-all treatment approach, AI could help personalize treatments for each patient to provide the best outcomes.
New AI music generator makes songs from text prompts
AI music generators — AIs that create new music based on users’ text prompts — are lowering the bar for music creation, for better or worse.
Up Next
Subscribe to Freethink for more great stories