Running stable diffusion in a Docker container is highly advantageous due to its ability to provide isolation, portability, and efficient dependency management. Docker containers encapsulate the model and its dependencies, ensuring stability and security while simplifying deployment across diverse environments. They facilitate version control, resource optimization, and easy scaling, making them ideal for production deployments. Additionally, Docker containers enhance security and streamline collaboration by encapsulating the entire environment, enabling rapid experimentation and testing of various configurations, and offering quick deployment for efficient model development and research.
Background
Ever since I startted dabbling in machine learning in 2019, I've been captivated by the idea that we can teach computers not just to compute, but to "imagine." While text-based AI like GPT-3 have been making headlines, I've been particularly enchanted by AI algorithms that generate images—creating digital artworks that can rival human creativity. Among the myriad of techniques out there, the Stable Diffusion algorithm has been a standout for me. Pioneering the balance between stochastic processes and deterministic outcomes, this algorithm utilizes an intricate play of noise patterns and guided training to yield visually compelling images.
The Stable Diffusion algorithm goes through a series of diffusion steps, which are essentially modifications to an image, guided by a neural network. It starts with an initial condition, adds noise, and then incrementally refines it to create stunning end results. The model is trained on a corpus of images to "learn" the aesthetics and features that make the end image something of a marvel.
There's a number of services out there where you can queue up to have a "cloud-based" AI generate your images from prompts and some services will even sell you compute credits for image generation. That's a great cash cow for the gpu owner, but I also have an Nvidia RTX 2070 Super installed in my desktop tower so I figured that I should have a pretty good shot at running an AI model such as Stable Diffusion, right?
After a quick Google search, I was able to get Stable Diffusion up-and-running inside of Docker. So this post is the chronicle of that adventure and the awe inspiring moments of seeing the AI model running on my desktop PC at home bring ChatGPT-generated image prompts to life, pixel by pixel.
Setup & Config
Clone the stable diffusion repo from GitHub
git clone [email protected]:ryanreece/stable-diffusion-webui-docker.git
Run the download profile with the --build flag
docker compose --profile download up --build
Run the auto profile with the --build flag
docker compose --profile auto up --build
Stop the container
docker compose down
Subsequent Runs
Run the auto profile in the background using the -d flag
docker compose --profile auto up -d
Viewing Running Containers
To view the running containers in docker at any time, run the docker ps command from anywhere in the shell.
docker ps
IMAGE COMMAND CREATED STATUS PORTS NAMES
sd-auto:67 "/docker/entrypoint.…" 35 hours ago Up 14 hours 0.0.0.0:7860->7860/tcp webui-docker-auto-1
Validating GPU Resources
Since Docker fundamentally uses the underlying host hardware, the GPU resources on the host machine should natively be available to the platform. To validate that docker can indeed access the GPU, utilize the docker exec functionality to launch the nvidia-smi cli tool within the stable diffusion docker container.
docker exec -it webui-docker-auto-1 sh -c "nvidia-smi"
Sun Sep 17 18:10:21 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.07 Driver Version: 537.34 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 2070 ... On | 00000000:01:00.0 Off | N/A |
| 0% 45C P8 7W / 215W | 1219MiB / 8192MiB | 4% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 20 G /Xwayland N/A |
| 0 N/A N/A 20 G /Xwayland N/A |
| 0 N/A N/A 22 G /Xwayland N/A |
| 0 N/A N/A 23 G /Xwayland N/A |
+---------------------------------------------------------------------------------------+
Web Interface
The web interface to generate imagry via prompts becomes available on localhost:7860
You can customize the theme of the GUI from the settings pane. 🎨
Using ChatGPT for Prompt Generation
In the sweet mix of language and visuals, ChatGPT and Stable Diffusion can come together to create a powerful toolset for generating stunning images. By using ChatGPT for prompt generation, we can articulate complex ideas, emotions, or scenes in natural language, which then serve as the creative blueprint for Stable Diffusion's image generation capabilities. This unlocks a world of possibilities, allowing for the creation of images that are not only visually compelling but also deeply resonant with the intricacies of human thought and expression. The concept of "verbaluce" comes to mind from Chris Hemsworth's Spiderhead movie which premiered on Netflix in 2022.
Some Examples
Prompt
Can you generate five prompts for me that I can use for a stable diffusion AI to create beautiful background images? I'd like the prompts to depict nature in a fantastic way and cover the various landscapes of planet earth such as forest, ocean, desert, tundra, urban.
Reply
Certainly! Here are five prompts that aim to depict nature in a fantastical manner, each tailored to a specific landscape:
- Forest: "Enchanted forest at dusk illuminated by bioluminescent mushrooms and fireflies, where ancient trees have faces and foliage forms intricate patterns."
- Ocean: "Surreal underwater kingdom with glowing coral reefs, iridescent fish, and majestic sea dragons swimming in harmony through tunnels of seaweed."
- Desert: "A magical oasis at nightfall in an endless desert, with golden sands reflecting the moonlight, and towering cacti that bloom with radiant, gem-like flowers."
- Tundra: "A frozen tundra under the northern lights, featuring mythical ice sculptures, ethereal snowflakes the size of your hand, and an aurora sky that dances to an unseen melody."
- Urban: "A bustling steampunk cityscape harmonized with nature, where ivy-covered skyscrapers touch the clouds, and mechanical birds share the sky with real ones."
Feel free to use or modify these prompts for your stable diffusion AI to create beautiful background images!
Beautiful Results
Forest
Ocean
Desert
Tundra
Urban
I wasn't the biggest fan of this urban rendering. From the images generated, the human made objects often looked too warped, skewed, or otherwise unnatural. Perhaps I could adjust some input parameters for better results, but initially I've been more impressed by the imagery which didn't include many human-made elements.
Further Prompt Tweaking
Prompt
Can you generate five more prompts which describe other types of fantastic and beautiful landscapes on Earth? Please exclude human made scenes.
Reply
Of course! Here are five additional prompts that describe fantastic and beautiful landscapes, all free from human-made elements:
- Mountain Range: "A celestial mountain range where each peak is crowned by a halo of light, and rivers of molten gold flow down the valleys, reflecting the constellations above."
- Island Archipelago: "A dreamlike archipelago where the islands float above crystal-clear water, each island hosting a unique, glowing flower that releases spores of light into the twilight sky."
- Rainforest: "A mystical rainforest filled with luminescent flora, towering waterfalls of liquid crystal, and elusive forest spirits that leave trails of shimmering mist as they move."
- Volcanic Landscape: "An otherworldly volcanic landscape with multi-colored lava flows, steam that materializes into fleeting sculptures, and phoenixes rising from the embers, lighting up the night."
- Savanna: "A mesmerizing savanna under a perpetual twilight sky, where bioluminescent grasses light the ground, and majestic, transparent elephants roam, their insides revealing intricate galaxies."
I hope these prompts inspire some stunning background images!
Fine Results
Mountain Range
Island Archipelago
Rainforest
Volcanic Landscape
Savanna
I was much more impressed by the quality of the output for these images. It's amazing to me how descriptive language from one AI model can influence another model to build stunning, near photo-realistic imagry. That isn't to discount the plethora of human-created data which was used to train these models, but being able to synthesize a scene based upon all the available data we humans created is truly a major leap in computer science in my mind.
Next Step? Content Delivery
Now that I've successfully been able to utilize ChatGPT to provide me with beautifully crafted prompts which I can then use to feed to the Stable Diffusion model running on my desktop, all I need to do is curate the results and make them highly available. Queue the need for a process to be able to upload these resources to a Content Delivery Network (CDN). There's a plethora of options available out there, but the most appropriate is likely going to be an S3-style object store.
In the next chronicle, I'll detail how I go about configuring a process to automatically deploy these generated image assets to a CDN. After all, all the images you see on this site are delivered via the cdn.reece.ai CDN! 😉