Stable Diffusion local setup

Generative AI text-to-image models have been growing at increasing rate and are fun. As of late 2024/early 2025, Stable diffusion made by Stability.AI is one of the most popular open-source AI image models, along with DALL-E and closed-source models like Midjourney. (Would also note here that I want to explore text-to-video models next)

Diffusion models

Stable Diffusion and DALL-E are diffusion models, which is one of the generative models and categorized as supervised learning in machine learning. Diffusion models areis also known as latent variable generative models. Without the math details, diffusion models work by destroying training data/images through the successive addition of Gaussian noise (Forward Diffusion or Noising), and then reversing the process to recover the data (Reverse Diffusion or Denoising). After training and learning the diffusion and denoising processes, the diffusion model can generate data by passing randomly sampled noise through the learned denoising process (Sampling procedure).

SD local set up guide

Here are the steps to set up Stable Diffusion XDSL on a local machine.

1. Get Stable Diffusion XDSL 1.0. Download sd_xl_base_1.0_0.9vae.safetensors from https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/tree/main. There could be multiple sources to find this, for example, go to https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0 -> https://github.com/Stability-AI/generative-models -> https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/tree/main

2. Get Stable Diffusion WebUI from https://github.com/AUTOMATIC1111/stable-diffusion-webui. Download the zip file, then unzip locally

3. Execute webui-user.bat` inside the folder to set up WebUI as well as install dependencies

4. This should bring up 127.0.0.1:7860, which is the Web UI.

   • At this point, you can do a quick test to verify things work correctly. On the top left corner, click “Stable Diffusion checkpoint” dropdown, select the default v1-5-pruned-emaonly.safetensors model. Then, enter a simple prompt and click Generate

5. To import the XDSL model, simply paste the sd_xl_base_1.0_0.9vae.safetensors file into to stable-diffusion-webui\models\Stable-diffusion\. Then, in Web UI, refresh the loaded models

   • Change the resolution from 512 to 768 since that works best with XDSL

6. Some sites worth checking out:

   • https://civitai.com/models, where users share LoRAs (Low-Rank Adaptation) of models and more
   • https://kling.kuaishou.com/en, text-to-video model which seems good for video platforms

Reference: https://www.youtube.com/watch?v=mcqREz9xE74