Updated on: 27 October 2026
Diffusion models are generative AI models that create images, video or sound by starting from random noise and removing it step by step, reversing a noising process they learned from training data. They are the technology behind Stable Diffusion, Google's Imagen and OpenAI's DALL·E 2, and distilled versions now need only 1 to 4 steps for an image.
This guide covers how diffusion models work, how a model is built, the main types and their history, how they compare with GANs, VAEs and autoregressive models, the popular models as of October 2026, and what diffusion can and cannot do for architects.
What Are Diffusion Models?
A diffusion model learns by watching data being destroyed. During training, real images are mixed with a little Gaussian noise at a time until only noise is left, and a neural network learns to predict the noise that was added at each step. To generate, the model runs the process backwards: it starts from pure noise and removes a little at a time until a new image appears. Because it learns the probability distribution of its training data rather than storing examples, each new starting noise gives a different result.
The name comes from physics. Jascha Sohl-Dickstein and his co-authors introduced the method in 2015, inspired by non-equilibrium statistical physics, where diffusion describes particles spreading until they are evenly mixed: the model gradually destroys the structure of the data with a forward diffusion process and learns a reverse process that restores it. Diffusion became the leading approach to image generation after DDPMs in 2020 and the 2021 paper "Diffusion Models Beat GANs on Image Synthesis".
How Do Diffusion Models Work?
A diffusion model works in two directions: a fixed forward process that adds noise to training images, and a learned reverse process that removes it. Creating an image takes four steps:
Forward diffusion (training only): a training image is mixed with a little Gaussian noise at each of many steps, following a noise schedule, until only random noise is left.
Training: a neural network sees a noisy image and its step number and learns to predict the noise that was added.
Reverse diffusion (sampling): to generate, the model starts from pure noise and repeatedly removes the noise it predicts, steered by a text prompt or another condition, until a clean image remains.
Decoding: in a latent diffusion model such as Stable Diffusion, the steps above run in a compressed latent space, and a decoder turns the final result into full-size pixels.
Forward Diffusion Process
The forward process needs no learning: at every step a small, scheduled amount of Gaussian noise is added, so fine detail disappears first and the overall shapes last. The noise schedule decides how fast this happens.
Training the Model
The network, usually a U-Net or a transformer, is shown a training image at a random noise level and asked to predict the noise in it. The difference between its guess and the noise that was actually added is the loss it minimizes. Repeated over a very large set of images and noise levels, this teaches the model what images look like at every stage between noise and detail.
Reverse Diffusion Process
After training, generation runs the chain backwards. The model starts from random noise, predicts the noise, removes part of it and repeats. The original DDPM sampler takes many steps, so it was slow; DDIM samplers produce comparable images 10 to 50 times faster with the same trained model, and distilled models such as SDXL Turbo and FLUX.1 [schnell] need only 1 to 4 steps. The overall composition is settled in the early steps and fine detail in the last ones.
How Text Guides the Image: Text Encoder and Guidance
In a text-to-image model, a text encoder turns the prompt into embeddings, and the denoising network reads them through attention layers at every step. Classifier-free guidance strengthens the effect: the model predicts the noise with and without the prompt and pushes the result toward the prompted version. A higher guidance scale follows the words more closely and gives less variety. That is why wording matters; our guide to AI architecture prompts shows how to describe a building. When words are the only condition, the result is a new picture of what the text describes, which is how text-to-image tools generate concept images.
Turn a written idea into architecture images Describe a building, a room or a landscape in a prompt and the Imagine module generates new images from your words in the browser; write the prompt in English for the best results. A new account starts with 3 free credits. |
Diffusion Model Architecture: Backbone, VAE, Text Encoder and Sampler
A modern image diffusion model is not one network but a pipeline of parts that work together. Their names show up in model cards and release notes, so it helps to know what each one does.
Denoising Backbone: U-Net or Diffusion Transformer (DiT)
The backbone is the network that predicts the noise. Earlier models, Stable Diffusion v1 included, use a U-Net, a convolutional network that shrinks the image to capture its layout and expands it again to restore detail. Diffusion Transformers (DiT), introduced by William Peebles and Saining Xie in December 2022, replace the U-Net with a transformer that works on patches of the latent image; newer models such as FLUX use transformer blocks.
Autoencoder (VAE): Working in Latent Space
Latent diffusion models do not denoise full-size pixels. A variational autoencoder (VAE) first compresses the image into a much smaller latent representation, diffusion runs there, and the VAE decoder turns the final latent back into pixels. Stable Diffusion v1's autoencoder reduces each side of the image by a factor of 8.
Text Encoder: CLIP, T5 or a Vision-Language Model
The text encoder turns the prompt into the embeddings that condition the backbone. Models use different ones: Stable Diffusion v1 uses a frozen CLIP ViT-L/14 text encoder, Google's Imagen used the T5 language model, and FLUX.2 couples a 24-billion-parameter vision-language model (Mistral-3) with its image transformer. The encoder also decides which languages a model understands well, which is why some tools ask for prompts in English.
Attention and Control Adapters
Attention layers let every part of the image take the rest of the image and the prompt into account, which keeps a composition coherent. Control adapters add new conditions without retraining the base model: ControlNet (Lvmin Zhang, Anyi Rao and Maneesh Agrawala, February 2023) locks a pretrained text-to-image model and trains a copy of its encoding layers on edges, depth maps, segmentation or human pose, so the lines of a sketch or the depth of a model view can steer the image.
Sampler and Noise Schedule
The sampler is the algorithm that runs the reverse process: how many steps it takes and how much noise it removes at each one. DDPM and DDIM samplers, and distilled models built for a few steps, trade speed against quality, and the guidance scale sets how strongly the prompt is followed. Together with the noise schedule used in training, these settings explain why the same model can give different results in two apps.
How Stable Diffusion Is Built
Stable Diffusion v1, the best-known latent diffusion model, combines the parts above: an autoencoder with a downsampling factor of 8, an 860-million-parameter U-Net and a CLIP ViT-L/14 text encoder with 123 million parameters. It was pretrained on 256 x 256 images and fine-tuned on 512 x 512 images from a subset of LAION-5B, and it runs on a GPU with at least 10 GB of video memory, according to CompVis's Stable Diffusion repository.
Types of Diffusion Models
Most diffusion models combine several of the types below: Stable Diffusion, for example, is a latent, conditional and guided model that can be sampled with DDIM.
Type | Key idea | Introduced |
|---|---|---|
Denoising diffusion probabilistic model (DDPM) | Adds noise in many small steps and learns to remove it | 2020 |
Score-based model and score SDE | Learns the score, the direction toward more likely data, at every noise level | 2019; SDE framework in 2020 |
Latent diffusion model (LDM) | Runs diffusion in the compressed latent space of an autoencoder | 2021 |
Conditional diffusion model | Takes text, a class label, a mask or an image as an extra input | Text-to-image models such as DALL·E 2 and Imagen, 2022 |
Guided diffusion | Pushes each step toward the condition, with a classifier or without one | 2021 (classifier guidance), 2022 (classifier-free) |
Denoising diffusion implicit model (DDIM) | A faster sampler for a model trained like a DDPM | 2020 |
Flow matching and rectified flow | Moves noise toward data along a learned path; includes diffusion as a special case | Used in FLUX.1 (2024) and FLUX.2 (2025) |
Distilled few-step model | A student model that samples in 1 to 4 steps | SDXL Turbo; FLUX.1 [schnell] (2024) |
Denoising Diffusion Probabilistic Models (DDPMs)
Denoising Diffusion Probabilistic Models, published by Jonathan Ho, Ajay Jain and Pieter Abbeel in 2020, made diffusion competitive for images. They add Gaussian noise over many timesteps, forming a Markov chain, and train a network on a weighted variational bound to reverse it. On 256 x 256 LSUN images, the paper reported sample quality similar to ProgressiveGAN, and most later diffusion models build on its recipe.
Score-Based Generative Models (SGMs) / Score SDEs
Score-based models learn a "score function", the gradient of the log probability density of the data, which points in the direction a sample should move to look more like real data. Yang Song and Stefano Ermon introduced them in 2019; the 2020 score SDE paper by Song and colleagues describes the forward process as a continuous-time stochastic differential equation, generates samples by solving the reverse-time equation, and showed that this framework covers both score-based models and diffusion probabilistic models.
Latent Diffusion Models (LDMs)
Latent Diffusion Models move the diffusion process from pixel space into the latent space of a pretrained autoencoder. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer introduced them in December 2021, noting that diffusion models working in pixel space often needed hundreds of GPU days to train and were expensive to sample. Working on a compact representation cuts that cost while keeping detail.
After denoising, the decoder turns the latent back into an image. Stable Diffusion, publicly released in August 2022, is the best-known latent diffusion model: CompVis built it on its LDM paper in collaboration with Stability AI and Runway. Not every famous model is latent: DALL·E 2 uses a prior that produces a CLIP image embedding and a diffusion decoder that turns it into an image.
Conditional Diffusion Models
Conditional diffusion models add external information as a guiding signal: text embeddings, class labels, segmentation masks or an image. In text-to-image systems, a written description shapes the composition, style and subject of the output; in image-to-image systems, an uploaded picture shapes it as well. A condition steers the result but does not fix every detail, which matters for architects, as the last section explains.
Guided Diffusion Models
Guided diffusion steers the reverse process toward a condition at sampling time, without retraining the base model. Classifier guidance, introduced by Prafulla Dhariwal and Alex Nichol in 2021, uses the gradients of a separately trained image classifier; classifier-free guidance, by Jonathan Ho and Tim Salimans (2022), trains one model with and without the condition and combines both predictions. In both cases, stronger guidance follows the condition more closely at the cost of variety.
Denoising Diffusion Implicit Models (DDIMs)
Denoising Diffusion Implicit Models, by Jiaming Song, Chenlin Meng and Stefano Ermon (2020), are not a separate kind of trained model but a faster way to sample one: they use the same training procedure as DDPMs and a non-Markovian reverse process that can be deterministic, so the same starting noise gives the same image. The paper reports high-quality samples 10 to 50 times faster in wall-clock time than DDPM sampling.
Flow Matching and Rectified Flow
Flow matching trains a model to move noise toward data along a learned path instead of reversing a fixed noising chain step by step. Black Forest Labs built FLUX.1 (August 2024, 12 billion parameters) on flow matching, which it describes as a general method that includes diffusion as a special case, and FLUX.2 (November 2025) on latent flow matching with a rectified flow transformer.
Few-Step and Distilled Diffusion Models
Distillation trains a fast student model to do in a few steps what a slower teacher does in many. SDXL Turbo uses Adversarial Diffusion Distillation to sample in 1 to 4 steps, down to a single network evaluation, and FLUX.1 [schnell] generates images in 1 to 4 steps. Few-step models are one reason image generators now return results in seconds; real-time engines that draw 3D scenes use a different technique, explained in our guide to real-time rendering.
A Short History of Diffusion Models
Who Invented Diffusion Models?
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan and Surya Ganguli introduced diffusion probabilistic models in March 2015. Yang Song and Stefano Ermon developed score-based generative models in 2019, and the DDPM paper by Jonathan Ho, Ajay Jain and Pieter Abbeel (June 2020) made the method produce high-quality images. Robin Rombach and his co-authors introduced latent diffusion in December 2021, the basis of Stable Diffusion.
When Were Diffusion Models Invented?
The first diffusion model was published in 2015, but early versions were experimental. DDPMs brought high-quality images in 2020, and in 2021 Prafulla Dhariwal and Alex Nichol showed diffusion models beating GANs on image synthesis. Text-to-image diffusion took off in 2022 with DALL·E 2, Imagen and the public release of Stable Diffusion. The timeline below lists the milestones with their sources.
Year | Milestone |
|---|---|
2015 | Diffusion probabilistic models (Sohl-Dickstein, Weiss, Maheswaranathan and Ganguli) |
2019 | Score-based generative models (Song and Ermon) |
2020 | DDPM (Ho, Jain and Abbeel), DDIM and score SDEs |
2021 | Diffusion models beat GANs (Dhariwal and Nichol); latent diffusion (Rombach et al.) |
2022 | DALL·E 2 (April) and Imagen (May) papers; public release of Stable Diffusion (August); classifier-free guidance; Diffusion Transformers (December) |
2023 | ControlNet adds spatial control to text-to-image models |
2024 | FLUX.1 by Black Forest Labs (August, flow matching); Stable Diffusion 3.5 (October) |
2025 | OpenAI describes its 4o image generation as autoregressive (March); FLUX.2 (November) |
2026 | DALL·E 2 and DALL·E 3 removed from the OpenAI API (12 May) |
Advantages and Disadvantages of Diffusion Models
Diffusion models became the leading approach to image generation because they combine image quality, variety and stable training. The price is slower sampling, heavy compute and a kind of control that is never exact.
Advantages of Diffusion Models
High-quality, photorealistic images: sampling refines an image gradually, which gives fine textures and coherent detail. In 2021, diffusion models surpassed the best generative models of the time on ImageNet while covering the variety of the data better.
Variety: every new starting noise gives a different image, so one prompt yields many options.
Stable training: one denoising objective and no contest between a generator and a discriminator make training more predictable than a GAN's and less prone to mode collapse.
Control: text, class labels, masks, edges, depth maps or a reference image can condition the result.
Many tasks from one idea: the same denoising handles inpainting, super-resolution, image-to-image, video and audio.
Disadvantages of Diffusion Models
Slower sampling than one-pass models: the network runs once per step, while a GAN generates in a single pass. DDIM samplers are 10 to 50 times faster than DDPM and distilled models need 1 to 4 steps, so the gap has narrowed a lot.
High compute, memory and energy costs: the LDM paper notes that pixel-space diffusion models often needed hundreds of GPU days to train. Stable Diffusion v1-4's model card lists 150,000 hours on A100 GPUs with an estimated 11,250 kg of CO2-equivalent emissions, and Stable Diffusion v1 needs a GPU with at least 10 GB of memory to run.
Large training sets: competitive models learn from very large image collections, such as a subset of LAION-5B for Stable Diffusion; some developers, such as Adobe for Firefly and Moonvalley, say they train on licensed content.
Sensitive to settings: a poor noise schedule, learning rate or guidance scale can slow training or degrade images.
Limited interpretability and exact control: the process is stochastic and hard to interpret, and a prompt or a reference image steers the result without fixing every detail, so outputs need checking.
Diffusion Models vs GANs, VAEs and Autoregressive Models
Diffusion is one of several ways to generate images. The table compares the main model families.
Model family | How it generates | Speed | Training | Examples |
|---|---|---|---|---|
Diffusion model | Removes noise from random noise over several steps | Slower; distilled models take 1 to 4 steps | Stable: one denoising objective | Stable Diffusion, Imagen, DALL·E 2 |
Flow matching | Moves noise toward data along a learned path | Few steps | Stable, like diffusion | FLUX.1, FLUX.2 |
GAN | A generator makes an image in one pass while a discriminator judges it | Fast: one pass | Can be unstable; risk of mode collapse | ProgressiveGAN, BigGAN |
VAE | Encodes data into a compact latent space and decodes it | Fast: one pass | Stable | Often blurrier images; the autoencoder inside latent diffusion |
Autoregressive model | Predicts the output piece by piece, each piece from the ones before | Many sequential predictions | Stable | OpenAI's 4o image generation; large language models |
GANs are fast but harder to train, VAEs are fast and stable but give softer images, and diffusion models trade speed for quality and variety. The families are often combined: latent diffusion uses a VAE to compress and decompress images, and SDXL Turbo's distillation adds an adversarial loss taken from GANs. Our guide to GANs in architecture shows how GANs are used in design.
Are All AI Image Generators Diffusion Models?
No. In its March 2025 system card, OpenAI says that the 4o image generation in ChatGPT "is an autoregressive model", unlike DALL·E, which it describes as a diffusion model; the architecture of its newer GPT Image 2 model is not stated. FLUX models use flow matching, which includes diffusion as a special case, and the product pages of Google's Nano Banana image models and of the Veo, Runway and Pika video models did not name an architecture when we checked in October 2026. For what ChatGPT can do in day-to-day design work, see ChatGPT for architects.
Diffusion Models vs LLMs
A large language model (LLM), such as the models behind ChatGPT, is autoregressive: it writes text one token at a time, each predicted from the ones before. A diffusion model produces the whole output at once and refines it over several denoising steps. The difference is the method, not the medium: researchers also build diffusion models for text and autoregressive models for images.
What Are Diffusion Models Used For?
Diffusion models generate and edit images, video and sound. These are their main uses.
Text-to-Image Generation
A written prompt becomes a new image. This is what Stable Diffusion, FLUX and Imagen were built for, and it is used for concept art, illustrations, product images and early design ideas.
Image-to-Image and Style Transfer
A model can also take an existing image as a condition and redraw it in another style, under other light or with other materials while keeping its overall composition. Control adapters such as ControlNet let the lines or the depth of the source image steer the result more closely.
Image Editing and Inpainting
Inpainting regenerates only a masked area so that it blends with the rest of the picture: removing an object, changing a floor or trying a new facade material. In ArchiVinci, AI inpainting works through the Edit module: mark an area with the brush or lasso, describe the change, and only that area, with a narrow margin around it, is redrawn.
Super-Resolution and Restoration
Diffusion upscalers predict the detail of a larger version instead of stretching pixels, so new fine lines are a prediction and worth checking. With a plan, ArchiVinci's Upscale enlarges a generated result up to 4,096 px on the long side, as the AI render enhancer page shows; our guide to AI upscaling explains when it helps.
Video Generation
Video diffusion models extend image generation over time, from a text prompt or from a still image; keeping objects, materials and light consistent from frame to frame is the hard part. The main video models are listed in the next section. ArchiVinci's AI image to video turns one still into a 10-second clip, paid with the coins that plans include.
Audio Synthesis
Diffusion also generates speech, music and sound effects by removing noise from audio waveforms or spectrograms instead of pixels.
Popular Diffusion Models in 2026
The tables list widely used image and video generators as of October 2026, with what each developer says about the architecture. Not every popular generator is a diffusion model, and many developers do not say.
Image Generation Models
Model | Developer | Status and licensing (October 2026) | Architecture, as the developer describes it |
|---|---|---|---|
CompVis with Stability AI and Runway; public release in August 2022 | Latest version 3.5 (October 2024), free for commercial and non-commercial use under the Stability AI Community License, which covers organizations with less than $1 million in annual revenue | Latent text-to-image diffusion model | |
Black Forest Labs; FLUX.1 in August 2024, FLUX.2 in November 2025 | FLUX.1 [schnell] open under the Apache 2.0 license, FLUX.1 [dev] with open weights, FLUX.1 [pro] through the API | Flow matching with transformer blocks (12 billion parameters for FLUX.1) | |
Google; paper in May 2022 | Shut down in the Gemini API; Google points to its Nano Banana models | Text-to-image diffusion model with a T5 text encoder | |
OpenAI; DALL·E 2 paper in April 2022 | Removed from the OpenAI API on 12 May 2026; OpenAI recommends its GPT Image models instead | OpenAI describes DALL·E as a diffusion model; DALL·E 2 combines a CLIP prior with a diffusion decoder | |
Adobe Firefly | Adobe | Trained, Adobe says, on licensed content such as Adobe Stock and on public domain content whose copyright has expired | Not described on the page we read |
Video Generation Models
Model | Developer | Status (October 2026) | Architecture, as the developer describes it |
|---|---|---|---|
Google DeepMind | Available in the Gemini app and Google Flow, and to developers | Not named on the model page | |
Runway Gen-3 Alpha and later | Runway; Gen-3 Alpha on 17 June 2024 | Gen-3 Alpha succeeded Gen-2; Runway's site now lists Gen-4 and Gen-4.5 | Not named on the Gen-3 Alpha page |
Pika | Pika | Its site offers Pika 2.5 and Pikaframes, plus hosted third-party models | Not named on its site |
ModelScope Text-to-Video | ModelScope team at Alibaba; technical report in August 2023 | Research model, English prompts only, about 1.7 billion parameters | Multi-stage text-to-video diffusion model that evolves from Stable Diffusion |
VideoCrafter2 | Tencent AI Lab; January 2024 | Open source; text-to-video, with image-to-video in VideoCrafter1 | Video diffusion model extended from Stable Diffusion |
Moonvalley | Text-to-video and image-to-video, trained on licensed data | Not named on its homepage |
Tools made for architects combine models like these with an architecture workflow; our list of the best AI rendering tools compares them.
Diffusion Models in Architecture: What They Can and Cannot Do
For architects, the useful question is not which network a tool uses but what the model is conditioned on, and what that means for the image you get back. Architecture image tools work in two modes, and both share the same limits.
Text-to-Image: Turning a Description into Architecture Images
In text-to-image mode, words are the only condition: the model starts from noise and steers it toward your description. That suits early concepts, moods and massing ideas, but the model knows nothing about your site, your brief or your dimensions. ArchiVinci's Imagine module, presented on the Dream Space AI page, works this way: describe a building, a room or a landscape and it generates new images from the words alone. Write the description in English for the best results.
Image-to-Image: Turning a Sketch or a Model View into an Image
In image-to-image mode, an uploaded picture is a second condition next to the text. Depending on the tool, the picture is turned into edges or a depth map that a control adapter such as ControlNet follows, or it is given to the model as a reference image. Either way, the model still draws a new image from noise: the upload steers it, it is not traced. ArchiVinci's AI render generator works this way: upload a sketch, a photo or a screenshot of a 3D model view, describe materials and light, and its default engine, ArchiVinci 4.0, generates a new image that follows your upload, in seconds. Line drawings and hand sketches also go through AI sketch to render.
Example from our AI render page. Left: the uploaded line drawing. Right: the result in the Interior module with the prompt "Warm modern living room with an oak floor, a beige linen sofa, a low travertine coffee table and sheer curtains.", the Daylight button and the default engine.
Why Two Runs Never Give the Same Image
Every run starts from new random noise, so the same prompt and the same upload give a different image each time. That is useful for exploring options and a limit for documentation: a condition is a request, not a constraint. In the example above, the prompt asked for a travertine coffee table and the model drew a wood top on a slim frame; railings, window mullions, signs and small text are redrawn in the same way. ArchiVinci's default engine is asked to keep your perspective and geometry, which is an instruction to an AI model, not a lock, so check the details that matter. Light is a condition too: the two images below come from the same upload and the same prompt, and only the lighting button changed.
Example from our render page, made in the Exterior module on the default engine from one uploaded picture of a house without materials and the prompt "Modern house with white plaster walls, timber cladding on the upper floor, large glass openings, a lawn and a few trees." Left: the Golden hour button. Right: the Twilight button.
Can a Diffusion Model Produce a Measured Drawing?
No. A diffusion model returns pixels, not geometry: it has no walls, dimensions or scale, only colors that look like them. It can restyle a plan image, which is what ArchiVinci's Floor plan module does when you color a floor plan with AI in one of four styles (3D view, Marker, Solid color or Drawing), but the lengths and areas in the result are not measured. For a plan you can measure and edit, draw it with Floor Plan Maker: walls, doors, windows and rooms with their areas in the browser, a 3D view, and export to DXF, SVG, PDF or PNG. It runs in desktop browsers, drawing and exporting are free, and an existing plan image or PDF can be rebuilt as an editable plan by AI (1 credit on a free account, refunded if the conversion fails).
ArchiVinci: AI Rendering That Follows Your Image
ArchiVinci is a browser app that renders architecture with AI. Upload a sketch, a photo or a screenshot of a view from any 3D, CAD or BIM tool (JPG, PNG or WebP up to 5 MB), add an optional prompt for materials and light, and the default engine generates a new photorealistic image that follows it, in seconds. The image is generated online, so no local graphics card or render farm is needed.
What you control after the upload:
Lighting: buttons such as Daylight, Golden hour and Twilight set the light without lighting words in the prompt; Twilight gives dusk with the lights on.
One area at a time: Edit render lets you mark an area with the brush or lasso and change only that part.
Several takes: every run draws a new image, and with a plan one run returns up to 4 images.
Upscale and Animate: with a plan, Upscale enlarges a result up to 4,096 px on the long side and Animate turns it into a short video.
An account is required, and a new account starts with 3 free credits. Commercial use, such as client work or marketing, needs an active plan; plans are on the pricing page.
See your sketch or model view as a photorealistic image Upload a sketch, a 3D model view or a photo and describe materials and light: ArchiVinci generates a photorealistic render that follows your image, in seconds. A new account starts with 3 free credits. |
Key Takeaways
Diffusion models generate images, video and sound by starting from random noise and removing it step by step, reversing a noising process learned from training data.
Most image models combine a denoising backbone (U-Net or transformer), a VAE for the latent space, a text encoder and a sampler; Stable Diffusion is the best-known latent diffusion model.
Compared with GANs, they train more stably and give more varied, higher-quality images; distilled models such as SDXL Turbo and FLUX.1 [schnell] cut sampling to 1 to 4 steps.
Not every AI image generator is a diffusion model: OpenAI describes the 4o image generation in ChatGPT as autoregressive, and FLUX uses flow matching.
A diffusion model returns pixels, not geometry: a prompt or an uploaded image steers it but does not lock it, so check the details, and draw measured plans in a plan tool.
Try it in ArchiVinci
Describe a building in a sentence, or upload a sketch or a 3D model view: ArchiVinci generates concept images that follow it. A new account starts with 3 free credits.
For early design concepts: AI concept generator
Render a sketch, photo or model view: AI rendering
Frequently Asked Questions
What Is Diffusion?
Diffusion is the natural process in which particles move from areas of high concentration to low concentration until balance is achieved. In machine learning, it describes how noise is gradually added to data and then reversed by a model to recover structure. This controlled disorder allows diffusion models to generate new data from randomness with physical and statistical grounding.
What Is the Goal of Diffusion Models?
The goal is to model complex data distributions and generate realistic samples that resemble real data. By mastering the transformation between noise and structure, diffusion models can synthesize coherent images, audio, and videos. They prioritize statistical accuracy and diversity rather than memorization, supporting creative yet consistent generation.
What Is the Difference Between DDPMs and DDIMs?
DDPMs sample with a random reverse process that adds fresh noise at every step and needs many steps. DDIMs use the same trained model with a reverse process that can be deterministic and sample 10 to 50 times faster, so DDIM is a choice of sampler, not a different model.
What Is a Noise Schedule in Diffusion Models?
A noise schedule defines how much noise is added at each step of the forward process, and so how fast data moves from order to randomness. Linear schedules add noise in constant increments, while cosine schedules let it grow more smoothly. The chosen schedule affects image sharpness and training stability.
What Loss Function Do Diffusion Models Use?
Most diffusion models minimize the mean squared error between the predicted and actual noise at each step. This approach helps the model learn how to reverse corruption accurately. Some variants use perceptual or latent-space losses to improve visual realism and semantic consistency in generated outputs.
How Many Denoising Steps Are Required?
It depends on the sampler and the model: the original DDPM sampler runs a long chain of steps, DDIM is 10 to 50 times faster, and distilled models such as SDXL Turbo and FLUX.1 [schnell] need only 1 to 4 steps.
What Is Classifier-Free Guidance?
Classifier-free guidance improves prompt control without a separate classifier network: the model learns conditional and unconditional modes and combines their predictions during sampling. A higher guidance scale follows the prompt more strictly; lower values allow more variety.
How Long Does It Take to Train a Diffusion Model?
From days to months, depending on model size, data and hardware: small models can be trained on a single GPU, while Stable Diffusion v1-4's model card lists 150,000 hours on A100 GPUs. Fine-tuning an existing model is much faster and within reach of individual creators.
Can You Run Diffusion Models on a CPU?
Yes, but slowly: the denoising steps are heavy parallel computation that GPUs handle far better, and Stable Diffusion v1 is documented for a GPU with at least 10 GB of video memory. In practice, people use a local GPU or an online service.
How Do You Fine-Tune a Diffusion Model?
Fine-tuning adapts a pretrained model to a style, a subject or a product. Methods such as DreamBooth, LoRA and Textual Inversion teach it new concepts from a small set of images while it keeps its general knowledge.
What Programming Languages Support Diffusion Models?
Python dominates the field because of PyTorch and TensorFlow. Libraries such as Hugging Face Diffusers make implementation simpler for both research and production. Models can also be exported to ONNX or integrated into C++ for optimized inference, though Python remains the main language for diffusion development.
Are Diffusion Models Reinforcement Learning?
No. A diffusion model is a generative model trained with a denoising objective on examples of real data: it learns to predict and remove noise, not to act in an environment for a reward.
What Are the Best Diffusion Models in 2026?
There is no single best model: it depends on the task, the license and whether you need open weights. As of October 2026, widely used options include Stable Diffusion 3.5 and Black Forest Labs' FLUX.1 and FLUX.2, some versions with open weights; the tables above show what each developer states about its models.