top of page

video credits

logging in...

3D render of a brain formed from glowing copper circuit traces rising from a motherboard surface

Rendering Algorithms: From Ray Tracing to NeRF, Gaussian Splatting and Diffusion

Mehmet Karaağaç

•

23 October 2025

•

en

Reading time: 13 minutes

26270 views

Updated on: 30 October 2026


A rendering algorithm is the method a program uses to turn a scene, or a set of photos, into an image. Classic algorithms such as rasterization, ray tracing and path tracing compute the picture from geometry, materials and lights. Neural methods such as NeRF and 3D Gaussian Splatting learn a scene from photographs. Generative models such as diffusion models synthesise a new picture from what they learned from large collections of images. Many renderers in 2026 combine them: a path tracer computes the light and a trained network removes the noise.

This guide is a map of the three families: how each one works, what it costs in time and hardware, and where architects meet it, from the denoiser in a render engine to a Gaussian splat of a building site. Each part is short and links a deeper guide.

Classic Rendering Algorithms

Classic algorithms simulate, or approximate, what light does in a 3D scene. They need a model with geometry, materials and lights, and the same scene with the same settings gives the same image.

Rasterization

Rasterization projects every triangle of a model onto the screen, works out which pixels it covers and shades those pixels. A graphics card does this extremely fast, which is why game engines and real-time rendering tools are built on it. It only approximates effects that depend on light bouncing around the scene, such as soft shadows, reflections and indirect light, so real-time engines add special techniques or some ray tracing for them.

Ray Tracing and Path Tracing

Ray tracing follows rays from the camera into the scene and, from each surface it hits, on toward the lights and toward reflective or transparent surfaces. That gives sharp reflections, refractions and accurate shadows. Path tracing extends the idea: for every pixel it follows many random light paths as they bounce from surface to surface and averages them, a Monte Carlo solution of the rendering equation. It is the method behind production renderers such as Blender's Cycles. Its weakness is noise: with too few samples per pixel the image looks grainy, and a clean image needs many samples. That is exactly where machine learning helps today.

Photon Mapping and Radiosity

Photon mapping works in two passes. It first shoots photons from the lights and stores where they land, then uses that map while it renders from the camera, which makes it good at caustics, the bright patterns light forms through glass or water. Radiosity, an older method, computes how diffuse light is exchanged between patches of the surfaces. Its result does not depend on the camera, so it can be computed once and viewed from anywhere. All of these are ways to compute global illumination: the indirect light that makes an interior look lit rather than spotlit.

Algorithm

How it computes the image

Speed

Typical use

Rasterization

Projects triangles onto the screen and shades the pixels

Real time

Games, real-time engines, modelling viewports

Ray tracing

Follows rays from the camera to surfaces and lights

Interactive to slow, depending on the effects and the hardware

Reflections, refractions, shadows

Path tracing

Averages many random light paths per pixel

Slow; noisy until there are enough samples

Production stills and animation

Photon mapping

Stores photons shot from the lights, then gathers them

Slow; two passes

Caustics, glass, water

Radiosity

Exchanges diffuse light between surface patches

Slow to precompute, then fast to view

Diffuse interiors, baked lighting

Where Machine Learning Speeds Up Rendering Today

Inside today's renderers, machine learning mostly does not replace the light simulation. It cleans, completes or enlarges what the classic algorithm produced, so that fewer samples, pixels or frames have to be computed.

AI Denoisers: OptiX and Open Image Denoise

A denoiser takes a noisy path-traced image and predicts what the clean image would look like. Two common ones, both neural networks:

  • NVIDIA OptiX AI-Accelerated Denoiser: based on an NVIDIA Research paper on a recurrent denoising autoencoder and, according to NVIDIA, trained on tens of thousands of images rendered from one thousand 3D scenes. It runs on NVIDIA GPUs.

  • Intel Open Image Denoise (OIDN): deep-learning denoising filters, open source under the Apache 2.0 license and recognised with a Technical Achievement Award by the Academy of Motion Picture Arts and Sciences. As of October 2026 it runs on CPUs and on Intel, NVIDIA, AMD and Apple GPUs (openimagedenoise.org).

Blender's Cycles offers both: its manual lists OpenImageDenoise, "an AI denoiser", and NVIDIA's OptiX AI denoiser, and notes that the more render passes the denoiser can use as input, the better the result. The same page describes path guiding, which learns the important light directions while Cycles renders and helps, for example, when a room is lit through a small door opening.

Upscaling and Frame Generation: DLSS

NVIDIA describes DLSS as a suite of neural rendering technologies that uses AI to raise frame rates, reduce latency and improve image quality. As of October 2026, DLSS 4.5 brings Dynamic Multi Frame Generation and a second-generation transformer model for Super Resolution and Ray Reconstruction, and DLSS 5 adds what NVIDIA calls 3D-Guided Neural Rendering for lighting and materials. It runs on GeForce RTX graphics cards, in the games and engines that support it. The principle: compute fewer pixels or frames with the classic algorithm and let a trained network fill in the rest.

For a finished still image the equivalent is AI upscaling, which enlarges a picture and invents plausible detail instead of interpolating pixels. In ArchiVinci, a render from any 3D program can first be improved in the Edit module with a written instruction, and with a plan the result can be upscaled to 4096 px on the long side; the AI render enhancer page shows the steps.

Neural Radiance Caching and Neural Texture Compression

Two NVIDIA Research methods show machine learning moving into the light simulation itself. Real-time Neural Radiance Caching for Path Tracing (2021) trains a small network while the scene renders and uses it as a cache of the light in the scene, so a path tracer can follow a few bounces and let the cache stand in for the rest. Random-Access Neural Compression of Material Textures (SIGGRAPH 2023) compresses a material's textures together and decompresses them in real time with a small network optimised for each material; the authors report two more levels of detail, 16 times more texels, at low bitrates.

What V-Ray, Twinmotion and Blender Offer

Product names matter here, because some articles credit render engines with AI features they do not have. As of October 2026:

  • V-Ray 7 (Chaos) lists several AI tools on its what's new page: AI Enhancer, which adds natural variation such as wear and dirt to large surfaces like floors, walls and terrain; the AI Material Generator, now with 4K textures; AI Mood Match (a preview), which matches a scene's lighting to a reference photo; relighting of Gaussian Splats; and, in V-Ray 7 Update 4, an integration with Veras for AI ideation, enhancement and image-to-video.

  • Twinmotion (Epic Games) is a real-time engine built on Unreal Engine, with a Path tracer for higher-quality images. We found no AI-assisted lighting or path-tracing feature in Epic's documentation.

  • Blender uses machine learning in Cycles for denoising (OpenImageDenoise and OptiX) and learns light directions with path guiding, as described above.

How an image generator compares with a production renderer in day-to-day work is the subject of ArchiVinci vs V-Ray.

Neural Scene Representations

Neural scene representations learn a 3D scene from photographs or video instead of having it modelled by hand. Once trained, they show the scene from viewpoints the camera never took, which researchers call novel view synthesis. They are the core of neural rendering.

NeRF (Neural Radiance Fields)

NeRF, introduced by Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi and Ng at ECCV 2020, stores a scene inside a fully connected neural network. The network takes a 5D coordinate, a position (x, y, z) and a viewing direction, and returns the volume density and the view-dependent colour at that point. To render a pixel, NeRF samples a few hundred points along the camera ray, queries the network at each one and combines the answers with volume rendering. It reproduces reflections and soft light well, but the original method renders slowly because the network runs for every sample, and it is trained on a set of photos of the scene taken from known camera positions.

3D Gaussian Splatting (3DGS)

3D Gaussian Splatting (Kerbl, Kopanas, Leimkühler and Drettakis, SIGGRAPH 2023) also learns a scene from multiple photos or videos, but it stores the scene explicitly, as a large set of 3D Gaussians: small, soft ellipsoids, each with a position, a scale and orientation (its anisotropic covariance), an opacity and a view-dependent colour. Training starts from the sparse points produced during camera calibration and adds, removes and adjusts Gaussians until the renders match the photos. To display the scene, the authors built their own fast, visibility-aware renderer that splats the Gaussians onto the screen. No network runs per pixel, and the paper reports real-time display at 100 frames per second or more at 1080p.

For architects, splats are a way to capture what already exists: a site, a street, a heritage building or a finished interior for a property listing. V-Ray 7 can relight Gaussian Splats so that a captured asset matches the lighting of a rendered scene.

The key difference from NeRF is storage: NeRF hides the scene inside a neural network and runs it for every sample along every ray, while 3D Gaussian Splatting keeps explicit Gaussians and splats them onto the screen, which is why it displays in real time and is easier to edit. Neither one designs anything: both reproduce a place that was photographed. The full NeRF vs Gaussian splatting comparison, with a table, is in our NeRF guide.

Signed Distance Functions and Occupancy Networks

Two related methods describe shape rather than appearance. A signed distance function (SDF) returns, for any point, its distance to the nearest surface, negative inside the object and positive outside; DeepSDF (Park et al., 2019) learns such a function with a network. An occupancy network (Mescheder et al., 2018) returns the probability that a point lies inside the object. Both define a surface continuously, and an algorithm such as marching cubes turns it into a mesh. They suit shape reconstruction from scans and modelling more than photorealistic pictures.

Generative Models: Another Route to the Finished Picture

Generative models do not simulate light in a scene at all. They learn from very large collections of images what pictures look like and synthesise a new one, guided by text, a sketch or an uploaded view. Strictly speaking this is image synthesis rather than rendering, but for architects it has become a second way to get from a design to a finished-looking picture; the CGI vs AI guide compares the two routes.

Diffusion Models

Diffusion models start from random noise and remove it step by step until an image appears, steered by the prompt or another input. Latent diffusion (Rombach et al., December 2021) made this practical by working in a compressed latent space instead of on pixels, with a U-Net doing the denoising; it is the basis of Stable Diffusion. Newer models moved to transformers: Diffusion Transformers (DiT, Peebles and Xie, December 2022) and Stable Diffusion 3, built on rectified flow transformers (Esser et al., March 2024).

GANs

GANs, generative adversarial networks (Goodfellow et al., 2014), train two networks against each other: a generator that makes images and a discriminator that tries to tell them from real ones. StyleGAN (Karras, Laine and Aila, NVIDIA, December 2018) set a new standard for controllable, high-quality image generation. Diffusion and transformer models have since taken over most image generation.

Video Models

The same ideas extend to moving images. As of October 2026, Stability AI's Stable Video Diffusion (November 2023) generates 14 or 25 frames at frame rates between 3 and 30 per second, Runway's current model is Gen-4.5 and Pika offers Pika 2.5. In ArchiVinci, the Video module animates a still render and uses coins, which come with every plan; the page on how to render to video with AI shows examples.

Where ArchiVinci Fits

ArchiVinci does not trace light through a 3D scene. Its default engine, ArchiVinci 4.0, takes the picture you upload (a 3D model view, a sketch or a photo) as a reference and generates a new image that follows it, in seconds. The Imagine module works from a written description alone (in English). ArchiVinci 4.0 keeps the perspective and geometry of your upload, and every run draws a new variation, so run it again for another option or refine a detail in the Edit module; using the results for clients or marketing needs an active plan. The AI render page shows more examples.

Two-story house without materials, all in plain white, and its AI render: white walls, timber cladding upstairs, a dark roof and a lawn.

Made in ArchiVinci's Exterior module on the default engine (ArchiVinci 4.0). Left: the uploaded image, a generated picture of a house without materials. Right: the result of the prompt "Modern house with white plaster walls, timber cladding on the upper floor, large glass openings, a lawn and a few trees." with the Daylight button.


See an AI render of your own project

No render engine to set up: upload a view of your project, write what you want to see, and get a photorealistic render that follows your image. A new account starts with 3 free credits.

Render your design with AI


Which Rendering Algorithm Fits Which Architecture Task

Task

Algorithm family

Where you meet it

Walk a client through a design model live

Rasterization, with some ray tracing

Real-time engines and game engines

Final marketing stills with accurate light

Path tracing plus an AI denoiser

Production renderers such as V-Ray or Blender's Cycles with OIDN or OptiX

Glass, water and caustics

Photon mapping, or path tracing with caustics support

Production renderers

Capture an existing site, street or interior in 3D

3D Gaussian Splatting, or NeRF

Splats made from photos or drone video

Higher frame rates in a real-time walkthrough

Neural upscaling and frame generation

DLSS on GeForce RTX graphics cards

A larger print of a finished image

AI upscaling

Upscalers; for ArchiVinci results, Upscale with a plan, up to 4096 px on the long side

Concept images from a sketch, a model view or a sentence, in seconds

Generative image models

ArchiVinci and other AI image generators

Research Directions

Differentiable rendering lets the error between a rendered image and a target image flow back through the renderer, so that geometry, materials or lighting can be adjusted until the render matches. It is how NeRF and 3D Gaussian Splatting are trained, and research also uses it to recover materials and lighting from photographs.

Generative 3D is the open question: models that produce whole 3D scenes, not single pictures, from text or images. Today the practical split stays as this guide describes it: classic algorithms for physically based light, neural representations for capturing what exists, and generative models for fast pictures of what does not exist yet.


Try it in ArchiVinci

Describe a building in a sentence, or upload a sketch or a 3D model view: ArchiVinci generates concept images that follow it. A new account starts with 3 free credits.

Frequently Asked Questions


What is a rendering algorithm?

It is the method a program uses to turn a 3D scene, or a set of photos, into an image. Rasterization, ray tracing, path tracing, photon mapping and radiosity compute the image from geometry and light; NeRF and 3D Gaussian Splatting learn a scene from photos; diffusion models generate a new image from learned examples.

What is the difference between rasterization and ray tracing?

Rasterization projects the model's triangles onto the screen and shades the pixels they cover, which is very fast but only approximates reflections, shadows and indirect light. Ray tracing follows rays of light through the scene, which is slower but handles those effects physically. Real-time engines often combine the two.

Do render engines use machine learning today?

Yes, mostly to remove noise and fill in pixels. Blender's Cycles uses the OpenImageDenoise and OptiX AI denoisers, DLSS generates pixels and frames on GeForce RTX cards, and as of October 2026 V-Ray 7 adds tools such as AI Enhancer and the AI Material Generator. The light itself is still computed by classic algorithms.

Is a diffusion model a rendering algorithm?

Not in the classic sense. A diffusion model does not simulate light in a 3D scene; it synthesises an image from what it learned, guided by text, a sketch or an uploaded view. For architects it is an alternative route to a finished-looking picture, fast but without a physical light simulation.

Will machine learning replace ray tracing?

Not in the near term. Today's networks make ray and path tracing cheaper: they denoise, upscale and cache light. Generative models replace the renderer only where a convincing picture matters more than a physically computed one, such as early design concepts.

bottom of page