Keep State Across Frames#

Goal. Let a node read what it wrote last frame.

Before you start. Publish, Overwrite and Read Back AOVs.

The Shape#

Mark the output stateful when you describe it. It is then never handed out fresh:

outputs["History"] = cuda.image(width, height, cuda.float4, cuda.stateful)

outputs["History"] = slang.image(image.shape, slang.float4, slang.stateful)

How It Works#

Highlighted: the stateful allocation, and directly beneath it two ordinary outputs, which are handed out fresh every frame. One marker is the entire difference.

TrailKernel.cu.lua, from the runnable stateful node example#
function trail(inputs, outputs)
    local image = inputs["Image"]
    assert(#image.shape == 2, "Input must be a 2D image")
    -- The input is HdrColor, linear radiance, so no exact dtype is pinned here.

    -- shape is 1-indexed: [1] = height (rows), [2] = width (columns).
    local height = image.shape[1]
    local width  = image.shape[2]

    -- The feedback framebuffer, and the only reason the effect exists.
    outputs["History"] = cuda.image(width, height, cuda.float4, cuda.stateful)
    -- Ordinary outputs: published as AOVs and handed out fresh each frame.
    outputs["Live"] = cuda.image(width, height, cuda.uchar4)
    outputs["Trail"] = cuda.image(width, height, cuda.uchar4)

    return cuda.kernel({
        args = {
            cuda.int(width),                            -- -> int width
            cuda.int(height),                           -- -> int height
            cuda.float(inputs["decay"]),                -- -> float decay
            cuda.TextureObject(image),                  -- -> cudaTextureObject_t inputImage
            cuda.SurfaceObject(outputs["History"]),     -- -> cudaSurfaceObject_t history
            cuda.SurfaceObject(outputs["Live"]),        -- -> cudaSurfaceObject_t liveImage
            cuda.SurfaceObject(outputs["Trail"]),       -- -> cudaSurfaceObject_t trailImage
        },
        block = { 32, 32 },
        grid  = { math.ceil(width / 32), math.ceil(height / 32) },
    })
end
TrailKernel.slang.lua, from the runnable stateful node example#
function trail(inputs, outputs)
    local image = inputs["Image"]
    assert(image.rank == 2, "Input must be a 2D image")
    -- The input is HdrColor, linear radiance. Texture2D<float4> converts the
    -- component type on load, so no exact dtype is pinned here.

    -- The feedback framebuffer, and the only reason the effect exists.
    outputs["History"] = slang.image(image.shape, slang.float4, slang.stateful)
    -- Ordinary outputs: published as AOVs and handed out fresh each frame.
    outputs["Live"] = slang.image(image.shape, slang.uchar4)
    outputs["Trail"] = slang.image(image.shape, slang.uchar4)

    return slang.dispatch({
        bind = {
            slang.ParameterBlock(
                slang.float(inputs["decay"])          -- -> float decay
            ),
            slang.Texture2D(image),                    -- -> Texture2D<float4>   g_InImage
            slang.RWTexture2D(outputs["History"]),     -- -> RWTexture2D<float4> g_History
            slang.RWTexture2D(outputs["Live"]),        -- -> RWTexture2D<float4> g_OutLive
            slang.RWTexture2D(outputs["Trail"]),       -- -> RWTexture2D<float4> g_OutTrail
        },
    })
end

stateful is a trailing argument to the allocator that describes the output, image or empty alike, on either language’s table.

A stateful resource is zero-initialised on first use. The first frame reads zeros rather than whatever was in memory, so a node needs no special case for it.

It need not be published. A stateful output is usually scratch space for the node, so it needs no RenderVar. It may use whatever format the algorithm wants, regardless of what the node publishes.

The scene proves it by omission. The highlighted orderedVars lists the input and the two published images. History, which the whole effect depends on, is not there and needs no RenderVar of its own.

trail_scene.usda, from the runnable stateful node example#
def Scope "Render"
{
    def RenderProduct "TrailDemo"
    {
        uniform int2 resolution = (1280, 720)
        rel camera = </World/Camera>
        rel orderedVars = [ <HdrColor>, <LdrLive>, <LdrTrail> ]

        def RenderVar "HdrColor"
        {
            uniform string sourceName = "HdrColor"
            opaque omni:rtx:aov
        }

        def RenderVar "LdrLive"
        {
            uniform string sourceName = "LdrLive"
            opaque omni:rtx:aov.connect = <../TrailKernel.outputs:Live>
        }

        def RenderVar "LdrTrail"
        {
            uniform string sourceName = "LdrTrail"
            opaque omni:rtx:aov.connect = <../TrailKernel.outputs:Trail>
        }

        def Shader "TrailKernel" (
            references = @TrailKernel.usda@
        )
        {
            opaque inputs:Image.connect = <../HdrColor.omni:rtx:aov>
        }
    }
}

Ordinary outputs carry no such guarantee. Do not rely on a per-frame output retaining anything, and write every pixel you intend to publish.

There are three ways to carry something into the next frame, and they differ in what owns the data. A stateful output is the node’s own resource, handed back to it and to nobody else. A previous-frame read is an AOV, so any node that connects to it sees the same history, and it reaches back further than one frame; refer to Read a Previous Frame. A cached computation never reaches the GPU at all and is about not rebuilding work in Lua; refer to Cache Work Across Frames. Pick by asking who needs the data and whether the GPU ever has to see it.

Verify It Worked#

Remove the stateful marker and compare. The feedback should collapse to a single frame’s worth of data. The stateful node example measures exactly this, and reports the lit area with and without the history: with feedback running it is more than ten times larger. The counts themselves move from run to run, because the image is still converging; the ratio is what carries the result.

When It Goes Wrong#

  • Nothing accumulates: the marker is missing, or the node is writing a different resource than it reads.

  • The first frame looks wrong: it is reading zeros, which is the defined behaviour.