Python: SPG Previous Frame

An SPG node that reads the same AOV twice, once as the renderer produced it this frame and once as it was one frame ago, and publishes the difference. Whatever moved lights up; whatever held still goes dark.

Running

uv run main.py                                  # CUDA
uv run main.py --scene motion_scene_slang.usda  # Slang

The first step compiles the kernel and can take up to 5 minutes on a cold shader cache. A successful run writes three PNGs to _output/: colour.png, the last frame the renderer drew; motion-moving.png, the difference while the ball moves; and motion-still.png, the difference once the ball has held still.

The Scene

The :-1 on one sourceName selects the previous frame. Highlighted: the two RenderVars the node reads, which name the same AOV at two different times.

motion_scene.usda

def Scope "Render"
{
    def RenderProduct "MotionDemo"
    {
        uniform int2 resolution = (960, 540)
        rel camera = </World/Camera>
        rel orderedVars = [ <LdrColor>, <PreviousColor>, <Motion> ]

        def RenderVar "LdrColor"
        {
            uniform string sourceName = "LdrColor"
            opaque omni:rtx:aov
        }

        # The same AOV, one frame back. The ":-1" suffix on sourceName
        # selects the previous frame.
        def RenderVar "PreviousColor"
        {
            uniform string sourceName = "LdrColor:-1"
            opaque omni:rtx:aov
        }

        def RenderVar "Motion"
        {
            uniform string sourceName = "Motion"
            opaque omni:rtx:aov.connect = <../MotionShader.outputs:Difference>
        }

        def Shader "MotionShader" (
            references = @MotionKernel.usda@
        )
        {
            opaque inputs:Current.connect = <../LdrColor.omni:rtx:aov>
            opaque inputs:Previous.connect = <../PreviousColor.omni:rtx:aov>
        }
    }
}

The Node

Two opaque ports; the scene decides which one reads the past frame:

def Shader "MotionKernel"
{
    uniform token info:implementationSource = "sourceAsset"
    uniform asset info:spg:sourceAsset = @MotionKernel.cu@
    uniform token info:spg:sourceAsset:subIdentifier = "motion"

    # Two resource-inputs. Which one is the past frame follows from
    # the RenderVar the scene connects it to.
    opaque inputs:Current
    opaque inputs:Previous
    float inputs:gain = 6.0
    opaque outputs:Difference
}

-- Two resource-inputs, bound the same way. Which one is the past frame is
-- settled in the scene, by the RenderVar each is connected to.
function motion(inputs, outputs)
    assert(inputs["Current"].rank == 2, "Current must be a 2D image")
    assert(inputs["Previous"].rank == 2, "Previous must be a 2D image")

    local height = inputs["Current"].shape[1]
    local width = inputs["Current"].shape[2]

    outputs["Difference"] = cuda.image(width, height, cuda.uchar4)

    return cuda.kernel({
        args = {
            cuda.int(width),
            cuda.int(height),
            cuda.float(inputs["gain"]),
            cuda.TextureObject(inputs["Current"]),
            cuda.TextureObject(inputs["Previous"]),
            cuda.SurfaceObject(outputs["Difference"]),
        },
    })
end

What the Two Halves Show

The moving half shows the ball and the hole it left; once the ball holds still, the difference collapses to sampling noise.

The still image is not completely black: a path-traced render keeps refining, so consecutive frames of a static scene differ slightly.

Refer to Read a Previous Frame for how far back :-N can read, and for how it differs from a stateful output.