Cache Work Across Frames#

Goal. Compute something once instead of every frame.

Before you start. The Lua Launch Script.

The Shape#

The launch script runs every frame. Wrap anything that does not change per frame in static:

BlurKernel.cu.lua, from the runnable blur example#
-- One row of Gaussian taps for the stated radius, normalised so the blur keeps
-- the image's brightness. It depends on nothing but the radius, so it is built
-- here rather than recomputed by every GPU thread.
--
-- The warning is what makes the caching visible: this line appears once per
-- distinct radius over a run, not once per frame.
local function gaussianWeights(radius)
    warning("blur: building the weight table for radius " .. radius)

    if radius == 0 then
        return cuda.array({ 1.0 }, cuda.float)
    end

    local sigma = radius / 3.0
    local taps, total = {}, 0.0
    for t = -radius, radius do
        local w = math.exp(-0.5 * (t / sigma) ^ 2)
        taps[#taps + 1] = w
        total = total + w
    end
    for i = 1, #taps do
        taps[i] = taps[i] / total
    end
    return cuda.array(taps, cuda.float)
end

-- at the call site, in place of gaussianWeights(radius)
cuda.static(gaussianWeights, radius)
BlurKernel.slang.lua, from the runnable blur example#
local function gaussianWeights(radius)
    warning("blur: building the weight table for radius " .. radius)

    if radius == 0 then
        return slang.array({ 1.0 }, slang.float)
    end

    local sigma = radius / 3.0
    local taps, total = {}, 0.0
    for t = -radius, radius do
        local w = math.exp(-0.5 * (t / sigma) ^ 2)
        taps[#taps + 1] = w
        total = total + w
    end
    for i = 1, #taps do
        taps[i] = taps[i] / total
    end
    return slang.array(taps, slang.float)
end

-- at the call site, in place of gaussianWeights(radius)
slang.static(gaussianWeights, radius)

How It Works#

static calls the function once and caches the result, keyed on the arguments. When an argument changes, the function runs again. Anything else reuses what was computed before.

This is about the launch script, not the GPU. It stops the table being rebuilt on every frame, which is Lua work under an instruction budget, and it is the cheapest of the three ways to avoid redoing work across frames: Keep State Across Frames keeps a resource the node itself wrote, and Read a Previous Frame keeps an AOV any node can read.

It does not stop the upload. SPG keeps an uploaded buffer resident across frames only for a tensor whose strides it reads as implicit row-major, and every array a launch script hands over carries explicit ones, so the bytes are copied to the GPU again each frame. The renderer log says so, once per frame per array:

allocateOrReuseStaticTensorResource: tensor '...' is not dense/row-major (size=68, dense=68); not caching

For a filter kernel of a few dozen values that costs nothing measurable. Size a design around the Lua saving rather than around the transfer.

Verify It Worked#

Call warning inside the cached function and count the lines over a run. Across a hundred frames with an unchanging argument the count must be one. One line per frame means the cache key changes every frame, and the usual cause is passing a value-input as the wrapper it arrives as, rather than as its .value.

Use warning rather than info: at the renderer’s default log level info is not printed, so an uncounted cache looks exactly like a working one. Refer to The Lua Launch Script.

The blur example does this and reports the count. It renders at two radii over some fifty frames and prints weight tables built: 2.

When It Goes Wrong#

  • It runs every frame: an argument is changing, often because a value-input is being passed as the wrapper rather than as .value.

  • The script is stopped part way through: it exceeded the sandbox’s instruction budget. That budget is fixed and renewed each frame, so heavy work belongs in the GPU code rather than in Lua.