Cache Work Across Frames#

Goal. Compute something once instead of every frame.

Before you start. The Lua Launch Script.

The Shape#

The launch function, the global function the launch script defines under the entry-point name, runs every frame; the rest of the script runs once, when it is loaded. Wrap anything that does not change per frame in static:

BlurKernel.cu.lua, from the runnable blur example#
-- One row of Gaussian taps for the stated radius, normalised so the blur keeps
-- the image's brightness. It depends only on the radius, so it is built
-- here rather than recomputed by every GPU thread.
local function gaussianWeights(radius)
    if radius == 0 then
        return cuda.array({ 1.0 }, cuda.float)
    end

    local sigma = radius / 3.0
    local taps, total = {}, 0.0
    for t = -radius, radius do
        local w = math.exp(-0.5 * (t / sigma) ^ 2)
        taps[#taps + 1] = w
        total = total + w
    end
    for i = 1, #taps do
        taps[i] = taps[i] / total
    end
    return cuda.array(taps, cuda.float)
end

-- at the call site, in place of gaussianWeights(radius)
cuda.static(gaussianWeights, radius)
BlurKernel.slang.lua, from the runnable blur example#
-- One row of Gaussian taps for the stated radius, normalised so the blur keeps
-- the image's brightness. It depends only on the radius, so it is built
-- here rather than recomputed by every GPU thread.
local function gaussianWeights(radius)
    if radius == 0 then
        return slang.array({ 1.0 }, slang.float)
    end

    local sigma = radius / 3.0
    local taps, total = {}, 0.0
    for t = -radius, radius do
        local w = math.exp(-0.5 * (t / sigma) ^ 2)
        taps[#taps + 1] = w
        total = total + w
    end
    for i = 1, #taps do
        taps[i] = taps[i] / total
    end
    return slang.array(taps, slang.float)
end

-- at the call site, in place of gaussianWeights(radius)
slang.static(gaussianWeights, radius)

How It Works#

static calls the function once and caches the result for its arguments. When an argument changes, the function runs again and its result replaces the cached one; calls with unchanged arguments reuse what was computed before. Numbers, strings and booleans are compared by value. A table is compared by which table it is, not by what it holds: changing a field of a table passed as an argument does not run the function again, and a table built inside the launch function, such as the one an array input’s .value returns, is a new argument on every frame. The function can be written inline at the call site: it is the same function on every launch.

An error raised by the function fails the launch. The failed call stores no result; a later successful call is cached.

One result per node and function. static keeps, for each node and function, the result of that node’s most recent call. Nodes that run the same launch script file never reuse or replace each other’s results, also when they pass equal arguments, so each node computes its own result once. A node built again, after an edit for instance, starts without results, and the results of a node that no longer exists are released. What the script computes with static while it loads, outside the launch function, belongs to the script, and no node reuses it. A table static returns is the same table on every call of that node that reuses it, so do not change it.

A cached result is also tied to the variables the function uses from outside itself, not to their values. A variable declared at the top level of the script, outside the launch function, is the same variable on every frame, so giving it a new value does not run the function again. A variable declared inside the launch function is a new variable on every frame, so a function that uses one runs again on every frame, even when its value does not change. Pass every value the function reads that can change between frames, or that is declared inside the launch function, as an argument instead. Pass the number, string or boolean itself, such as settings.radius, not a table that holds it.

This is about the launch script, not the GPU. It stops the table being rebuilt on every frame, which is Lua work under an instruction budget, and it is the cheapest of the three ways to avoid redoing work across frames: Keep State Across Frames keeps a resource the node itself wrote, and Read a Previous Frame keeps an AOV any node can read.

The upload is separate. An array whose bytes stay the same from frame to frame stays on the GPU and is not copied again, whether or not it came from static; an array whose bytes change is copied again.

Verify It Worked#

Call warning inside the cached function and count the lines over a run. Across a hundred frames with an unchanging argument, the count must be one for each node that calls the function. One line per frame means the cache key changes every frame, and the usual cause is passing a value-input as the wrapper it arrives as, rather than as its .value.

Use warning rather than info: at the renderer’s default log level info is not printed, so an uncounted cache looks exactly like a working one. Refer to The Lua Launch Script.

When It Goes Wrong#

  • It runs every frame: an argument is changing, often because a value-input is being passed as the wrapper rather than as .value.

  • It runs every frame although its arguments do not change: the function uses a variable declared inside the launch function, or an argument is a table built on every frame. Pass the values themselves as arguments.

  • The result does not follow a change: the function reads a value that is not one of its arguments, such as a variable declared at the top level of the script, or a field of a table argument whose contents change. Pass that value itself as an argument.

  • The node’s graph stops after its first run: the script reached a sandbox limit, such as one million instructions per call, and the log names the node. The limits are fixed and renewed each frame, so heavy work belongs in the GPU code rather than in Lua.