Read the Scene

Goal. Read a camera’s transform and lens values, the attributes of a prim of your own, or one attribute across many prims, inside a launch script, and hand the result to the GPU code.

Before you start. A working node, from Your First Node, and Types and Values for how a number reaches the GPU.

The Shape

A launch script reaches the scene through the rtx table. The product the node’s prim sits under is rtx.product. A prim comes as a prim table: it holds the prim’s path and reads the scene each time you ask it for a value. Refer to Prim Tables.

local camera = rtx.product.cameras[1]       -- the camera this product is bound to
local world = camera.world_matrix           -- 16 numbers, row-major
local focalLength = camera["focalLength"]   -- any attribute of the prim, by name

Nothing here names a camera. Point the product at another one and the script reads that.

How It Works

Start from the product. rtx.product is the RenderProduct the node’s prim sits under. Its cameras and render_vars are lists of prim tables, in authored order, and its attributes read by name, such as resolution. It is nil when the node’s prim is not under a product being rendered. Refer to The Product.

MarkerOverlayKernel.cu.lua, from the runnable scene query example

-- The render product this node sits under, and the camera it is bound to. The camera is a
-- prim table: its attributes read by name, its two transforms by field.
local product = rtx.product
local camera = product.cameras[1]
local cameraWorld = camera.world_matrix
local focalLength = camera["focalLength"]
local horizontalAperture = camera["horizontalAperture"]
local verticalAperture = camera["verticalAperture"]

Read a prim through its table. A prim table answers path, its two transforms as local_matrix and world_matrix, and any attribute by its USD name, prim.focalLength or prim["focalLength"]. What it lacks reads nil, so check before you index a result. What each attribute type reads as is listed in Values.

Reach any prim by path. rtx.stage.prim(path) gives the same kind of table for any prim on the stage, and rtx.stage.prim(base, relative) finds one relative to another, as in rtx.stage.prim(rtx.product, "Shader"). When no prim is there it returns nil and logs why. Here the path comes from a value-input, so the scene decides which prim the node reads:

MarkerOverlayKernel.cu.lua, from the runnable scene query example

-- A prim of the scene's own, by the path a value-input carries. Its relationship reads as
-- a list of path strings, its float as a number.
local settings = rtx.stage.prim(settingsPath)
local markerPaths = settings["markers"]
local crosshairSize = settings["crosshairSize"]

Read many prims at once. rtx.stage.query_attributes(paths, names) reads the named attributes of every path in one call and returns a table keyed by path. LOCAL_MATRIX and WORLD_MATRIX name the two transforms. A prim that is not on the stage is left out of the result, so a script can branch on it:

MarkerOverlayKernel.cu.lua, from the runnable scene query example

-- Many prims, one attribute each: a batch query reads exactly what it names. A world
-- transform is flat and row-major, so a prim's position is entries 13 to 15.
local markers = rtx.stage.query_attributes(markerPaths, { WORLD_MATRIX })
local markerPositions = {}
for _, path in ipairs(markerPaths) do
    local marker = markers[path]
    local world = marker[WORLD_MATRIX]
    markerPositions[#markerPositions + 1] = world[13]
    markerPositions[#markerPositions + 1] = world[14]
    markerPositions[#markerPositions + 1] = world[15]
end

Publish the prims you read. Loading a scene puts its prims on the stage, but the custom attributes of a prim of your own, such as the settings prim here, arrive only when the load names that prim. In Python the load takes a description with a selector:

main.py, from the runnable scene query example

# Loading a scene publishes what the renderer needs of its prims. /World/Overlay is a prim of
# this scene's own, and its custom attributes arrive only because the load names it.
print(f"Loading {args.scene}...", file=sys.stderr)
ordinal = 1
settings = ovstage.population.Selector(
    prim_predicate=ovstage.population.PrimPredicate.has_path(SETTINGS_PRIM),
    property_predicate=ovstage.population.PropertyPredicate.all(),
)
ovstage.population.open_usd_with_desc(
    stage,
    str(args.scene),
    ordinal=ordinal,
    time=ovstage.population.DEFAULT_TIME,
    descs=[ovstage.population.Desc(domains=ovstage.PopulationDomain.RENDERING, selectors=[settings])],
)
stage.advance_write_floor(ordinal, ovstage.Scope.ALL).wait()

Refer to What Is on the Stage.

Hand it to the GPU. A matrix is sixteen numbers and a position list is three per prim, so both go through array, as any Lua table does; refer to Upload Your Own Data. The kernel reads the world matrix as USD stores it: the first three rows are the prim’s axes in world space and the fourth its position.

CUDA

MarkerOverlayKernel.cu.lua, from the runnable scene query example

function markerOverlay(inputs, outputs)
    local height = inputs["Image"].shape[1]
    local width = inputs["Image"].shape[2]

    local scene = readScene(inputs["settings"].value)

    outputs["Overlay"] = cuda.image(width, height, cuda.uchar4)

    return cuda.kernel({
        args = {
            cuda.int(width),
            cuda.int(height),
            cuda.array(scene.cameraWorld, cuda.float),      -- -> const float* cameraWorld
            cuda.float(scene.focalLength),
            cuda.float(scene.horizontalAperture),
            cuda.float(scene.verticalAperture),
            cuda.array(scene.markerPositions, cuda.float),  -- -> const float* markerPositions
            cuda.int(scene.markerCount),
            cuda.float(scene.crosshairSize),
            cuda.TextureObject(inputs["Image"]),
            cuda.SurfaceObject(outputs["Overlay"]),
        },
    })
end

Slang

MarkerOverlayKernel.slang.lua, from the runnable scene query example

function markerOverlay(inputs, outputs)
    local height = inputs["Image"].shape[1]
    local width = inputs["Image"].shape[2]

    local scene = readScene(inputs["settings"].value)

    outputs["Overlay"] = slang.image(width, height, inputs["Image"].dtype)

    return slang.dispatch({
        bind = {
            slang.ParameterBlock(
                slang.int(width),                          -- -> int   width
                slang.int(height),                         -- -> int   height
                slang.float(scene.focalLength),            -- -> float focalLength
                slang.float(scene.horizontalAperture),     -- -> float horizontalAperture
                slang.float(scene.verticalAperture),       -- -> float verticalAperture
                slang.int(scene.markerCount),              -- -> int   markerCount
                slang.float(scene.crosshairSize)           -- -> float crosshairSize
            ),
            slang.StructuredBuffer(slang.array(scene.cameraWorld, slang.float)),      -- -> g_CameraWorld
            slang.StructuredBuffer(slang.array(scene.markerPositions, slang.float)),  -- -> g_MarkerPositions
            slang.Texture2D(inputs["Image"]),                                         -- -> g_Image
            slang.RWTexture2D(outputs["Overlay"]),                                    -- -> g_Overlay
        },
        -- The shader's [numthreads(16, 16, 1)] places the dispatch; the grid follows from the
        -- output's shape.
    })
end

Keep the table, not the values. A prim table kept across frames reads the current frame, but a value read from it keeps what it held when it was read. To follow a prim over frames, keep its table in a script-level local and read the values each frame.

Verify It Worked

Change what the script reads, and the output must follow. In the scene query example, moving the camera’s rig or a marker moves the crosshair drawn on that marker in _output/overlay.png.

When It Goes Wrong

The script stopped with attempt to index a nil value. A read came back nil: the prim path is wrong, or the attribute is misspelt or was not published. Compare one name with nil to tell which. For a custom attribute, check that the load named its prim.

rtx.product is nil. The node’s Shader prim is authored outside the product. The node still runs; reach the product by path with rtx.stage.prim, or author the node under the product.

Text came back as a number. The attribute is a string published through a selector. Author it as a token.

A transform is nil. The prim is not placed in the scene, as RenderProducts and RenderVars are not. Refer to Transforms.

The node failed on an assignment. Prim tables are read-only views of the scene. Keep the value in a local variable instead.

A lookup returned nil. The log says why: a wrong argument, a relative path that cannot be resolved, or no prim at that path.

Refer to Nothing Appears for the general checks.

Related

The rtx Lua Table, Types and Values, Upload Your Own Data, Work Across Render Products