Sensor Configuration#

To get rendered output from a sensor in ovrtx, the USD stage must describe what to render and which outputs to produce. This is done through two USD prim types from the UsdRender schema:

  • RenderProduct – represents a single sensor output configuration (resolution, which sensor to use, which variables to output).

  • RenderVar – declares a named output variable (e.g., LdrColor, HdrColor) that the renderer produces for a RenderProduct.

These prims can either exist in the USD file being loaded, or be injected at runtime using ovrtx’s USD composition API.

../_images/render-structure.svg

RenderProduct#

Each sensor that you want output from needs a corresponding RenderProduct prim in the stage. The RenderProduct ties together:

  1. A sensor (camera, lidar, etc.) via the rel camera relationship.

  2. One or more output variables via the rel orderedVars relationship, which points to RenderVar prims.

  3. Settings controlling the rendering of the sensor, such as the output image resolution for a camera, or the render mode used to render it. For camera render modes and settings, see Render Modes.

Here is a minimal RenderProduct in USDA that renders LdrColor from a camera at /World/Camera:

def "Render" {
    def RenderProduct "Camera" {
        int2 resolution = (1920, 1080)
        rel camera = </World/Camera>
        rel orderedVars = [<LdrColor>]

        def RenderVar "LdrColor" {
            string sourceName = "LdrColor"
        }
    }
}

render_scope = stage.DefinePrim("/Render")
camera_product = UsdRender.Product.Define(stage, "/Render/Camera")
camera_product.GetResolutionAttr().Set(Gf.Vec2i(1920, 1080))
camera_product.GetCameraRel().SetTargets(["/World/Camera"])

ldr_var = UsdRender.Var.Define(stage, "/Render/Camera/LdrColor")
ldr_var.GetSourceNameAttr().Set("LdrColor")

camera_product.GetOrderedVarsRel().SetTargets(["/Render/Camera/LdrColor"])

The rel camera Relationship#

rel camera must point to the USD prim path of the sensor whose output you want. For camera sensors, this is typically a UsdGeomCamera prim.

rel camera = </World/Camera>

It is also possible to specify multiple sensors in a single RenderProduct:

rel camera = [</World/Environment1/Camera>, </World/Environment2/Camera>]

When a RenderProduct targets multiple sensors, RTX splits the output frame into tiles, one tile per sensor. Use this pattern when the sensors share the same output variables and resolution and the application wants one tiled output for throughput. For a complete camera example, see Python: Tiled Rendering.

For more information about OpenUSD Relationships, see the Learn OpenUSD page on Relationships.

The rel orderedVars Relationship#

rel orderedVars lists the RenderVar prims that this RenderProduct should output. Each entry is a path to a RenderVar prim. You can request multiple outputs from the same sensor:

rel orderedVars = [<LdrColor>, <HdrColor>]

The RenderVar prims can be defined as children of the RenderProduct (as shown above), or in a shared Vars scope elsewhere in the stage.

For more information about OpenUSD Relationships, see the Learn OpenUSD page on Relationships.

RenderVar#

A RenderVar prim declares a named output. The sourceName attribute specifies which renderer output to bind:

def RenderVar "LdrColor" {
    string sourceName = "LdrColor"
}

The available source names depend on the sensor type. See Outputs for camera sensor outputs, Lidar Sensors for lidar point-cloud channels, and Radar Sensors for radar point-cloud channels.

For sensors that produce multi-tensor outputs – lidar and radar point clouds, for example – the RenderVar prim also carries a channels attribute that selects which tensors the output should include. Only listed channels are produced (the sensor model auto-enables a small set like Counts and Flags regardless):

    def "Render"
    {
        def "Products"
        {
            def RenderProduct "LidarProduct"
            {
                rel camera = </World/Lidar>
                rel orderedVars = [<../../Vars/PointCloud>]
            }
        }

        def "Vars"
        {
            def RenderVar "PointCloud"
            {
                uniform string sourceName = "PointCloud"
                string[] channels = [
                    "Coordinates",
                    "Intensity",
                    "Counts",
                    "TimeOffsetNs"
                ]
            }
        }
    }

A RenderVar produces an ovrtx render variable at runtime; a render variable carries one or more named tensors and zero or more named params. See Sensor Outputs for what a render variable output looks like and how to map it in C and Python, and Reading Sensor PointClouds for lidar/radar PointCloud readback.

Independent RenderProducts#

Create one RenderProduct per sensor when each sensor needs independent output variables, resolution, stepping, device pinning, or result handling:

def "Render" {
    def RenderProduct "FrontCamera" {
        rel camera = </World/FrontCamera>
        rel orderedVars = [<../Vars/LdrColor>]
    }

    def RenderProduct "RearCamera" {
        rel camera = </World/RearCamera>
        rel orderedVars = [<../Vars/LdrColor>]
    }

    def "Vars" {
        def RenderVar "LdrColor" {
            string sourceName = "LdrColor"
        }
    }
}

render_scope = stage.DefinePrim("/Render")

front_product = UsdRender.Product.Define(stage, "/Render/FrontCamera")
front_product.GetCameraRel().SetTargets(["/World/FrontCamera"])
front_product.GetOrderedVarsRel().SetTargets(["/Render/Vars/LdrColor"])

rear_product = UsdRender.Product.Define(stage, "/Render/RearCamera")
rear_product.GetCameraRel().SetTargets(["/World/RearCamera"])
rear_product.GetOrderedVarsRel().SetTargets(["/Render/Vars/LdrColor"])

vars_scope = stage.DefinePrim("/Render/Vars")
ldr_var = UsdRender.Var.Define(stage, "/Render/Vars/LdrColor")
ldr_var.GetSourceNameAttr().Set("LdrColor")

When stepping the renderer, pass all the independent RenderProduct paths you want output for:

(
    subLayers = [
        @../../data/ovrtx-test-base.usda@
    ]
)

def Camera "DocsCamera" (
    prepend apiSchemas = ["OmniSensorGenericCameraCoreAPI"]
)
{
}

def "Render"
{
    def RenderProduct "DocsCamera"
    {
        rel camera = </DocsCamera>
        rel orderedVars = [<LdrColor>]

        def RenderVar "LdrColor"
        {
            string sourceName = "LdrColor"
        }
    }
}

products = renderer.step(
    render_products={"/Render/FrontCamera", "/Render/RearCamera"},
    delta_time=1.0 / 60,
)

ovx_string_t rp_paths[] = {
    {"/Render/FrontCamera", strlen("/Render/FrontCamera")},
    {"/Render/RearCamera",  strlen("/Render/RearCamera")},
};
ovrtx_render_product_set_t render_products = {};
render_products.render_products = rp_paths;
render_products.num_render_products = 2;

ovrtx_step_result_handle_t step_handle = 0;
enqueue_result = ovrtx_step(renderer, render_products, 1.0 / 60.0, &step_handle);

Adding RenderProducts at Runtime#

If the USD file you are loading does not already contain the RenderProduct and RenderVar prims, you can compose them into an inline root layer without editing the original scene.

Create an inline USDA string that uses subLayers to include the original scene and authors the RenderProducts in the same root layer. Load that string with open_usd_from_string (Python) or ovrtx_open_usd_from_string (C):

renderer.open_usd_from_string(f'''
#usda 1.0
(
    subLayers = [
        @{scene_path}@
    ]
)

def "Render" {{
    def RenderProduct "Camera" {{
        int2 resolution = (1920, 1080)
        rel camera = </Camera0>
        rel orderedVars = [<LdrColor>, <HdrColor>]

        def RenderVar "LdrColor" {{
            string sourceName = "LdrColor"
        }}

        def RenderVar "HdrColor" {{
            string sourceName = "HdrColor"
        }}
    }}
}}
''')

products = renderer.step(
    render_products={"/Render/Camera"},
    delta_time=1.0 / 60,
)

    std::string scene_path = get_test_data_dir() + "/simple_camera.usda";
    std::string usda = make_sublayer_usda(scene_path, R"usda(
def "Render" {
    def RenderProduct "Camera" {
        int2 resolution = (640, 480)
        rel camera = </Camera0>
        rel orderedVars = [<LdrColor>, <HdrColor>]

        def RenderVar "LdrColor" {
            string sourceName = "LdrColor"
        }

        def RenderVar "HdrColor" {
            string sourceName = "HdrColor"
        }
    }
}
)usda");
    ovrtx_enqueue_result_t enqueue_result = ovrtx_open_usd_from_string(renderer, {usda.c_str(), usda.size()});
    ASSERT_API_SUCCESS(enqueue_result.status);

    ovrtx_op_wait_result_t wait_result;
    result = ovrtx_wait_op(renderer, enqueue_result.op_index,
                           ovrtx_timeout_infinite, &wait_result);
    ASSERT_API_SUCCESS(result.status);
    ASSERT_NO_OP_ERRORS(wait_result);

Tip

Use add_usd_reference / add_usd_reference_from_string only when you need to add removable referenced content after a root stage is already open. For documentation snippets that need one composed root stage, prefer the inline subLayers pattern shown above.

Advanced RenderProduct Controls#

The sections below describe controls authored on the RenderProduct after the basic sensor/output wiring is in place: renderer settings, warm-up behavior, and per-product GPU device allow-lists. They are collected here because they affect how a RenderProduct produces output. For camera-mode-specific settings and examples, see Render Modes.

Render Settings#

Render settings control how the renderer produces output – for example, the maximum number of path tracing bounces, denoiser configuration, or post-processing parameters. In ovrtx, render settings are written as attributes on the RenderProduct prim using the standard attribute write API.

Setting names use the omni:rtx: namespace prefix. For example, omni:rtx:rtpt:maxBounces controls the maximum number of bounces in the real-time path tracer.

After changing a render setting, call reset() (Python) or ovrtx_reset() (C) and run warm-up frames to allow the renderer to reconverge with the new setting.

Use write_attribute() to write the setting to the RenderProduct prim:

# Set a render setting on the RenderProduct prim
renderer.write_attribute(
    prim_paths=["/Render/Camera"],
    attribute_name="omni:rtx:rtpt:maxBounces",
    tensor=np.array([max_bounces], dtype=np.int32),
)

Use ovrtx_write_attribute() with a DLTensor containing the setting value:

// Set a render setting on the RenderProduct prim
ovx_string_t rp = ovx_str("/Render/Camera");
int32_t value = max_bounces;
size_t count = 1;

DLDataType int32_type = {kDLInt, 32, 1};
DLTensor tensor = ovrtx_make_write_cpu_tensor(&value, &count, int32_type);

ovrtx_input_buffer_t buffer{};
buffer.tensors = &tensor;
buffer.tensor_count = 1;

ovrtx_binding_desc_or_handle_t binding =
    ovrtx_make_binding_desc(&rp, 1, ovx_str("omni:rtx:rtpt:maxBounces"),
                            OVRTX_SEMANTIC_NONE, int32_type);

enqueue_result = ovrtx_write_attribute(renderer_, &binding, &buffer, OVRTX_DATA_ACCESS_SYNC);
ASSERT_API_SUCCESS(enqueue_result.status);
result = ovrtx_wait_op(renderer_, enqueue_result.op_index, ovrtx_timeout_infinite, &wait_result);
ASSERT_API_SUCCESS(result.status);
ASSERT_NO_OP_ERRORS(wait_result);

Warming Up the Renderer#

After loading a scene or changing render settings, the first few rendered frames will not be production quality. There are two independent reasons:

  • Texture streaming – ovrtx streams textures on demand. Initial frames use low-resolution mip levels while higher-resolution data loads in the background.

  • Path tracing convergence – The path tracer accumulates samples over successive frames. Early frames are noisy; quality improves as more samples are gathered.

To get a good quality image, run warm-up frames before capturing output. 40 frames is a conservative default that handles both texture streaming and convergence for typical scenes.

You should warm up after any of the following:

  • Loading a scene with open_usd() / open_usd_from_string() or adding a reference with add_usd_reference* (new textures need to stream in).

  • Calling reset() (accumulated path tracing samples are discarded).

  • Changing render settings that invalidate the accumulation buffer (e.g., bounce counts).

# Warm up - step enough frames for texture streaming to finish loading
# high-res mips and for path tracing to converge to a good quality image.
WARMUP_FRAMES = 40
for _ in range(WARMUP_FRAMES):
    renderer.step(render_products={"/Render/Camera"}, delta_time=1.0 / 60)

// Warm up - step enough frames for texture streaming to finish loading
// high-res mips and for path tracing to converge to a good quality image.
ovx_string_t rp_path = ovx_str("/Render/Camera");
ovrtx_render_product_set_t render_products{};
render_products.render_products = &rp_path;
render_products.num_render_products = 1;

ovrtx_step_result_handle_t step_handle = 0;
int const WARMUP_FRAMES = 40;
for (int i = 0; i < WARMUP_FRAMES; ++i) {
    enqueue_result = ovrtx_step(renderer_, render_products, 1.0 / 60.0, &step_handle);
    ASSERT_API_SUCCESS(enqueue_result.status);
    result = ovrtx_wait_op(renderer_, enqueue_result.op_index, ovrtx_timeout_infinite, &wait_result);
    ASSERT_API_SUCCESS(result.status);
    ASSERT_NO_OP_ERRORS(wait_result);
    ovrtx_destroy_results(renderer_, step_handle);
}

Note

Warm-up frames still produce results. In Python you can ignore them (they are garbage-collected). In C you must call ovrtx_destroy_results() for each step to avoid resource leaks.

RenderProduct Device Pinning#

ovrtx can restrict a RenderProduct to a list of CUDA-visible device indices. Author the uint[] deviceIds attribute directly on the RenderProduct before loading the stage.

deviceIds is an allow-list of indices after CUDA_VISIBLE_DEVICES has filtered and remapped the process-visible GPUs. A singleton list such as [0] constrains the RenderProduct to CUDA-visible GPU 0. A list such as [0, 1] allows ovrtx to choose either visible device.

    def RenderProduct "Camera"
    {
        rel camera = </World/Camera>
        uint[] deviceIds = [0]
        int2 resolution = (640, 320)
        rel orderedVars = <LdrColor>

        def RenderVar "LdrColor"
        {
            string sourceName = "LdrColor"
        }
    }

Use RenderProduct device pinning when:

  • A test or application must run a specific RenderProduct on a deterministic CUDA-visible device.

  • Renderer-level active_cuda_gpus is broader than the device set allowed for one product.

  • Viewport picking is required. In the current ovrtx version, picking works only for RenderProducts running on CUDA-visible GPU 0, so picking products should author deviceIds = [0].