Picking and Selection#

ovrtx can perform viewport picking against a RenderProduct and can draw selection outlines for prims that the application marks as selected. Picking answers the question “which prims are under this normalized RenderProduct region?” Selection drawing answers a separate question: “which prims should the renderer outline in future frames?”

The two features are designed to be composed by the application. A viewport UI usually turns a click or drag rectangle into an NDC pick query, resolves the picked path ids into prim paths, prints or stores those names, and then writes selection outline group ids for the next rendered frame.

Concepts#

Picking is a one-step query. Queue the query before ovrtx_step(); the next step consumes it and returns a synthetic render var named OVRTX_RENDER_VAR_PICK_HIT. The pick result is not a USD-authored RenderVar. It appears only for a step that consumed a pick query.

Pick rectangles are always in normalized RenderProduct coordinates, not window coordinates. Values use [0, 1] top-left-origin NDC: x increases left-to-right and y increases top-to-bottom. Interactive applications that render into a window, swapchain, or scaled framebuffer must convert the UI coordinates to RenderProduct NDC before enqueueing the query.

In the current version, picking only works for RenderProducts running on CUDA-visible GPU 0. On multi-GPU systems, author uint[] deviceIds = [0] on RenderProducts used for picking. deviceIds is an allow-list of indices into CUDA_VISIBLE_DEVICES; ovrtx can choose any CUDA-visible GPU from the list, so [0] is required when picking must run on CUDA-visible GPU 0.

Pick hit records contain ovx_primpath_t handles, not strings. Resolve those handles through the renderer path dictionary before printing names, updating UI selection state, or setting selection outline groups.

Selection outlines are persistent renderer state. Enable the outline pass when creating the renderer, set a non-zero group id on the prims that should be outlined, and set group 0 to clear the outline for a prim. Different non-zero group ids are distinct outline groups. Global renderer-creation settings control outline width and fill mode; runtime per-group settings control outline and fill colors. Prims opt into a style by passing that group’s id to set_selection_outline_group() / ovrtx_set_selection_outline_group().

Interactive Viewport Workflow#

For click selection, use the NDC rectangle that encloses the clicked RenderProduct pixel. For marquee selection, convert the drag start and end points into normalized RenderProduct coordinates, clamp to [0, 1], and use the normalized rectangle as the query bounds.

After fetching the pick-hit output, validate the schema params before reading the named tensors. Resolve primPath ids when printing or storing prim names. If the viewport should also show selection outlines, clear the previous selection by setting group 0 on its prims, then set group 1 or another non-zero group on the new selection. Duplicate prim paths are allowed when setting selection outline groups; the last occurrence wins.

The picking workflow is:

  1. Queue an NDC pick rectangle with ovrtx_enqueue_pick_query() before the next ovrtx_step().

  2. Fetch the step results and find the synthetic render var named OVRTX_RENDER_VAR_PICK_HIT.

  3. Map that render var on the CPU with ovrtx_map_render_var_output().

  4. Validate the magic and version params, read hitCount, then consume the named tensors such as primPath and worldPositionM.

  5. Resolve each primPath value through the renderer path dictionary from ovrtx_get_path_dictionary().

  6. Optionally set selection outline groups with ovrtx_set_selection_outline_group() so selected prims are outlined in future rendered frames.

Pick Rectangles#

Describe a pick region with ovrtx_pick_query_desc_t.

The rectangle is expressed in normalized RenderProduct coordinates with top-left-origin NDC semantics. It uses the same rectangle convention as the old pixel API: right_ndc and bottom_ndc mark the edge just past the last included pixel. For example, the old one-pixel rectangle [50, 50, 51, 51] on a 100 x 100 RenderProduct becomes [0.50, 0.50, 0.51, 0.51]. The full RenderProduct is [0, 0, 1, 1].

Callers should clamp UI drag endpoints to [0, 1] before enqueueing a query. The API rejects out-of-bounds rectangles, but old pixel-space values that already fall inside [0, 1] are valid NDC and can be ambiguous during migration. Boundary values can be clamped to the valid range to account for floating-point roundoff.

renderer.enqueue_pick_query(
    render_product_path="/Render/Camera",
    left_ndc=left_ndc,
    top_ndc=top_ndc,
    right_ndc=right_ndc,
    bottom_ndc=bottom_ndc,
)
products = renderer.step(
    render_products={"/Render/Camera"},
    delta_time=1.0 / 60.0,
    ordinal=ordinal,
)

ovrtx_pick_query_desc_t pick_desc = {};
pick_desc.render_product_path = rp_path;
pick_desc.left_ndc = left_ndc;
pick_desc.top_ndc = top_ndc;
pick_desc.right_ndc = right_ndc;
pick_desc.bottom_ndc = bottom_ndc;
pick_desc.flags = 0;

ovrtx_enqueue_result_t enqueue_result = ovrtx_enqueue_pick_query(renderer, &pick_desc);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer, enqueue_result.op_index);

Use a one-pixel-equivalent NDC rectangle for click picking. For a RenderProduct of size width by height, the old pixel rectangle [px, py, px + 1, py + 1] is represented by left_ndc = px / width, top_ndc = py / height, right_ndc = (px + 1) / width, and bottom_ndc = (py + 1) / height. If multiple pick queries are queued for the same RenderProduct before one ovrtx_step(), the last query wins.

Pick query flags:

  • OVRTX_PICK_FLAG_GIZMO also requests gizmo picking.

  • OVRTX_PICK_FLAG_INCLUDE_TRACKED_INFO requests tracked hit metadata such as object type, geometry instance id, world position, and world normal when available.

Pick Results#

Pick results are returned by the next step as the synthetic render var OVRTX_RENDER_VAR_PICK_HIT. It is not authored in the USD RenderProduct; it appears only when a pick query is queued.

Map the output on the CPU and always validate the schema params before reading tensors:

mapping = pick_var.map(device=ovrtx.Device.CPU)
magic = int(np.from_dlpack(mapping.params["magic"]).reshape(-1)[0])
version = int(np.from_dlpack(mapping.params["version"]).reshape(-1)[0])
hit_count = int(np.from_dlpack(mapping.params["hitCount"]).reshape(-1)[0])
prim_paths = np.from_dlpack(mapping["primPath"]).copy().reshape(-1)
object_types = np.from_dlpack(mapping["objectType"]).copy().reshape(-1)
geometry_instance_ids = np.from_dlpack(mapping["geometryInstanceId"]).copy().reshape(-1)
world_positions = np.from_dlpack(mapping["worldPositionM"]).copy().reshape((-1, 3))
world_normals = np.from_dlpack(mapping["worldNormal"]).copy().reshape((-1, 3))
mapping.unmap()

if magic != ovrtx.OVRTX_PICK_HIT_MAGIC or version != ovrtx.OVRTX_PICK_HIT_VERSION:
    raise RuntimeError("Unexpected pick-hit schema")

hits = []
for i in range(hit_count):
    prim_path = int(prim_paths[i])
    if prim_path == 0:
        raise RuntimeError("Pick hit has an empty prim path id")
    hits.append(
        {
            "prim_path": prim_path,
            "object_type": int(object_types[i]),
            "geometry_instance_id": int(geometry_instance_ids[i]),
            "world_position": tuple(float(x) for x in world_positions[i]),
            "world_normal": tuple(float(x) for x in world_normals[i]),
        }
    )

DLTensor const* magic_param = find_param(pick_output, "magic");
DLTensor const* version_param = find_param(pick_output, "version");
DLTensor const* hit_count_param = find_param(pick_output, "hitCount");
DLTensor const* prim_path_tensor = find_tensor(pick_output, "primPath");
DLTensor const* world_position_tensor = find_tensor(pick_output, "worldPositionM");
DLTensor const* world_normal_tensor = find_tensor(pick_output, "worldNormal");

EXPECT_NE(magic_param, nullptr);
EXPECT_NE(version_param, nullptr);
EXPECT_NE(hit_count_param, nullptr);
EXPECT_NE(prim_path_tensor, nullptr);
EXPECT_NE(world_position_tensor, nullptr);
EXPECT_NE(world_normal_tensor, nullptr);
if (!magic_param || !version_param || !hit_count_param || !prim_path_tensor || !world_position_tensor ||
    !world_normal_tensor) {
    ovrtx_cuda_sync_t no_sync = {};
    EXPECT_API_SUCCESS(ovrtx_unmap_render_var_output(renderer, pick_output.map_handle, no_sync).status);
    return pick_result;
}
EXPECT_NE(magic_param->data, nullptr);
EXPECT_NE(version_param->data, nullptr);
EXPECT_NE(hit_count_param->data, nullptr);
EXPECT_NE(prim_path_tensor->data, nullptr);
EXPECT_NE(prim_path_tensor->shape, nullptr);
EXPECT_NE(world_position_tensor->data, nullptr);
EXPECT_NE(world_position_tensor->shape, nullptr);
EXPECT_NE(world_normal_tensor->data, nullptr);
EXPECT_NE(world_normal_tensor->shape, nullptr);
if (!magic_param->data || !version_param->data || !hit_count_param->data || !prim_path_tensor->data ||
    !prim_path_tensor->shape || !world_position_tensor->data || !world_position_tensor->shape ||
    !world_normal_tensor->data || !world_normal_tensor->shape) {
    ovrtx_cuda_sync_t no_sync = {};
    EXPECT_API_SUCCESS(ovrtx_unmap_render_var_output(renderer, pick_output.map_handle, no_sync).status);
    return pick_result;
}

uint32_t magic = *static_cast<uint32_t const*>(magic_param->data);
uint32_t version = *static_cast<uint32_t const*>(version_param->data);
uint32_t hit_count = *static_cast<uint32_t const*>(hit_count_param->data);
EXPECT_EQ(magic, OVRTX_PICK_HIT_MAGIC);
EXPECT_EQ(version, OVRTX_PICK_HIT_VERSION);
EXPECT_EQ(prim_path_tensor->ndim, 1);
EXPECT_GE(prim_path_tensor->shape[0], static_cast<int64_t>(hit_count));
EXPECT_EQ(world_position_tensor->ndim, 2);
EXPECT_GE(world_position_tensor->shape[0], static_cast<int64_t>(hit_count));
EXPECT_GE(world_position_tensor->shape[1], 3);
EXPECT_EQ(world_normal_tensor->ndim, 2);
EXPECT_GE(world_normal_tensor->shape[0], static_cast<int64_t>(hit_count));
EXPECT_GE(world_normal_tensor->shape[1], 3);
if (prim_path_tensor->ndim != 1 || prim_path_tensor->shape[0] < static_cast<int64_t>(hit_count) ||
    world_position_tensor->ndim != 2 || world_position_tensor->shape[0] < static_cast<int64_t>(hit_count) ||
    world_position_tensor->shape[1] < 3 || world_normal_tensor->ndim != 2 ||
    world_normal_tensor->shape[0] < static_cast<int64_t>(hit_count) ||
    world_normal_tensor->shape[1] < 3) {
    ovrtx_cuda_sync_t no_sync = {};
    EXPECT_API_SUCCESS(ovrtx_unmap_render_var_output(renderer, pick_output.map_handle, no_sync).status);
    return pick_result;
}

const auto* prim_paths = static_cast<const ovx_primpath_t*>(prim_path_tensor->data);
const auto* world_positions = static_cast<const double*>(world_position_tensor->data);
const auto* world_normals = static_cast<const float*>(world_normal_tensor->data);
std::vector<ovx_primpath_t> prim_path_ids;
for (uint32_t i = 0; i < hit_count; ++i) {
    EXPECT_NE(prim_paths[i], 0u);
    EXPECT_TRUE(std::isfinite(world_positions[i * 3 + 0]));
    EXPECT_TRUE(std::isfinite(world_positions[i * 3 + 1]));
    EXPECT_TRUE(std::isfinite(world_positions[i * 3 + 2]));
    EXPECT_TRUE(std::isfinite(world_normals[i * 3 + 0]));
    EXPECT_TRUE(std::isfinite(world_normals[i * 3 + 1]));
    EXPECT_TRUE(std::isfinite(world_normals[i * 3 + 2]));
    if (std::find(prim_path_ids.begin(), prim_path_ids.end(), prim_paths[i]) == prim_path_ids.end()) {
        prim_path_ids.push_back(prim_paths[i]);
    }
}

The mapped render var exposes uint32 params named magic, version, and hitCount plus named tensors such as primPath, objectType, geometryInstanceId, worldPositionM, and worldNormal.

Resolving Picked Prim Names#

Pick hit records store ovx_primpath_t handles, not strings. Python exposes resolve_prim_path_id() for these ids. In C, get the renderer path dictionary once and resolve path ids with path_dictionary_get_tokens_from_paths() and path_dictionary_get_strings_from_tokens():

picked_paths = {
    renderer.resolve_prim_path_id(hit["prim_path"])
    for hit in hits
}
picked_paths.discard("")

path_dictionary_instance_t path_dictionary = {};
ovrtx_result_t path_dictionary_result = ovrtx_get_path_dictionary(renderer, &path_dictionary);
EXPECT_API_SUCCESS(path_dictionary_result.status);

if (path_dictionary_result.status == OVRTX_API_SUCCESS) {
    for (ovx_primpath_t prim_path : prim_path_ids) {
        std::string path = docs_resolve_primpath(&path_dictionary, prim_path);
        if (!path.empty()) {
            pick_result.paths.insert(path);
            pick_result.prim_path_ids.push_back(prim_path);
        }
    }
}

The C helper used above expands each path id into tokens, then expands each token into the path components:

static std::string docs_resolve_primpath(path_dictionary_instance_t* pd, ovx_primpath_t p) {
    ovx_token_t token_buf[64];
    ovx_token_t* tokens_out = nullptr;
    size_t num_tokens = 0;
    size_t num_processed = 0;
    ovx_api_result_t r = path_dictionary_get_tokens_from_paths(
        pd, &p, 1, token_buf, 64, &tokens_out, &num_tokens, &num_processed);
    if (r.status != OVX_API_SUCCESS || num_processed == 0) {
        return "";
    }
    std::string out;
    for (size_t i = 0; i < num_tokens; ++i) {
        ovx_string_t s{};
        if (path_dictionary_get_strings_from_tokens(pd, &tokens_out[i], 1, &s).status ==
            OVX_API_SUCCESS) {
            out += "/";
            out.append(s.ptr, s.length);
        }
    }
    return out;
}

Selection Outlines#

Selection outlines are disabled by default. Enable them when creating the renderer:

log_file_path = str(output_dir / "picking_selection.ovrtx.log")

config = ovrtx.RendererConfig(
    selection_outline_enabled=True,
    log_file_path=log_file_path,
)
renderer = ovrtx.Renderer(config=config)
stage = ovstage.Stage("ovrtx.docs.picking")
renderer.attach_ovstage(stage)

// Selection-outline requires the renderer be created with the config
// entry flipped on. Attach an ovstage instance to populate USD scenes
// into it; the pick + set_selection_outline_group calls below run on
// the same renderer.
std::string log_path = (get_output_dir() / "PickingSelectionTest-ovrtx.log").string();
ovx_string_t log_path_view = {log_path.c_str(), log_path.size()};

ovrtx_config_entry_t entries[] = {
    ovrtx_config_entry_log_file_path(log_path_view),
    ovrtx_config_entry_selection_outline_enabled(true),
};
ovrtx_config_t config = {entries, 2};

ovrtx_result_t result = ovrtx_create_renderer(&config, &suite_renderer_);
ASSERT_API_SUCCESS(result.status);

Then mark selected prims with non-zero group ids:

center_path_ids = [hit["prim_path"] for hit in center_hits]
renderer.set_selection_outline_group(center_path_ids, 1)

const std::vector<ovx_primpath_t>& selected_path_ids = pick_result.prim_path_ids;
uint8_t outline_group = 1;

ovrtx_enqueue_result_t enqueue_result =
    ovrtx_set_selection_outline_group(renderer_, selected_path_ids.data(), selected_path_ids.size(), &outline_group);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Group 0 clears the outline for a prim. Different non-zero group ids map to distinct outline groups.

renderer.set_selection_outline_group(center_path_ids, 0)

outline_group = 0;

enqueue_result =
    ovrtx_set_selection_outline_group(renderer_, selected_path_ids.data(), selected_path_ids.size(), &outline_group);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Selection Styling#

Selection style has a global part and a per-group part. Configure global outline width and fill mode when creating the renderer:

log_file_path = str(output_dir / "picking_selection_styled.ovrtx.log")

config = ovrtx.RendererConfig(
    selection_outline_enabled=True,
    selection_outline_width=8,
    selection_fill_mode=ovrtx.SelectionFillMode.GROUP_FILL_COLOR,
    log_file_path=log_file_path,
)
renderer = ovrtx.Renderer(config=config)
stage = ovstage.Stage("ovrtx.docs.picking-styled")
renderer.attach_ovstage(stage)

// Same attached-mode wiring as PickingSelectionTest, plus the two
// extra config entries that opt into fixed-width outlines and the
// per-group fill-color path.
std::string log_path = (get_output_dir() / "SelectionStyleTest-ovrtx.log").string();
ovx_string_t log_path_view = {log_path.c_str(), log_path.size()};

ovrtx_config_entry_t entries[] = {
    ovrtx_config_entry_log_file_path(log_path_view),
    ovrtx_config_entry_selection_outline_enabled(true),
    ovrtx_config_entry_selection_outline_width(8),
    ovrtx_config_entry_selection_fill_mode(OVRTX_SELECTION_FILL_MODE_GROUP_FILL_COLOR),
};
ovrtx_config_t config = {entries, 4};

ovrtx_result_t result = ovrtx_create_renderer(&config, &suite_renderer_);
ASSERT_API_SUCCESS(result.status);

Then set runtime colors for the selection groups your application uses:

renderer.set_selection_group_styles({
    1: ovrtx.SelectionGroupStyle(
        outline_color=(1.0, 0.0, 0.0, 1.0),
        fill_color=(0.0, 1.0, 0.0, 1.0),
    ),
    2: ovrtx.SelectionGroupStyle(
        outline_color=(0.0, 0.0, 1.0, 1.0),
        fill_color=(1.0, 0.0, 1.0, 1.0),
    ),
})

const uint8_t group_ids[] = {1u, 2u};
const ovrtx_selection_group_style_t styles[] = {
    {{1.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 1.0f, 0.0f, 1.0f}},
    {{0.0f, 0.0f, 1.0f, 1.0f}, {1.0f, 0.0f, 1.0f, 1.0f}},
};

ovrtx_enqueue_result_t enqueue_result = ovrtx_set_selection_group_styles(renderer_, group_ids, styles, 2);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Finally, assign those group ids to selected prims. This per-prim group value is what connects a prim to its style:

renderer.set_selection_outline_group_strings(["/World/CenterCube", "/World/LeftCube"], [1, 2])

ovx_string_t selected_paths[] = {
    ovx_str("/World/CenterCube"),
    ovx_str("/World/LeftCube"),
};
uint8_t outline_groups[] = {1u, 2u};

enqueue_result = ovrtx_set_selection_outline_group_strings(renderer_, selected_paths, 2, outline_groups);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Fill colors are visible only when the renderer’s fill mode uses per-group fill color, such as GROUP_FILL_COLOR / OVRTX_SELECTION_FILL_MODE_GROUP_FILL_COLOR. Outline dashing and stippling are not supported by the underlying outline pass.

Pickable Prims#

Use set_pickable() / ovrtx_set_pickable() to opt prims out of viewport picking where supported:

center_path_ids = [hit["prim_path"] for hit in center_hits]
renderer.set_pickable(center_path_ids, False)

// ovrtx_set_pickable marks the given prims as excluded from future pick
// queries. Deprecated in ovrtx 0.4 with no ovstage counterpart today —
// retained for compatibility until an ovstage equivalent lands.
const std::vector<ovx_primpath_t>& unpickable_path_ids = pick_result.prim_path_ids;
bool pickable = false;

ovrtx_enqueue_result_t enqueue_result =
    ovrtx_set_pickable(renderer_, unpickable_path_ids.data(), unpickable_path_ids.size(), &pickable);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Reference#

Primary functions:

Primary types and constants: