Picking and Selection#

ovrtx can perform viewport picking against a RenderProduct and can draw selection outlines for prims that the application marks as selected. Picking answers the question “which prims are under this RenderProduct pixel region?” Selection drawing answers a separate question: “which prims should the renderer outline in future frames?”

The two features are designed to be composed by the application. A viewport UI usually turns a click or drag rectangle into a pick query, resolves the picked path ids into prim paths, prints or stores those names, and then writes selection outline group ids for the next rendered frame.

Concepts#

Picking is a one-step query. Queue the query before ovrtx_step(); the next step consumes it and returns a synthetic render var named OVRTX_RENDER_VAR_PICK_HIT. The pick result is not a USD-authored RenderVar. It appears only for a step that consumed a pick query.

Pick rectangles are always in RenderProduct pixel coordinates, not window coordinates. Interactive applications that render into a window, swapchain, or scaled framebuffer must convert the UI coordinates to RenderProduct pixels before enqueueing the query.

In the current version, picking only works for RenderProducts running on CUDA-visible GPU 0. On multi-GPU systems, author uint[] deviceIds = [0] on RenderProducts used for picking. deviceIds is an allow-list of indices into CUDA_VISIBLE_DEVICES; ovrtx may choose any CUDA-visible GPU from the list, so [0] is required when picking must run on CUDA-visible GPU 0.

Pick hit records contain ovx_primpath_t handles, not strings. Resolve those handles through the renderer path dictionary before printing names, updating UI selection state, or writing selection outline groups.

Selection outlines are persistent renderer state. Enable the outline pass when creating the renderer, write a non-zero group id to the prims that should be outlined, and write group 0 to clear the outline for a prim. Different non-zero group ids are distinct outline groups. Global renderer-creation settings control outline width and fill mode; runtime per-group settings control outline and fill colors. Prims opt into a style by writing that group’s id to omni:selectionOutlineGroup.

Interactive Viewport Workflow#

For click selection, use a 1x1 pick rectangle around the clicked RenderProduct pixel. For marquee selection, convert the drag start and end points into RenderProduct pixels, clamp to the RenderProduct extent, and use the normalized rectangle as the query bounds.

After fetching the pick-hit output, validate the schema params before reading the named tensors. Resolve every primPath id, deduplicate the resolved paths, then print or store the prim names. If the viewport should also show selection outlines, clear the previous selection by writing group 0 to its prims, then write group 1 or another non-zero group to the new selection.

The picking workflow is:

  1. Queue a pick rectangle with ovrtx_enqueue_pick_query() before the next ovrtx_step().

  2. Fetch the step results and find the synthetic render var named OVRTX_RENDER_VAR_PICK_HIT.

  3. Map that render var on the CPU with ovrtx_map_render_var_output().

  4. Validate the magic and version params, read hitCount, then consume the named tensors such as primPath and worldPositionM.

  5. Resolve each primPath value through the renderer path dictionary from ovrtx_get_path_dictionary().

  6. Optionally write selection outline groups with ovrtx_set_selection_outline_group() so selected prims are outlined in future rendered frames.

Pick Rectangles#

Describe a pick region with ovrtx_pick_query_desc_t.

The rectangle is expressed in RenderProduct pixel coordinates. left and top are inclusive. right and bottom are exclusive, so a single clicked pixel uses right = left + 1 and bottom = top + 1.

renderer.enqueue_pick_query(
    render_product_path="/Render/Camera",
    left=left,
    top=top,
    right=right,
    bottom=bottom,
)
products = renderer.step(
    render_products={"/Render/Camera"},
    delta_time=1.0 / 60.0,
)

ovrtx_pick_query_desc_t pick_desc = {};
pick_desc.render_product_path = rp_path;
pick_desc.left = left;
pick_desc.top = top;
pick_desc.right = right;
pick_desc.bottom = bottom;
pick_desc.flags = 0;

ovrtx_enqueue_result_t enqueue_result =
    ovrtx_enqueue_pick_query(renderer_, &pick_desc);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Use a 1x1 rectangle for click picking. For marquee selection, convert window or framebuffer coordinates into RenderProduct pixels, clamp to the RenderProduct extent, then use max + 1 for the exclusive edge. If multiple pick queries are queued for the same RenderProduct before one ovrtx_step(), the last query wins.

Pick Results#

Pick results are returned by the next step as the synthetic render var OVRTX_RENDER_VAR_PICK_HIT. It is not authored in the USD RenderProduct; it appears only when a pick query is queued.

Map the output on the CPU and always validate the schema params before reading tensors:

mapping = pick_var.map(device=ovrtx.Device.CPU)
magic = int(np.from_dlpack(mapping.params["magic"]).reshape(-1)[0])
version = int(np.from_dlpack(mapping.params["version"]).reshape(-1)[0])
hit_count = int(np.from_dlpack(mapping.params["hitCount"]).reshape(-1)[0])
prim_paths = np.from_dlpack(mapping["primPath"]).copy().reshape(-1)
object_types = np.from_dlpack(mapping["objectType"]).copy().reshape(-1)
geometry_instance_ids = np.from_dlpack(mapping["geometryInstanceId"]).copy().reshape(-1)
world_positions = np.from_dlpack(mapping["worldPositionM"]).copy().reshape((-1, 3))
world_normals = np.from_dlpack(mapping["worldNormal"]).copy().reshape((-1, 3))
mapping.unmap()

if magic != ovrtx.OVRTX_PICK_HIT_MAGIC or version != ovrtx.OVRTX_PICK_HIT_VERSION:
    raise RuntimeError("Unexpected pick-hit schema")

hits = []
for i in range(hit_count):
    prim_path = int(prim_paths[i])
    if prim_path == 0:
        raise RuntimeError("Pick hit has an empty prim path id")
    hits.append(
        {
            "prim_path": prim_path,
            "object_type": int(object_types[i]),
            "geometry_instance_id": int(geometry_instance_ids[i]),
            "world_position": tuple(float(x) for x in world_positions[i]),
            "world_normal": tuple(float(x) for x in world_normals[i]),
        }
    )

DLTensor const* magic_param = find_param(pick_output, "magic");
DLTensor const* version_param = find_param(pick_output, "version");
DLTensor const* hit_count_param = find_param(pick_output, "hitCount");
DLTensor const* prim_path_tensor = find_tensor(pick_output, "primPath");
DLTensor const* world_position_tensor = find_tensor(pick_output, "worldPositionM");
DLTensor const* world_normal_tensor = find_tensor(pick_output, "worldNormal");

EXPECT_NE(magic_param, nullptr);
EXPECT_NE(version_param, nullptr);
EXPECT_NE(hit_count_param, nullptr);
EXPECT_NE(prim_path_tensor, nullptr);
EXPECT_NE(world_position_tensor, nullptr);
EXPECT_NE(world_normal_tensor, nullptr);
if (!magic_param || !version_param || !hit_count_param ||
    !prim_path_tensor || !world_position_tensor || !world_normal_tensor) {
    ovrtx_cuda_sync_t no_sync = {};
    EXPECT_API_SUCCESS(ovrtx_unmap_render_var_output(renderer_, pick_output.map_handle, no_sync).status);
    return paths;
}
EXPECT_NE(magic_param->data, nullptr);
EXPECT_NE(version_param->data, nullptr);
EXPECT_NE(hit_count_param->data, nullptr);
EXPECT_NE(prim_path_tensor->data, nullptr);
EXPECT_NE(prim_path_tensor->shape, nullptr);
EXPECT_NE(world_position_tensor->data, nullptr);
EXPECT_NE(world_position_tensor->shape, nullptr);
EXPECT_NE(world_normal_tensor->data, nullptr);
EXPECT_NE(world_normal_tensor->shape, nullptr);
if (!magic_param->data || !version_param->data || !hit_count_param->data ||
    !prim_path_tensor->data || !prim_path_tensor->shape ||
    !world_position_tensor->data || !world_position_tensor->shape ||
    !world_normal_tensor->data || !world_normal_tensor->shape) {
    ovrtx_cuda_sync_t no_sync = {};
    EXPECT_API_SUCCESS(ovrtx_unmap_render_var_output(renderer_, pick_output.map_handle, no_sync).status);
    return paths;
}

uint32_t magic = *static_cast<uint32_t const*>(magic_param->data);
uint32_t version = *static_cast<uint32_t const*>(version_param->data);
uint32_t hit_count = *static_cast<uint32_t const*>(hit_count_param->data);
EXPECT_EQ(magic, OVRTX_PICK_HIT_MAGIC);
EXPECT_EQ(version, OVRTX_PICK_HIT_VERSION);
EXPECT_EQ(prim_path_tensor->ndim, 1);
EXPECT_GE(prim_path_tensor->shape[0], static_cast<int64_t>(hit_count));
EXPECT_EQ(world_position_tensor->ndim, 2);
EXPECT_GE(world_position_tensor->shape[0], static_cast<int64_t>(hit_count));
EXPECT_GE(world_position_tensor->shape[1], 3);
EXPECT_EQ(world_normal_tensor->ndim, 2);
EXPECT_GE(world_normal_tensor->shape[0], static_cast<int64_t>(hit_count));
EXPECT_GE(world_normal_tensor->shape[1], 3);
if (prim_path_tensor->ndim != 1 ||
    prim_path_tensor->shape[0] < static_cast<int64_t>(hit_count) ||
    world_position_tensor->ndim != 2 ||
    world_position_tensor->shape[0] < static_cast<int64_t>(hit_count) ||
    world_position_tensor->shape[1] < 3 ||
    world_normal_tensor->ndim != 2 ||
    world_normal_tensor->shape[0] < static_cast<int64_t>(hit_count) ||
    world_normal_tensor->shape[1] < 3) {
    ovrtx_cuda_sync_t no_sync = {};
    EXPECT_API_SUCCESS(ovrtx_unmap_render_var_output(renderer_, pick_output.map_handle, no_sync).status);
    return paths;
}

const auto* prim_paths =
    static_cast<const ovx_primpath_t*>(prim_path_tensor->data);
const auto* world_positions =
    static_cast<const double*>(world_position_tensor->data);
const auto* world_normals =
    static_cast<const float*>(world_normal_tensor->data);
std::vector<ovx_primpath_t> prim_path_ids;
for (uint32_t i = 0; i < hit_count; ++i) {
    EXPECT_NE(prim_paths[i], 0u);
    EXPECT_TRUE(std::isfinite(world_positions[i * 3 + 0]));
    EXPECT_TRUE(std::isfinite(world_positions[i * 3 + 1]));
    EXPECT_TRUE(std::isfinite(world_positions[i * 3 + 2]));
    EXPECT_TRUE(std::isfinite(world_normals[i * 3 + 0]));
    EXPECT_TRUE(std::isfinite(world_normals[i * 3 + 1]));
    EXPECT_TRUE(std::isfinite(world_normals[i * 3 + 2]));
    prim_path_ids.push_back(prim_paths[i]);
}

The mapped render var exposes uint32 params named magic, version, and hitCount plus named tensors such as primPath, objectType, geometryInstanceId, worldPositionM, and worldNormal.

Resolving Picked Prim Names#

Pick hit records store ovx_primpath_t handles, not strings. Python exposes resolve_prim_path_id() for these ids. In C, get the renderer path dictionary once and resolve path ids with path_dictionary_get_tokens_from_paths() and path_dictionary_get_strings_from_tokens():

picked_paths = {
    renderer.resolve_prim_path_id(hit["prim_path"])
    for hit in hits
}
picked_paths.discard("")

path_dictionary_instance_t path_dictionary = {};
ovrtx_result_t path_dictionary_result =
    ovrtx_get_path_dictionary(renderer_, &path_dictionary);
EXPECT_API_SUCCESS(path_dictionary_result.status);

if (path_dictionary_result.status == OVRTX_API_SUCCESS) {
    for (ovx_primpath_t prim_path : prim_path_ids) {
        std::string path = docs_resolve_primpath(&path_dictionary, prim_path);
        if (!path.empty()) {
            paths.insert(path);
        }
    }
}

The C helper used above expands each path id into tokens, then expands each token into the path components:

static std::string docs_resolve_primpath(path_dictionary_instance_t* pd, ovx_primpath_t p) {
    ovx_token_t token_buf[64];
    ovx_token_t* tokens_out = nullptr;
    size_t num_tokens = 0;
    size_t num_processed = 0;
    ovx_api_result_t r = path_dictionary_get_tokens_from_paths(
        pd, &p, 1, token_buf, 64, &tokens_out, &num_tokens, &num_processed);
    if (r.status != OVX_API_SUCCESS || num_processed == 0) {
        return "";
    }
    std::string out;
    for (size_t i = 0; i < num_tokens; ++i) {
        ovx_string_t s{};
        if (path_dictionary_get_strings_from_tokens(pd, &tokens_out[i], 1, &s).status ==
            OVX_API_SUCCESS) {
            out += "/";
            out.append(s.ptr, s.length);
        }
    }
    return out;
}

Selection Outlines#

Selection outlines are disabled by default. Enable them when creating the renderer:

log_file_path = str(output_dir / "picking_selection.ovrtx.log")

config = ovrtx.RendererConfig(
    selection_outline_enabled=True,
    log_file_path=log_file_path,
)
renderer = ovrtx.Renderer(config=config)

std::string log_path = (get_output_dir() / "PickingSelectionTest-ovrtx.log").string();
ovx_string_t log_path_view = {log_path.c_str(), log_path.size()};

ovrtx_config_entry_t entries[] = {
    ovrtx_config_entry_log_file_path(log_path_view),
    ovrtx_config_entry_selection_outline_enabled(true),
};
ovrtx_config_t config = {entries, 2};

ovrtx_result_t result = ovrtx_create_renderer(&config, &renderer_);
ASSERT_API_SUCCESS(result.status);

Then mark selected prims with non-zero group ids:

picking_renderer.write_attribute(
    prim_paths=["/World/CenterCube"],
    attribute_name=ovrtx.OVRTX_ATTR_NAME_SELECTION_OUTLINE_GROUP,
    tensor=np.array([1], dtype=np.uint8),
)

ovx_string_t selected_path = ovx_str("/World/CenterCube");
uint8_t outline_group = 1;

ovrtx_enqueue_result_t enqueue_result =
    ovrtx_set_selection_outline_group(renderer_, &selected_path, 1, &outline_group);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Group 0 clears the outline for a prim. Different non-zero group ids map to distinct outline groups.

picking_renderer.write_attribute(
    prim_paths=["/World/CenterCube"],
    attribute_name=ovrtx.OVRTX_ATTR_NAME_SELECTION_OUTLINE_GROUP,
    tensor=np.array([0], dtype=np.uint8),
)

outline_group = 0;

enqueue_result =
    ovrtx_set_selection_outline_group(renderer_, &selected_path, 1, &outline_group);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Selection Styling#

Selection style has a global part and a per-group part. Configure global outline width and fill mode when creating the renderer:

log_file_path = str(output_dir / "picking_selection_styled.ovrtx.log")

config = ovrtx.RendererConfig(
    selection_outline_enabled=True,
    selection_outline_width=8,
    selection_fill_mode=ovrtx.SelectionFillMode.GROUP_FILL_COLOR,
    log_file_path=log_file_path,
)
renderer = ovrtx.Renderer(config=config)

std::string log_path = (get_output_dir() / "SelectionStyleTest-ovrtx.log").string();
ovx_string_t log_path_view = {log_path.c_str(), log_path.size()};

ovrtx_config_entry_t entries[] = {
    ovrtx_config_entry_log_file_path(log_path_view),
    ovrtx_config_entry_selection_outline_enabled(true),
    ovrtx_config_entry_selection_outline_width(8),
    ovrtx_config_entry_selection_fill_mode(OVRTX_SELECTION_FILL_MODE_GROUP_FILL_COLOR),
};
ovrtx_config_t config = {entries, 4};

ovrtx_result_t result = ovrtx_create_renderer(&config, &renderer_);
ASSERT_API_SUCCESS(result.status);

Then set runtime colors for the selection groups your application uses:

styled_selection_renderer.set_selection_group_styles({
    1: ovrtx.SelectionGroupStyle(
        outline_color=(1.0, 0.0, 0.0, 1.0),
        fill_color=(0.0, 1.0, 0.0, 1.0),
    ),
    2: ovrtx.SelectionGroupStyle(
        outline_color=(0.0, 0.0, 1.0, 1.0),
        fill_color=(1.0, 0.0, 1.0, 1.0),
    ),
})

const uint8_t group_ids[] = {1u, 2u};
const ovrtx_selection_group_style_t styles[] = {
    {{1.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 1.0f, 0.0f, 1.0f}},
    {{0.0f, 0.0f, 1.0f, 1.0f}, {1.0f, 0.0f, 1.0f, 1.0f}},
};

ovrtx_enqueue_result_t enqueue_result =
    ovrtx_set_selection_group_styles(renderer_, group_ids, styles, 2);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Finally, assign those group ids to selected prims. This per-prim group value is what connects a prim to its style:

styled_selection_renderer.write_attribute(
    prim_paths=["/World/CenterCube", "/World/LeftCube"],
    attribute_name=ovrtx.OVRTX_ATTR_NAME_SELECTION_OUTLINE_GROUP,
    tensor=np.array([1, 2], dtype=np.uint8),
)

ovx_string_t selected_paths[] = {
    ovx_str("/World/CenterCube"),
    ovx_str("/World/LeftCube"),
};
uint8_t outline_groups[] = {1u, 2u};

enqueue_result =
    ovrtx_set_selection_outline_group(renderer_, selected_paths, 2, outline_groups);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Fill colors are visible only when the renderer’s fill mode uses per-group fill color, such as GROUP_FILL_COLOR / OVRTX_SELECTION_FILL_MODE_GROUP_FILL_COLOR. Outline dashing and stippling are not supported by the underlying outline pass.

Pickable Prims#

Use ovrtx_set_pickable() or write OVRTX_ATTR_NAME_PICKABLE to opt prims out of viewport picking where supported:

picking_renderer.write_attribute(
    prim_paths=["/World/CenterCube"],
    attribute_name=ovrtx.OVRTX_ATTR_NAME_PICKABLE,
    tensor=np.array([0], dtype=np.uint8),
)

ovx_string_t unpickable_path = ovx_str("/World/CenterCube");
bool pickable = false;

ovrtx_enqueue_result_t enqueue_result =
    ovrtx_set_pickable(renderer_, &unpickable_path, 1, &pickable);
ASSERT_API_SUCCESS(enqueue_result.status);
docs_wait_no_errors(renderer_, enqueue_result.op_index);

Reference#

Primary functions:

Primary types and constants: