Visual Sensors

This section describes two post-processing pipelines that simulate camera sensor outputs from renderer outputs. Each model reads AOVs from a RenderProduct and publishes its results on that same RenderProduct. The models that trace the scene to simulate non-visual measurements are under Non-Visual Sensors.

The robot lineup from three cameras at the same pose. Left: rendered LdrColor. Center: OmniCameraSensorPostIsp with a sample camera profile. Right: DepthSensorOutputVisualization, black where no measurement is reported.

The Cameras section and its Outputs page describe the render modes and the AOVs a camera product can deliver, such as LdrColor, HdrColor, distance and segmentation. This section describes post-processing of those AOVs. Each model’s Outputs page lists its required inputs. Apply OmniSensorGenericCameraCoreAPI to the Camera prim for the Camera generic core model, and OmniSensorGenericDepthSensorSingleViewAPI to the Camera prim for the Camera single-view stereo depth model. Both models use the existing camera and product prims. A camera carries one of the two schemas, never both; requesting a model’s outputs on the camera’s RenderProduct runs its pipeline, and the schema supplies the parameters. Sensor Configuration states that rule under Creating a sensor, and Output Configuration how a RenderProduct requests outputs.

Each model has a directory of its own below. Its overview says what the model simulates and in what vocabulary, where it begins relative to the rendered view and what it leaves to other parts of ovrtx; under it hang the USD attributes the model answers to, one how-to page per task, and an Outputs page that says what each output holds and which attributes shape it. Read the overview first, then the others as you need them.

The models

The Camera generic core model, a Camera prim with OmniSensorGenericCameraCoreAPI applied, is for consumers that expect frames the way a camera sensor sends them, RAW or processed by an image signal processor (ISP), rather than the renderer’s clean output. It delivers the image at each stage of the sensor and ISP chain, so a perception stack or an ISP under test can be fed from a simulated scene. Start with the overview; its Outputs page lists the outputs each stage publishes.

The Camera single-view stereo depth model, a Camera prim with OmniSensorGenericDepthSensorSingleViewAPI applied, is for consumers that expect a stereo depth camera’s output, with occlusion holes, disparity noise and quantization. From one rendered view it delivers a depth image and a per-pixel point cloud. The overview describes the model; Outputs lists its outputs.

The two models are independent pipelines that can share one RenderProduct. Which camera outputs each one reads is stated on its own pages: Where the model begins for the Camera generic core model and Inputs for the Camera single-view stereo depth model.

• Camera generic core model
• Camera single-view stereo depth model

Related

• Cameras — the section that makes the picture these models start from.

• Cameras: Outputs — every AOV per render mode, the distance outputs among them.

• Render Modes — the render modes behind the input image and what each one delivers.

• Sensor Outputs — mapping an output and reading it back.

• Glossary — AOV and the other terms every sensor page shares, defined once.