[URCapX] Camera Creator

Warning

The AI Accelerator requires ROS 2 Humble, so PolyScope X 10.12.1 is the latest compatible release. PolyScope X 10.13 and later use ROS 2 Jazzy and are not currently supported.

Warning

Kit camera: Install orbbec-gemini-v335Lg-1.3.10.urcapx on the robot together with AIA Core and the SDK URCapX — see SDK — Install URCapX on the robot. This page describes the Camera Creator architecture for custom camera integrations. A custom Camera Creator image must still run on the same Docker host and network as the AIA Core backend.

This guide provides insights into the deployment architecture and demonstrates how components interact and communicate at a system abstraction level.

Why read this guide?

Whether you are developing your own camera solution or integrating cameras from third-party manufacturers with existing ROS2 drivers and integration capabilities, this documentation will help you understand the underlying architecture and design principles. Our goal is to provide you with the knowledge and tools necessary to successfully implement camera solutions within your robotic applications.

Camera Creator sits between the physical camera and the rest of the platform. It is one of the three AI Accelerator components — see the AI Accelerator Core page for the full system overview.

Architecture overview

        flowchart TB
    aiafe["<b>AIA Core frontend</b><br/>Angular (teach pendant)"]
    flask["<b>aia-core-backend</b><br/>REST API"]
    camsvc["<b>Camera Manager Service</b><br/>NodeControl / NodeInfo / NodeList"]
    camnode["<b>Camera Node</b><br/>Orbbec ROS 2 driver"]
    ark["<b>pandai_ark</b><br/>perception + world model"]

    aiafe -- "REST" --> flask
    flask -- "ROS 2 NodeControl" --> camsvc
    camsvc -- "start / stop / restart" --> camnode
    camnode -. "model_config (registration)" .-> flask
    camnode -- "color / depth / camera_info" --> ark

    classDef fe fill:#D7DDDF,stroke:#002B39,stroke-width:1.5px,color:#002B39
    classDef svc fill:#FFFFFF,stroke:#002B39,stroke-width:1.5px,color:#002B39
    classDef node fill:#F7F8F7,stroke:#8FA3AB,stroke-width:1px,color:#33484F
    class aiafe fe
    class flask,camsvc svc
    class camnode,ark node
    

Figure 1CC: Abstract system architecture.

The Camera Creator, the AIA Core URCapX, and the Pandai ARK are deployed in separate docker containers. While AIA Core’s front and backend communicate through a REST API, the components in Fig. 1CC communicate solely through ROS. The Camera Creator is a composite instance that provides a Camera Manager Service, enabling administrative control over Camera Nodes through ROS. The Camera Node operates as a ROS2 node, while the Camera Manager functions as a ROS2 service, facilitating seamless integration and management within the ROS2 ecosystem.

Tip

The AI Accelerator is delivered with the ORRBEC Gemini 335Lg as the default configuration and thus, the framework architecture is optimized for this specific camera model. However, the system is designed to be flexible and can accommodate any camera model with appropriate drivers and ROS2 integration software.

System Flow

        sequenceDiagram
    participant CN as Camera Node
    participant ROS as ROS 2 graph
    participant AIA as AIA Core backend
    Note over CN: 1. Compile YAML config on startup
    CN->>ROS: 2. Publish base64 config on camera_name/model_config
    AIA->>ROS: 3. Monitor for model_config topics
    ROS-->>AIA: matching topic found, subscribe
    AIA->>AIA: 4. Decode YAML, register camera
    Note over AIA: Camera available for calibration, settings, and AI features
    

Figure 2CC: Startup logic and registration flow.

The Camera Creator deployment exposes a Node Control Service that provides three primary control actions: start, stop, and restart camera nodes. The AIA Core URCapX implements an automated discovery and registration mechanism to seamlessly integrate camera nodes into the system.

As illustrated in Fig. 2CC, the registration process follows these steps:

1. Node Initialization: Upon startup, the Camera Node compiles its YAML configuration file.

2. Configuration Transmission: The compiled configuration is published as a (base64) std_msgs/msg/String message to a designated ROS topic (<camera_name>/model_config).

3. Discovery Service: The AIA Core URCapX backend continuously monitors for topics matching this specific message format and automatically subscribes to them.

4. Integration: Once the YAML configuration is received and processed, the camera is registered and integrated into the system.

Following successful registration, the camera becomes fully integrated, enabling calibration procedures and settings adjustments through the URCap interface, as well as application integration with AI-based features provided by Pandai ARK.

Warning

Beware the formatting. Please refer to Camera settings for detailed information on parsing, encoding, and decoding YAML configuration files.

[Minimal example] | NodeControl client:

The following code demonstrates how to implement the Node Control Service client:

from camera_node_backend.srv import NodeControl
import json
import rclpy

# Initialize the service client
self.client = self.create_client(NodeControl, 'node_control_service')
while not self.client.wait_for_service(timeout_sec=5.0):
    self.get_logger().info('Waiting for Node Control Service...')

# Convert parameter dict to JSON string for easy parsing on service side
param_json = json.dumps(parameters)

# Create request - send JSON as single parameter
request = NodeControl.Request()
request.action = "start"  # Options: "start", "stop", or "restart"
request.node_name = camera_node_name
request.node_type = 'camera'
request.parameters = [param_json] if parameters else []

# Call the service
future = self.client.call_async(request)
rclpy.spin_until_future_complete(self, future)
return future.result()

Topics and services

The Camera Node publishes and serves the following interfaces. {ns} is the robot namespace (see below).

Interface	Kind	Direction	Purpose
{ns}_camera/model_config	Topic (std_msgs/String, base64)	Camera Node → AIA Core	Registration advertisement for auto-discovery
{ns}_camera/color/image_raw	Topic (sensor_msgs/Image)	Camera Node → ARK	RGB stream for perception
{ns}_camera/depth/image_raw	Topic (sensor_msgs/Image)	Camera Node → ARK	Depth stream for pose estimation
{ns}_camera/color/camera_info	Topic (sensor_msgs/CameraInfo)	Camera Node → ARK / calibration	Intrinsics
control_node_{name}_camera_service	Service (NodeControl)	AIA Core → Camera Node	Start / stop / restart

Namespace convention

All AIA stack nodes share a namespace derived from the robot serial, and the camera name is derived from the ROS 2 namespace. For robot serial UR20185300059 the camera prefix is /UR20185300059_camera, so model_config becomes /UR20185300059_camera/model_config. All nodes in the stack must share the same ROS_DOMAIN_ID.

Camera Settings

Camera configurations are represented in YAML format and are based on the settings exposed by the camera hardware and software. The ORRBEC Gemini 335Lg camera provides comprehensive configuration options, which can be found in the official launch file. For the ORRBEC camera all settings have default values, eliminating the need for manual configuration. We have fine-tuned specific parameters and prepared an optimized configuration file that serves as the starting point for the Camera Creator. However, you can customize these settings according to your requirements or integrate your own camera drivers.

Required YAML Configuration Keys

The following topic namespaces must be supported by the Camera Node to ensure full compatibility with the AI Accelerator pipeline:

# Camera identification
camera_name: "ORRBEC_Camera_V_0_0_1"

# Registration topic for AIA Core communication
aia_core_registration_topic: "ORRBEC_Camera_V_0_0_1/model_config"

# Hardware connection specification
connection_channel: "GMSL Channel ORRBEC"

# Data streaming topic definitions
image_camera_info_topic: "ORRBEC_Camera_V_0_0_1/color/camera_info"
image_topic: "ORRBEC_Camera_V_0_0_1/camera/color/image_raw"
image_compressed_topic: "ORRBEC_Camera_V_0_0_1/color/image_raw/compressed"
depth_camera_info_topic: "ORRBEC_Camera_V_0_0_1/depth/camera_info"
depth_topic: "ORRBEC_Camera_V_0_0_1/color/image_raw/depth"
depth_compressed_topic: "ORRBEC_Camera_V_0_0_1/color/image_raw/depth/compressed"

YAML Configuration Formatting

Configuration settings defined in the YAML file are exposed through the user interface, allowing users to modify values dynamically. By organizing configurations into logical categories (Device, Color, Infrared, Depth, and Point Cloud), you can control which sections the AIA Core URCapX displays in the configuration interface.

Example category structure:

# Color settings category
toggle_color_auto_exposure: false
limit_color_exposure_time: "100:1,1000:1"
select_color_resolution: "1080p:720p,1080p,4K"
..

When camera settings are updated, the AIA Core Manager Client automatically restarts the Camera Node to apply the new configuration.

Configuration Parameter Types

The system supports four parameter types, each with specific formatting requirements:

• static: Hidden settings that are locked to specified values.

• toggle: Boolean settings displayed as checkboxes (true/false).

• select: Drop-down menu selections with predefined options.

• limit: Numeric settings displayed as sliders with min/max/step values.

Example configuration with all parameter types:

# Static settings (prefixed with "static_")
static_device_preset: "High Accuracy"

# Toggle settings (prefixed with "toggle_")
toggle_depth_registration: true

# Select settings (prefixed with "select_")
# Format: "<current_value>:<option_0>,<option_1>,<option_N>"
select_laser_on_off_mode: "0:-1,0,1"

# Limit settings (prefixed with "limit_")
# Format: "<current_value>:<min_value>,<max_value>:<step>"
limit_angular_vel_cov: "0.01:0.01,6.2:0.01"

The default configuration file can be overwritten in the image:

# Use the existing camera_creator image as base
FROM camera-node-urcap-backend:latest

# Set working directory
WORKDIR /workspaces/orbbec-camera

# Copy custom configuration file
# You can mount this at build time or runtime
COPY custom_camera_config.yml /workspaces/orbbec-camera/camera_config.yml

Launch Drivers and Files

The Camera Creator framework provides a flexible Docker-based deployment system that allows integration of custom camera drivers and launch configurations. This system enables you to replace the default ORRBEC driver with manufacturer-specific drivers while maintaining compatibility with the AI Accelerator pipeline.

Driver replacement

The Dockerfile supports driver replacement through build arguments, allowing seamless integration of third-party camera drivers. The system automatically handles driver installation, launch file configuration, and ROS2 package rebuilding to ensure proper functionality.

Build Process

Use the following build process to utilize other driver software:

FROM camera-node-urcap-backend:latest

# Build arguments for custom driver integration
ARG MANUFACTURER_DRIVER_URL
ARG CUSTOM_LAUNCH_FILE

WORKDIR /workspaces/orbbec-camera

# Replace default driver with manufacturer's driver
RUN if [ -n "$MANUFACTURER_DRIVER_URL" ]; then \
        echo "Installing manufacturer driver from: $MANUFACTURER_DRIVER_URL" && \
        rm -rf src/OrbbecSDK_ROS2 && \
        git clone "$MANUFACTURER_DRIVER_URL" src/OrbbecSDK_ROS2 && \
        cd src/OrbbecSDK_ROS2 && \
        git lfs pull && \
        cd /workspaces/orbbec-camera; \
    fi

# Replace default launch file with manufacturer's custom launch
RUN if [ -n "$CUSTOM_LAUNCH_FILE" ] && [ -f "$CUSTOM_LAUNCH_FILE" ]; then \
        echo "Installing manufacturer launch file: $CUSTOM_LAUNCH_FILE" && \
        cp "$CUSTOM_LAUNCH_FILE" src/camera_node_backend/launch/camera.launch.py; \
    elif [ -f "manufacturer_camera.launch.py" ]; then \
        echo "Installing default manufacturer launch file" && \
        cp manufacturer_camera.launch.py src/camera_node_backend/launch/camera.launch.py; \
    else \
        echo "No custom launch file provided, keeping default"; \
    fi

# Rebuild ROS2 packages with new driver
RUN . /opt/ros/humble/setup.sh && \
    rm -rf build/ install/ log/ && \
    colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release

ENTRYPOINT ["/workspaces/orbbec-camera/entrypoint.sh"] 

The build process performs the following operations:

1. Driver Replacement: Clones and installs custom camera drivers from specified Git repositories.

2. Launch File Configuration: Integrates custom launch files or uses manufacturer-provided defaults.

3. Package Rebuilding: Rebuilds ROS2 packages with the new driver to ensure compatibility.

4. Environment Setup: Configures the runtime environment for optimal performance.

Use the following command to build a custom camera image with your specific driver and launch file:

docker build -f your.Dockerfile \
    --build-arg MANUFACTURER_DRIVER_URL="https://git.your.camera.drive.url" \
    --build-arg CUSTOM_LAUNCH_FILE="your_camera.launch.py" \
    -t visiontech-camera:latest .

Further Resources

Troubleshooting

Common issues and solutions when working with the Camera Creator:

• Camera not detected: Ensure the camera is properly connected and drivers are installed

• Configuration errors: Verify YAML syntax and required parameter formats

• ROS2 communication issues: Check network connectivity and topic availability

• Docker build failures: Verify build arguments and ensure all dependencies are available

Integration Examples

For specific integration scenarios, refer to:

• AI Accelerator Core — platform overview and how the components connect

• Orbbec Viewer — install the camera GUI on the compute box

• URScript API — robot programming interface

• SDK — install the SDK and enter the container