Detection

Warning

The AI Accelerator requires ROS 2 Humble, so PolyScope X 10.12.1 is the latest compatible release. PolyScope X 10.13 and later use ROS 2 Jazzy and are not currently supported.

Load a detection model, configure inference, and retrieve 3D object poses. Requires ark_init() (see General).

Typical call order: ark_init() → ark_load_detection_model → ark_set_detection_params → ark_infer_detection.

ark_load_detection_model(model)

ark_load_detection_model(model)

Loads a detection model on the AI Accelerator.

Arguments

  • model (string): Model filename or path relative to /workspaces/isaac_ros-dev/data/models/ (that prefix is prepended automatically)

Returns

  • struct(error=ArkError(...)) — error mirrors res.error_state.error from the load service

Errors / failure modes

  • Missing model file or load failure sets error.error from the service

  • Creates a one-shot /load_detection_model client (not the long-lived clients from ark_init)

ROS

  • Service: ARK_ROS2Namespace + "/load_detection_model"

  • Type: ur_ark_ros_common_msgs/srv/LoadModel

  • Load request uses full image ROI struct(x=0., y=0., width=480., height=640.) and min_confidence=1.0 (confidence for inference is set later via ark_set_detection_params)

Call order

Usually after ark_init(), before ark_set_detection_params / ark_infer_detection.

Example

ark_init()
load_res = ark_load_detection_model("detection_active/model.onnx")
if load_res.error.error:
  popup("Failed to load detection model", blocking=True)
end

ark_set_detection_params(labels=[], min_confidence=0.3, context="on_surface", context_model="table")

ark_set_detection_params(labels=[], min_confidence=0.3, context="on_surface", context_model="table")

Configures detection confidence, label filter, and how 3D pose is estimated. Triggers the detection pipeline with these query parameters.

Arguments

  • labels (string list, default []): Labels to return; empty list means all labels

  • min_confidence (float, default 0.3): Minimum detection confidence

  • context (string, default "on_surface"): 3D pose estimation mode

    • "on_surface": Infer 3D pose from 2D location, camera geometry, and a surface constraint (context_model)

    • "fp": Estimate 3D pose with FoundationPose (context_model not used)

  • context_model (string, default "table"): Surface frame / model name when context == "on_surface"

Returns

  • void

Errors / failure modes

  • Requires ark_init() (uses the /trigger_detection client)

  • Invalid context may yield empty or failed subsequent inferences

ROS

  • Service: ARK_ROS2Namespace + "/trigger_detection"

  • Type: ur_ark_ros_common_msgs/srv/TriggerDetection

  • Request fields: query=context, model=context_model, label_list=labels, confidence=min_confidence

Call order

After loading a model; before or as part of each detection cycle before ark_infer_detection.

Example

ark_set_detection_params(["part"], 0.5, "on_surface", "table")

ark_infer_detection()

ark_infer_detection()

Blocks until detections are available (or the wait times out), then returns poses, frame IDs, and confidences.

Arguments

  • None

Returns

  • struct(error=ArkError(False), poses=<list>, frame_ids=<list>, confidences=<list>)

    • Up to 50 detections

    • poses: list of poses

    • frame_ids: TF frame id string per detection (for later TF / planning use)

    • confidences: float confidence per detection

Errors / failure modes

  • Blocking wait uses a 20000 ms timeout on /wait_detection

  • Preamble currently always returns error=ArkError(False) after the wait; treat empty lists as “no detections”

  • Requires a prior ark_set_detection_params (trigger) so the wait has something to retrieve

ROS

  • Service: ARK_ROS2Namespace + "/wait_detection"

  • Type: ur_ark_ros_common_msgs/srv/WaitDetection

Call order

ark_init() → load model → ark_set_detection_params → ark_infer_detection.

Example

ark_init()
ark_load_detection_model("detection_active/model.onnx")
ark_set_overlay("detection")
ark_set_detection_params([], 0.3, "on_surface", "table")
det = ark_infer_detection()
i = 0
while i < length(det.poses):
  textmsg(det.frame_ids[i] + " conf=" + to_str(det.confidences[i]))
  i = i + 1
end