Dynamic Picking

Warning

The AI Accelerator requires ROS 2 Humble, so PolyScope X 10.12.1 is the latest compatible release. PolyScope X 10.13 and later use ROS 2 Jazzy and are not currently supported.

Dynamic picking combines detection, FoundationPose, and cuMotion path planning so the robot can pick a part whose pose is not known in advance. Complete Pose Estimation (detector + textured mesh) and review Path Planning Basic first.

What you need

  • Trained detector checkpoint under data/models/rtdetr_active/

  • Textured OBJ (+ JPG/PNG): texture, Y-up, real-world dimensions — see Pose Estimation

  • Collision model aligned with the scene (collision_basic.yaml)

  • Example program: ~/aia_sdk/polyscope/programs/aia_example_dynamic_picking.urpx

  • Gripper activate / open / close nodes filled in for your gripper URCap

Steps

Step 1 — Enter the SDK container

cd ~/aia_sdk/ros
./scripts/run_sdk_setup.sh

Step 2 — Point the launch file at your mesh and collision model

Confirm the dynamic picking launch references your textured mesh and that config.yaml / collision YAML match the physical box and tools.

Step 3 — Launch dynamic picking

./scripts/run_example_dynamic_picking.sh

If you edit the collision YAML, RViz updates live but you must restart this script for cuMotion to use the new geometry.

Step 4 — Set joint limits

Warning

Set joint angle limits as described in Path Planning Advanced. Unrestricted joints can cause large motions and tangle the camera cable.

Step 5 — Load the PolyScope program and checkpoint

  1. Load aia_example_dynamic_picking.urpx from ~/aia_sdk/polyscope/programs/.

  2. In Before Start, load the correct detection checkpoint (same pattern as pose estimation).

  3. This example uses frames (not waypoints): capture, pick / pick_approach, place / place_approach, and part.

Dynamic picking teach positions

Step 6 — Teach frames and run

  1. Teach capture on one side of the accessory box (fixed imaging pose).

  2. Set speed to 10%, play, and wait for pose estimation. Confirm poses in globals and optionally check RViz.

  3. Stop, then teach pick relative to the part frame without nudging the part. pick_approach is already above the part.

Teach pick relative to part

  1. Run again so the robot picks and retracts to pick_approach. Stop with the part in hand and teach a fixed place (preferably on the other side of the box). place_approach is already defined above place.

  2. Run the full loop at low speed first. The program estimates pose, path-plans around the workspace, picks, and places, then repeats so you can return the part to the capture area.

Dynamic planning is used to the approach points; a fixed short move is used into contact for pick/place to reduce protective stops.