Classification

A commonly utilized robot manufacturing process is in operation inspections. The robot uses a camera on the part it is inspecting. Machine learning determines whether the part meets quality specifications. The camera is used to inspect parts for a single feature following a data capture and training. This demo shows how to automate a quality inspection application.

The objective of this demo is to show a two-state classifier. There are only two possible outcomes of this recognition.

Step by step

The example should be run from inside the Jetson container which is started by running run_sdk_setup.sh as described in the SDK article.

Warning

The Region of Interest (ROI) is very important in classification. All collected images will be cropped to this predefined area before inference so it is critical it is set correctly before training. As you will see below, you collect images and train on the GUI but the robot first must be in the desired position for later testing inference via a robot program. The ROI is configurable for training on the GUI in ros/config/config.yaml, but when running the robot program it is redefined under the script node “local roi_query”. Please keep this in mind so that inference works both on the GUI and on the robot.

  1. Move the robot in to position from which the object recognition would eventually be performed.

  2. On the Compute module start the required ROS2 nodes: ./scripts/run_example_classification.sh.

  3. On the Compute module use the GUI. The blue bounding box (configurable in ros/config/config.yaml) is the region of interest.

  4. Start recording images for the state_0

  5. Move the object you want to recognize within the bounding box of the camera view.

  6. Take at least 20-30 images varying the object position and if desired, the object orientation. Images are stored in ros/data/datasets/classification_active/raw

  7. Record images of a second object, as the state_1

  8. Click the Train model, this process can take several minutes.

You can follow the progress in the Terminal window on the compute module.

Training is complete when the message “onnx conversion completed” appears in Terminal and the model is written as a file in ros/data/models/classification_active

To use the model

  1. Load the model. In the UI, tap the Load model. This loads the active model from ros/data/models/classification_active

  2. Tap Classify to test.

  3. The Terminal window outputs the class (either state_0 or state_1) and probability (recognition certainty).

Example of using recognition results

Warning

A specific robot position stored in this program. Before executing this program check that robot can freely move to each of the stored waypoints and poses no risks. Please remember that wp_detect should match the position the robot was in when you were training your model (see ROI note above). The waypoints are fixed markers to physically denote the decision of the robot and in no way is any pose estimation being used.

  1. Set program speed to 10%.

  2. Select wp_detect and tap Move Here.

  3. Verify that there are no obstructions.

  4. Repeat these steps for state_0 and state_1.

  5. If necessary, Freedrive robot to a new position and save it as detect_wp.

Included with the AI Accelerator SDK you can find example of a robot program using the recognition results.

  1. On the robot open aia_example_classify program, that was copied from the sdk directory on the compute module.

  2. Run ./scripts/run_example_classification.sh on the compute module.

  3. Load the classification model.

  4. The robot program uses three waypoints wp_recognize, state_0 and state_1.

  5. Recognize waypoint is where camera looks at the object for recognition. Then we conditionally move robot to either state_0 waypoint or state_1 waypoint depending on the recognition results.

The value of variable detected_class is assigned by function “ark_classification_retrieve(). You can see details of ROS communication in URScript code.

Improving Classification Accuracy

If the trained model does not perform well during inference, try the following:

  • Capture more diverse training data—record additional images for each state from multiple angles, different lighting conditions, and varying distances from the ROI. Having broader coverage of how the scene can look reduces overfitting to a narrow perspective.

  • If the current model architecture still struggles, consider switching to a different classification model. That usually requires updating the training source code or model definition and rebuilding the package, so plan that effort carefully before making large changes.