Computer Vision

Overview

The computer vision subsystem of the rover is utilized in the Navigation, Surface Sampling and Maintenance Tasks of ERC. It is accomplished through the use of the rover cameras: Gripper camera (digital Arducam), Bottom chassis camera (digital Arducam), Front chassis camera (Depth). The subsystem can be further divided in two sections, ArUco Detection and Object detection, both aiding in different parts of the ERC competition.

ArUco Detection and Pose Estimation

During the navigation task, there will be some landmarks of note that the rover can use to establish its position relative to the Mars Yard. These landmarks will be identified through fixed wooden poles that display an ArUco tag to identify them. The ArUco tags can be thought of as special QR codes that embed identifying numbers within their pixel arrangement. Read more about ArUco tags in the first paragraphs of this OpenCV documentation page: https://docs.opencv.org/4.8.0/d5/dae/tutorial_aruco_detection.html

Thanks to a combination of the OpenCV library and the ROS2 TF2 framework, it is possible to extract the coordinates of these ArUco tags by using any type of camera. The above link for ArUco detection is a great starting point and much of its implementation has been reflected into the rover's code as well. For information regarding TF2, it is advisable to read the ROS2 documentation: https://docs.ros.org/en/humble/Tutorials/Intermediate/Tf2/Introduction-To-Tf2.html

The obtained coordinates from the ArUco Detection and Pose Estimation process through the 'aruco_track' will be then passed to the behavior node of the ROS2 architecture where they will become input data for appropriate localization and Odometry calibration. Therefore, these coordinates will pass through the nodes Behavior, Planner, Smoother, Controller within the Jetson to determine an optimal navigation path. Once the path is determined, the appropriate driving commands will be passed through a published topic onto the Communications node and handed down into the appropriate hardware boards. This will be accomplished with the use of Protobufs .

Object Detection and Pose Estimation