Skip to main content

Overview

This subsystem is still in experimentation phase and WILL change.

The autonomous navigation of the rover is supported by the use of an Intel RealSense D435i depth camera. This sensor provides synchronized RGB, depth, and inertial measurements (IMU), which are essential for perception and mapping tasks.

The software interface for the camera is provided through the realsense-ros ROS2 wrapper package realsenseai/realsense-ros: ROS Wrapper for RealSense™ Cameras, which enables integration of the camera with the ROS 2 ecosystem. This repository has been cloned into the workspace and is included as a package. Additionally, the librealsense SDK realsenseai/librealsense: RealSense SDK, which provides the low-level drivers and APIs for the camera—is also included as a separate package.

The primary package responsible for SLAM (Simultaneous Localization and Mapping) and map generation is rtt_slam. Within this package, all required dependencies are declared in the package.xml file, ensuring proper integration with the rest of the ROS 2 system.

The process takes the RGB and depth frames along with the IMU data and produces an STVL (Spatio-Temporal Voxel Layer) map as the output Using an External Costmap Plugin (STVL) — Nav2 1.0.0 documentation. The Nav2 framework is utilized to produce the map and for autonomous navigation.

To launch SLAM:

ros2 launch rtt_slam slam.launch.py

The launch file starts:

  1. Realsense camera drivers
  2. RTAB-Map odometry and SLAM
  3. IMU filtering
  4. Navigation2 (Nav2)
  5. Costmap Visualization utilities

Launch File Documentation

The launch file defines and orchestrates all nodes and processes required for perception, SLAM, and data streaming. It ensures proper initialization order and parameter configuration.

Database Reset:

ExecuteProcess(
    cmd=['rm', '-r', os.path.expanduser('~/.ros/rtabmap.db')],
    output='screen',
)

Before starting SLAM, any previously stored map database is deleted. This ensures that mapping starts from a clean state and avoids conflicts with prior runs.

Realsense Camera Launch:

The launch file is included with specific parameters:

  • IMU enabled (enable_gyro, enable_accel)
  • IMU fusion (unite_imu_method=2)
  • Depth aligned to RGB (align_depth.enable=True)
  • RGB-D output enabled
  • Point cloud generation enabled
  • Synchronization enabled (enable_sync=True)
  • Matching RGB and depth resolutions (424×240 @ 30Hz)

This configuration ensures consistent and synchronized sensor data for downstream processing.

IMU Filtering:

Node(
    package='imu_filter_madgwick',
    executable='imu_filter_madgwick_node',
)

The imu_filter_madgwick ROS package node processes raw IMU data:

  • Converts raw IMU data into a filtered orientation estimate
  • Uses the Madgwick filter algorithm
  • Magnetometer is disabled (use_mag=False)
  • Outputs orientation in the ENU (East-North-Up) frame

Topic remapping ensures compatibility with RealSense IMU topics.

RTAB-MAP SLAM

The SLAM system is launchedcomprised via:of four launch files:

1. camera_launch.yaml

Responsible for starting the depth camera.

The launch file:

  • rtabmapLoads RealSense configuration from config/camera_realsense.yaml
  • Starts the RealSense camera driver

Key configurations:Configuration:

  • SubscribesEnables the gyroscope, accelerometer, IMU synchronization, and point cloud generation.
  • Aligns depth images to the RGB camera.
  • Reduces image resolution for improved stability and performance.


2. rtabmap_launch.yaml

Responsible for localization and map generation.

The launch file:

  • Loads RTAB-Map configuration from config/rtabmap.yaml
  • Remaps camera RGB, depth, and camera info, and IMUinfo topics
  • UsesStarts approximateRGB-D timeodometry
  • Starts synchronizationRTAB-Map (approx_sync=True)SLAM
  • Starts the Madgwick IMU filter
  • SetsPublishes a static transform between base_link asand camera_link

Configuration:

  • Uses synchronized RGB and depth images from the robotRealSense reference framecamera.
  • EnablesUses filtered IMU orientation estimates for odometry.
  • Deletes any existing RTAB-Map database on startup.
  • Optionally launches the RTAB-Map visualization tool.


3. navigation_launch.yaml

Responsible for autonomous navigation and costmap generation.

The launch file:

  • Loads navigation parameters from config/navigation.yaml
  • Starts the Nav2 navigation stack
  • Starts costmap visualization nodes

Configuration:

  • Uses the RTAB-Map occupancy grid generationas the global map.
  • Uses a voxel-based local costmap generated from depthRealSense datapoint clouds.
  • Enables obstacle marking and clearing.
  • Uses the DWB local planner for path following.

Important

grid

4. parameters:slam_launch.yaml

Responsible for launching the complete SLAM system.

The launch file:

  • Launches Grid/RangeMaxcamera_launch.yaml: 2.0 m (maximum sensing range)
  • Launches Grid/RangeMinrtabmap_launch.yaml: 0.05 m (minimum sensing range)
  • Launches Grid/CellSizenavigation_launch.yaml: 0.02 m (map resolution)
  • Voxel filtering enabled

This setup enables real-time 3D mapping and localization.

Static Transform

Node(
    package='tf2_ros',
    executable='static_transform_publisher',
)

A static transform is published between:Configuration:

  • base_linkCombines (robotperception, frame)localization, mapping, and navigation into a single launch command.
  • camera_linkEnsures (cameraall frame)required subsystems are started together.

This defines the spatial relationship between the robot and the sensor. This is necessary as the default TF-tree is not compatible with Nav2.


Video Streaming to Basestation

ExecuteProcess(
    cmd=['/bin/bash', os.path.expanduser('~/stream.sh')],
)

A separate script (stream.sh) is executed to stream camera data to the base station.


Streaming Script Documentation

This script uses GStreamer to transmit video over UDP.

Paragraph Breakdown:

rosimagesrc ros-topic="/camera/camera/color/image_raw"

Captures image directly from a ROS topic.


videoconvert ! video/x-raw,format=I420

Converts image format for encoder compatibility


x264enc tune=zerolatency speed-preset=superfast

Encodes video using H.264 with low-latency settings


rtph264pay pt=96

Packages encoded video into RTP packets


udpsink host=145.126.xx.XXX port=4500

Sends the stream to the base station via UDP


RViz2

RViz2 is used as the primary visualization interface for the rover system. It allows real-time monitoring of sensor data such as RGB images, depth maps, point clouds, and TF frames. Additionally, it provides tools to visualize occupancy grids and SLAM outputs, aiding in debugging and system validation. It is used to test the implementation of the mapping and the working of the camera.




Back to top