Visual-inertial odometry and mapping

Cameras are cheap. A platform that navigates with nothing but cameras and an IMU can be very cheap too. It is also passive: no active sensing is required, so the platform sends out no laser beams and no radio signals.

This video shows how far that takes you, on a boat. Our software fuses three cameras and an IMU to follow the motion of the boat over time. That is visual-inertial odometry. Along the way, it builds a 3D map of the surroundings. The cameras are:

  • A front-facing stereo camera
  • A front-facing infrared camera
  • A 360° omnidirectional camera

The infrared camera does not depend on daylight. With it, the system delivers the same results at night.

Three cameras is only this setup. The system works with any number of cameras, and it is a complete odometry and place recognition system: it follows the motion, and it recognizes a place that it has seen before.

Watch the dots in the camera images. The green dots are features: points that the software can find again in the next image, such as corners. The pink dots are landmarks, the features that hold up over time. The software uses them for mapping and localization.

The 3D point cloud in the video comes from vision alone. No lidar is involved.