Robot Vision and Depth Sensors

The main ways robots see the world and judge distance, from a single camera to two-camera stereo, depth cameras and lidar. Each type comes with its strengths, its weaknesses and example robots that use it. Ultrasonic, infrared, radar and other simple sensors are covered too, along with how robot vision compares with animal eyes.

Sensor comparison

SensorHow it measures distanceTypical rangeWorks in the darkSensor cost
Single cameraDoesn't measure it directly. Software estimates depth from motion, known sizes or a trained AI model.No measured range. It sees far, but distances are only estimates.NoA few dollars to tens of dollars
Stereo camerasCompares the images from two cameras side by side, like a pair of eyesBest within a few meters, wider camera spacing reaches fartherNoTens to hundreds of dollars
Active stereo and structured lightProjects an infrared pattern and measures how it lands on surfacesAbout 0.2–4 m on small robot camerasYesHundreds of dollars
Time-of-flightTimes how long a pulse of light takes to bounce backCentimeters to several metersYesA few dollars (single point) to hundreds (3D camera)
LidarSweeps a laser and times each reflectionAbout 10 m on hobby units, 100 m or more on car lidarsYesUnder $100 (2D) to thousands (3D)
UltrasonicTimes the echo of a sound pulse, like a batA few centimeters to about 4 mYesA few dollars

Many robots combine several of these. A modern robot vacuum may carry lidar for mapping, a structured-light sensor for low obstacles, a camera to recognize cables and socks, and cliff sensors to avoid stairs.

Single camera

One camera is the cheapest and most common way for a robot to see. It gives a flat color image, which is ideal for spotting faces, objects, colors, markers and lines. On its own it can't tell whether an object is small and close or large and far away.

Software fills the gap in a few ways. As the robot moves, nearby things shift across the image faster than distant ones, and visual SLAM uses that shift to build a 3D map. A known object size, such as a charging dock or a face, also gives its distance. Newer AI models estimate a full depth map from a single photo, much as a person can judge depth with one eye closed.

Strengths

  • Cheap, small and low power
  • Rich detail, including color, text and faces
  • Huge range of free vision software and AI models

Weaknesses

  • No direct distance reading
  • Fails in the dark unless paired with an infrared light
  • Depth from motion or AI needs a fair amount of computing

Vector uses one wide-angle camera for faces and its charger marker, plus a single-point distance sensor in front. Hobby kits like the PiCar-X and JetBot also rely on one camera, often combined with an ultrasonic sensor.

Vector 2.0$249.99NVIDIA JetBot$399.99 – $499.99…PiCar-X$89.99 (no Pi)Zumi$179.99CozmoWas $179.99 (2016)Asimov 1$20,000 kit

Stereo cameras

A stereo camera uses two cameras a fixed distance apart, the same way two eyes work. An object appears in a slightly different spot in each image. The closer it is, the bigger that shift, so the robot can calculate distance for every point it can match between the two images.

Accuracy falls off quickly with distance. In theory, doubling the distance makes the depth error about four times larger. Spacing the cameras farther apart helps, which is why stereo heads on large robots are wider than the tiny modules on small ones. Some newer robots use AI models trained to match the two images, which copes better with plain surfaces than older matching methods.

Strengths

  • Real 3D depth from ordinary cameras, with no laser or light emitted
  • Works outdoors in bright sun, where infrared sensors struggle
  • Color and depth come from the same images
  • No moving parts, and closest to how animals see

Weaknesses

  • Needs visible texture, so blank walls, glass and mirrors give poor depth
  • Needs light, like any camera
  • Accuracy drops fast beyond a few meters
  • The two cameras must stay precisely aligned, and matching takes computing power

Spot carries five stereo camera pairs around its body and treats lidar as an add-on. The NEO humanoid and the Walker S2 use stereo cameras in the head, with no lidar. Walker S2 uses plain color cameras with learned stereo depth. MiRo-E has stereo cameras in its eyes. Drones such as the Skydio R1 and Parrot ANAFI Ai avoid obstacles mostly with stereo pairs, since lidar is heavy.

Spot~$75,000+NEO$20,000 or $499/moWalker S2$246,600 (reseller)MiRo-EQuote onlySkydio R1Was $2,499 (2018)ToddlerBot 2.0~$6,000 DIY · $4,299 built

Active stereo and structured light

Most depth cameras sold for robots add their own light. Active stereo cameras, such as most Intel RealSense models and the OAK-D Pro, are stereo pairs with an infrared projector. The projector sprays thousands of invisible dots onto the scene, so even a blank wall has texture to match. Structured light sensors project a known pattern or line and measure how it bends over objects. Robot vacuums use this on the front bumper to spot shoes and cables.

Strengths

  • Dense, reliable depth on plain surfaces
  • Works in the dark
  • Ready-made cameras with ROS 2 drivers are easy to add

Weaknesses

  • Short range, often about 0.2–4 m on small robots
  • Sunlight drowns out the infrared pattern outdoors
  • Larger and more power-hungry than a plain camera

The TurtleBot 4 pairs an OAK-D Pro depth camera with a 2D lidar. Stretch has RealSense cameras in its head and gripper, and the Unitree H1 combines a RealSense with a 3D lidar. The MentorPi T1 kit uses a structured-light depth camera, and the Roborock Qrevo vacuums use structured light for obstacles.

TurtleBot 4 / 4 Lite$1,572 / $2,575Hello Robot Stretch$24,950 (Stretch 3) /…H1 / H1-2~$90,000+MentorPi T1$329.99 – $909.99Roborock Qrevo$385 – $900

Time-of-flight sensors

Time-of-flight (ToF) sensors send out light and measure how long it takes to come back. The simplest ones read a single distance, like a laser tape measure the size of a fingernail. 3D ToF cameras measure a whole grid of distances at once, giving a low-resolution depth image.

Strengths

  • Measures distance directly, with almost no computing
  • Tiny, and works in the dark
  • Single-point sensors cost only a few dollars

Weaknesses

  • 3D ToF cameras have low resolution compared with normal cameras
  • Strong sunlight reduces range
  • Shiny, black or see-through surfaces can give false readings

Loona pairs a camera with a 3D ToF sensor. aibo has a ToF sensor alongside its nose and mapping cameras. The Roborock Saros Z70 vacuum uses 3D ToF to guide its arm, and Vector uses a single-point ToF sensor to avoid walls.

Loona Petbot$499aibo$3,199.99Roborock Saros Z70$950 – $2,599

Lidar

Lidar measures distance with laser pulses in many directions. A 2D lidar spins a single beam to scan one flat slice of the room, usually at about knee height or lower. It is the standard sensor for SLAM mapping on robot vacuums and ROS robots. A 3D lidar scans many angles at once to build a cloud of points around the robot. That lets walking robots see stairs, curbs and overhangs.

Strengths

  • Accurate to about a centimeter, and the error barely grows with distance
  • Works in any lighting, including total darkness
  • Mapping software is mature and needs little computing

Weaknesses

  • A 2D lidar sees only one slice, so it can miss table tops and objects on the floor
  • No color, so it can't tell what an object is
  • Glass, mirrors and black surfaces can confuse it
  • Spinning parts wear out, and 3D units cost far more

The Roomba 105 and most current robot vacuums map with a small spinning lidar. The TurtleBot 4, Stretch and MentorPi M1 carry 2D lidars for ROS 2 navigation. The Go2 robot dog has a 3D lidar built into the front of its body, and the Unitree G1 humanoid has one too.

iRobot Roomba 105$280 – $470MentorPi M1$430 – $880Go2 Air / Pro / X$1,600 / $2,800 / $4,500G1$13,500 – $73,900

Other distance sensors

Ultrasonic sensors

Ultrasonic sensors send a chirp of sound too high to hear and time the echo, just as bats do. The two round "eyes" on many kit robots are the speaker and microphone of an ultrasonic sensor. They are cheap and simple, and they detect glass that fools light-based sensors. Their beam is wide and fuzzy, though, so they report the nearest object somewhere ahead but not exactly where. Soft fabric absorbs the sound.

mBot$69.99 – $149.99Otto DIY / HP Robots Otto$46 – $260Hexy the Hexapod$250

Infrared, cliff and bump sensors

Simple infrared sensors shine a light and check how much reflects back. They handle line following and nearby obstacles. Pointed down, they work as cliff sensors that stop a robot driving off a stair or table edge. Bump switches are the last line of defense, detecting a collision after it happens. The iRobot Create 3 has no camera at all and gets around with these sensors alone.

iRobot Create 3Was ~$299–$449Zumi$179.99

Radar

Radar uses radio waves instead of light. It sees through fog, dust, rain and smoke, and it measures the speed of moving objects directly. Cars use it for adaptive cruise control. Its resolution is coarse, so robots use it mainly as a backup sensor outdoors or in harsh places. It is rare on consumer robots.

Thermal cameras

Thermal cameras see heat rather than light, so people and animals stand out even in total darkness or smoke. Security and inspection robots carry them to find people, hot motors and leaks. The Sarcos Guardian S snake robot offers one as an option.

Event cameras

Event cameras report only the pixels that change, the instant they change, rather than whole frames. That makes them very fast and good in difficult lighting. They are mostly used in research on fast drones and tracking.

Robot vision compared with animal eyes

Animals solved the depth problem long before robots, and robot sensors often mirror their tricks.

Lidar has no real equivalent in nature. It became popular because it gives reliable distances with little computing. As AI vision improves, more robots are moving toward cameras alone. Examples include the Walker S2 with learned stereo depth and the Optimus humanoid with car-style camera vision. A camera-only robot still struggles in darkness and on blank surfaces, so most commercial robots keep at least one active sensor.

Sensor choice by goal

The Sensors section of Parts of a Robot covers IMUs, encoders and other sensors that tell a robot about its own body.