Skip to content
Merry'n'Pippin
Go back

Behold and see

A robot arm can know the position of every joint and still be completely unaware of the marker sitting in front of its gripper.

After assembling and calibrating my LeRobot SO-101 leader-follower setup, the next step was to give the follower useful visual observations. I wanted it to see both the entire work area and the fine detail around the gripper. Those are two different jobs, so I did not try to force one camera to perform both of them.

The resulting system uses two complementary perspectives:

One camera setup answers, “What is happening in the workspace?” The other answers, “What is happening right here?”

Why a single camera was not enough

A fixed camera is excellent at preserving context. Because it stays in the same place, the table, robot base, task objects, and working boundaries remain visually stable. During data collection, it can observe where an object starts, how the arm approaches it, and where it ends up.

Its weakness is distance. As the gripper moves around the table, a small object occupies only part of the image. The arm may also move between the camera and the point of interaction. Details that matter during grasping - the gap between the fingers, the edge of an object, or the alignment above a target - can become difficult to see.

A wrist camera has the opposite personality. It stays close to the action and follows the gripper, so the object can remain prominent while the arm changes pose. But because the entire image moves with the wrist, it is a less stable description of the wider scene.

Combining the two gives the learning system both context and proximity.

The table view: Logitech Brio 100

For the external view, I bought a set of Logitech Brio 100 webcams. Logitech specifies Full HD 1080p capture and a fixed 58.9-degree diagonal field of view. In my LeRobot configuration, I run the cameras at 640 x 480 and 30 FPS, which keeps the observation stream consistent with the rest of the pipeline.

The webcams are attached to a clamp-mounted NEEWER camera stand. A clamp stand is convenient for a small robotics station: it avoids occupying more of the table and makes it possible to aim the camera down at the working area.

The original mounting geometry was not quite right for the position I wanted. I started from this printable Thingiverse camera mount, modified it, and printed my own version so the Brio would remain at the correct angle on the stand.

This was a small modification, but camera placement is part of the dataset. If the camera slowly sags, rotates between sessions, or is reattached at a different angle, the visual distribution changes even when the physical task remains identical. A rigid, repeatable mount is therefore not merely a convenience; it is part of keeping experiments consistent.

IMG_1581.jpeg

The fixed camera watches the entire workspace from a repeatable position above the table.

The close view: DFRobot FIT0701

For the gripper perspective, I chose the DFRobot FIT0701. It is a small, driver-free USB 2.0 camera module with a native resolution of 640 x 480. According to DFRobot’s specifications, the board measures approximately 30 x 25 mm, with a total module depth of 21.4 mm.

Those numbers are modest compared with a modern webcam, but they make sense on a moving robot arm. The camera needs to be light and compact, and the image will be captured close to the object. In this role, viewpoint can matter more than an impressive resolution printed on the box.

I printed TheRobotStudio’s optional SO-ARM101 wrist-camera mount and installed the module above the gripper. The orange mount continues the follower’s color scheme while holding the lens forward, where it can watch the fingers and the region immediately ahead of them.

IMG_1205.jpeg

The compact FIT0701 in its printed orange wrist mount, looking over the SO-101 gripper.

The finished installation makes the camera look a little like a head perched above the gripper. That is not entirely misleading: wherever the arm moves, this viewpoint moves with it.

Routing one more moving cable

Mounting the camera was only half the job. Unlike the servo leads, which travel relatively short distances between neighboring joints, the USB cable has to follow the moving arm back toward the computer.

It needs enough slack for the joints to reach their useful range without pulling on the camera connector. Too much slack, however, creates loops that can catch on the gripper, the table, or the arm itself. Cable routing therefore becomes a small kinematics problem of its own.

I routed the wrist-camera lead alongside the arm, leaving controlled loops near the moving joints. The photographs show the practical result: functional rather than invisible, and deliberately loose enough to move.

IMG_1209.jpeg

The complete follower arm with the wrist camera installed and its USB cable routed along the moving links.

This is an area I expect to keep refining. The mount must remain rigid, but the cable must remain flexible. Strain relief near the small camera connector is especially important because repeated wrist motion should bend the cable gradually rather than pull directly on the board.

From physical cameras to model inputs

The two viewpoints also need stable identities in software. In my LeRobot configuration, I use the logical names top and wrist:

top    -> fixed view of the table
wrist  -> moving view above the gripper

The physical OpenCV indices can change when Linux reorders USB video devices after a reboot. The meaning of the names should not. Before collecting demonstrations or running inference, I check which device belongs to each view and then assign the correct index to the correct logical camera.

This matters because an imitation-learning policy does not understand that camera index 0 was supposed to be the table view. It learns from feature names, image content, and the relationships present in the training data. Accidentally swapping top and wrist during inference would give the model two familiar images in unfamiliar roles.

IMG_1210.jpeg

Side view of the completed installation, showing the camera mount, cable path, arm, and controller arrangement.

Why 640 x 480 can be enough

Both cameras are configured to produce 640 x 480 frames at 30 FPS for the robot-learning workflow. That is below the Brio 100’s maximum resolution but exactly matches the FIT0701.

Higher resolution is not automatically more useful. It increases storage, video-encoding work, data-loading cost, and the amount of image information that must eventually be resized or processed by the policy. TheRobotStudio’s wrist-camera installation guidance likewise suggests 640 x 480 at 30 FPS as a practical setting for many model pipelines.

The more important questions are whether the object is visible, whether the view is stable, and whether the same semantic camera appears under the same name during collection and inference.

What I learned from adding vision

Adding cameras looked simple on the parts list: buy two camera types, print two kinds of mounts, and connect the cables. In practice, it revealed several useful rules:

Most importantly, the best camera is not necessarily the one with the largest number of pixels. It is the one that consistently sees the information the policy needs.

Two views, one task

The finished camera system gives the follower two very different descriptions of the same action.

The Brio cameras see the stage: the table, the objects, and the arm moving through the workspace. The FIT0701 sees the performance up close: the gripper approaching, aligning, closing, and carrying.

Neither view is complete on its own. Together, they turn a collection of joint positions into something much closer to a robot-learning observation: what the arm is doing, where it is doing it, and what is happening at the point of contact.

The cameras did not merely document the workstation. They became part of the robot.

References


Share this post:

Previous Post
The Commands Behind My LeRobot Workflow
Next Post
Calibrating the LeRobot SO-101