Claude Stacks A Cube

We gave Claude Code an SSH connection to an SO-101 arm, two black cubes on a table, and one task: stack them. It had no teleoperation and no demonstrations. It succeeded in 32 minutes.

What it did

• Ignored a bad calibration. The overhead camera's pixel-to-table fit was about 4 cm off, so the agent located cubes with the wrist camera instead.
• Calibrated by moving. Two 2 cm test moves showed how the wrist image maps to arm motion.
• Missed twice, then switched approach. Both misses came from computing a position from pixels. The third grab worked by moving until the wrist image matched a reference of a held cube.
• Made its own landmark. It put cube A back exactly where it had gripped it, so A's position was known to the millimetre. It then used the view above A as a template to find and grab cube B on the first try.
• Detected contact without force sensing. While lowering B onto A, it treated the cube slipping in the jaws as the signal that it had touched.

By the numbers

56 robot commands · 4 grab attempts, 2 misses · about half the session spent running commands, the rest deciding what to do next.

Caveat

This is one episode, so it shows the approach can work, not how reliably it does. Next is a proper eval: many sessions with randomized cube positions.