Mobile Manipulation · Peanut Robotics
My first job out of grad school was building robots that clean hotel bathrooms.
July 2022 to April 2024 at Peanut Robotics as a Robotics Engineer, owning the manipulation subsystem, sensor integration, perception, and operator tooling on a mobile manipulator deployed in real hotels.
- Shipped into 2 hotels with 10 active robots cleaning on a paid hourly contract.
- Operator training for a new surface dropped from 2 to 4 hours to 15 minutes.
- Evaluated six depth-camera systems across Orbbec, RealSense, Luxonis, and ZED, and chose on specular performance.
- Robust four-camera bringup with a heartbeat and automatic recovery.
- Point-cloud registration that measures room-to-room geometric variation so one taught trajectory transfers.
I joined Peanut right out of CMU. The office was a single room in San Francisco - around 19 people, and a lot of robots. Peanut built custom mobile manipulators tasked for cleaning bathrooms in commercial hotels. We were deployed in 2 hotels, with 10 active robots cleaning and getting paid by the hour.
As a Robotics Engineer, I worked cross-functionally across the manipulation stack, navigation, hardware, testing and sensor integration. My main ownership was the robot arm and manipulation stack. My primary projects focused on manipulation.
1. Live Trajectory
Operators taught trajectories with a joystick, jogging the arm to capture joint positions one at a time and chaining them into a path. For a complex object like a toilet, that took a skilled operator 2 to 4 hours, plus a lot of retraining. Since every new location had to be taught this way, teaching speed was a hard ceiling on how fast Peanut could scale.
Built a live trajectory system that let operators teleoperate the arm by hand, physically guiding it through the cleaning motion while it recorded the trajectory directly. The work wasn't just capture: I had to plug into the existing system's recovery behaviors for failure cases, spraying logic, and more. Under the hood the captured path is time-parameterized with TOPPRA and executed through ros_control with the tool actuating in sync. Getting it right meant going deep on the operators' actual pain points and building something they would genuinely use.
Impact
The training-time number is the headline, but it was not the only thing that changed. Four effects came out of it:
- Training time collapsed. 15 minutes against 2 to 4 hours, and not only on the easy surfaces. It held up on complex geometries too, which is where joystick teaching cost the most time in the first place.
- Cleaning quality improved on edges and corners. These are precisely the places a joystick-jogged path handles badly, because approximating a curve through sparse captured joint positions rounds off exactly the detail that matters at a boundary.
- Trajectories became something you could prototype. Cheap paths meant we could try the same task across different surfaces, swap in different end effectors, and run quick experiments like pressure testing, rather than committing to one authored path.
- Human-intuitive wrist motion became expressible. Guiding the arm by hand lets an operator apply pressure and roll the wrist the way a person actually scrubs, which is difficult to specify through a joystick at all.
That third point is the one with the longest tail. Cheap trajectories turned into a testing instrument for the rest of the robot, and the team used it to run:
- Cleaning tests, checking scrubber contact quality against the surface
- Comparisons between different cleaning tools
- Corner and ledge tests
- Scrubber RPM tests
- Flat-surface behavior
- Pressure tests on difficult surfaces such as mirrors
Where it fell short
- Recording had to start from zero velocity.
- It required near-perfect choreography from the operator, since any hesitation was faithfully recorded as a pause in the trajectory.
- Manipulating the arm by hand was awkward in tight spaces, which a bathroom has plenty of.
- Operators could record physically "illegal" moves, such as accelerations the arm should not reproduce.
- Resume functionality broke on live-recorded paths.
Where it was going next
- Trajectory stitching, so separately recorded segments could be joined instead of choreographed in one take.
- Post-processing to strip out pauses, which removes the choreography burden rather than asking operators to be perfect.
- Remote-control arm operation, sidestepping the tight-space problem.
- Partially torquing the wrist motors to limit wrist movement and keep recordings within what the arm can safely execute.
- Resuming with TOPPRA trajectories, to fix the broken resume path.
2. Room to Room Variation
Even hotel bathrooms with the same layout vary, since fixtures sit a little differently from room to room. A trajectory taught in one bathroom doesn't directly transfer to the next: run it as-is and the arm collides or misses the surface, and the system can't scale to many rooms. I ran an in-depth study of where the variation actually lived, and found the toilet pose varies the most.
Built a point cloud registration pipeline that measures how far the toilet has shifted in a given bathroom, then applies that delta to the cleaning trajectory, either by moving the base or by shifting the arm trajectory itself. Used passive stereo depth (ZEDx) for the scan, and experimented with Cartesian planning through mission control to apply the correction.
Getting reliable depth in a bathroom
Registration is only as good as the depth behind it, and bathrooms are a worst case for depth cameras. Light bounces off glossy plastic seats and porcelain (specular surfaces) and you get wavy, unreliable point clouds. On top of that, the robot needed four cameras running at once without one silently dropping out and degrading the whole stack.
Evaluated six depth-camera systems across Orbbec, RealSense, Luxonis, and ZED, weighing specular performance, range, width, and software tradeoffs, and chose the Orbbec, which had the best filtering for the bouncing light. Then built the integration: robust bringup of four cameras with a system heartbeat and an auto-recovery system, plus easy toggling and selection of depth streams to manage compute, debugging deep into the Orbbec SDK along the way.
3. Optitrack
Because the arm is built in-house, there were real inconsistencies to chase down. The most notable was droop, where the arm sags from its commanded position under load. For contact cleaning, a few millimeters of error directly hurts clean quality, and it breaks sim-to-real because the model stops matching the physical robot.
Set up an Optitrack motion capture rig on the arm in-house and connected it to ROS, building out the kinematic chain, a lot of transform and frame math, and URDF cleanup. Profiled droop under different loads, then used the profiles to correct it and update the kinematic model.
Optitrack didn't stay a one-off droop fix. Once the rig was up, it became our ground-truth measurement tool across a whole range of arm, elevator, and base work: end effector and joint errors, controller and tracking accuracy, arm vibration, tower vibration on the elevator, and base drift and repeatability. The board below was our running list of things to measure with it, and the highlighted cells are the ones I dug into.
4. Other
Manufacturing test suite
Robots kept breaking down in the field, and diagnosing them remotely was slow. When something went wrong, an operator with no engineering background needed a way to find out what was broken without an engineer on a call.
Built a manufacturing test suite operators could run as scripts to get a clear report of what was and wasn't working, from the low level (individual arm motors, IDs, baud rates, udev rules) up to full-system checks (complete motion, reported currents, spikes, end-effector behavior), at small, medium, and large scale. Also shipped a simple arm calibration tool so operators could calibrate the arm themselves.
Navigation
Alongside the manipulation work, I spent time on the navigation stack: SLAM mapping with
gmapping, testing move_base and then Nav2 with the MPPI controller
in simulation and on the real robot, including obstacle-avoidance and wall-following runs. The
robot has to get itself to and around the work area reliably before the arm can do anything.
Hardware & sensors
A mobile manipulator is many subsystems (arm, base, elevator, scrubber, and a stack of sensors), and a deployed robot only works if all of them do. I did bring-up and debugging across Dynamixel servos, DC and stepper motors, a BLDC base motor, hall sensors (latching and distance), crush sensors, encoders, limit switches, and a magnetometer. Plus low-level system and performance work: a udev-rules wizard to automate device setup, and optimizing Dynamixel syncRead from 15 ms to 10 ms to tighten the arm's control-loop rate.
[INSERT: title for your reflection]
[INSERT: in your own words. What this place meant to you, what you learned, what surprised you about building robots for the real world. This is also where you can acknowledge that Peanut ran out of funding and wound down, an honest, human part of the story.]
