Find the Transform From Any Depth Sensor to Any Robot
My method finds the extrinsic transform from a depth sensor to a robot from a single view that includes rotationally unique robot features. This is useful for obtaining point clouds in the robot's base frame for visuomotor control, for egocentric and external depth sensor placements.
My method constructs a synthetic robot mesh using the robot’s URDF and joint positions, then applies ICP to align the mesh with the depth sensor’s view of the robot. This precisely estimates the sensor’s transform relative to a desired robot link. The approach generalizes across different robots and depth sensors, provided that the sensor captures a rotationally unique part of the robot, the depth cloud is cropped to primarily include robot points, and a rough initial sensor pose is given. For example, on a humanoid robot, a head-mounted depth camera can be localized relative to the head using a single view of the robot’s hands. Similarly, a depth camera mounted on a robot arm can be localized relative to the arm’s end by viewing the robot’s base links.
My code will hopefully be released soon!
