Balloon Catching with Franka Robot Arm

The objective of this project is to play the game 'Don't let the balloon drop to the ground' with a Franka Emika Panda arm. It does this by tracking a falling balloon in real time, predicting where the balloon will land, moving to the location before the balloon lands, and pushing it up. This process is done semi-autonomously, with some human-robot interaction, and utilizes open-source computer vision and motion planning software packages. Please find the GitHub repo here!

Balloon Trajectory Prediction

To successfully catch the balloon, I implemented a robust position prediction algorithm that predicts the balloon's trajectory in 3D space, and the balloon's approximate position is sent to the trajectory planning module, while leaving enough time for motion planning and trajectory execution.

Arm Motion Planning and Execution

The ROS2 node allows users to input target position and orientation for the robot arm end effector, and the planning module based on MoveIt2 package can output an executable pose trajectory which can then be executed by the Franka robot arm.

The planning and execution node can generate two kinds of trajectories for the end effector: first choice is cartesian path, which basically moves the end effector in a straight-line in space; the second choice is screw trajectory, which allows end effector more flexibility by moving it in an arbitrary but viable path in space.