[PhD 2021] Data-driven anticipation for whole-body teleoperation

Supervisors: Jean-Baptiste Mouret & Serena Ivaldi (team LARSEN)

Contact: jean-baptiste.mouret@inria.fr 

Our general objective is to accurately teleoperate humanoid robots from whole-body motion capture system. Like the human operator, the robot must synchronize its many degrees of freedom to achieve the desired motion while keeping its balance. 

We recently obtained promising results with the iCub humanoid robot for whole-body teleoperation [1,2].

Videos:
https://www.youtube.com/iZVAacyvYhM
https://www.youtube.com/MokoPcHvhlQ?start=167

This technique (called “motion retargeting”) does not anticipate the motion of the operator, as it only aims at matching the positions of the end-effectors at each time step. For instance, the robot does not know when the operator will stop its motion. This lack of anticipation can, however, cause substantial tracking errors because the kinematics and the dynamics of the robot do not match those of the human perfectly: the robot might need more anticipation that the human to stop the trajectory of its hands because it is heavier. In addition, the absence of anticipation forces the robot to always be in static balance, since the motion can stop at any instant. Moreover, the anticipation can allow the robot to discriminate behaviors that can be tracked partially (walking) and moves that must be followed closely like actions using legs or foot (eg. kicks).

The objective of this PhD is to design a whole-body teleoperation system that anticipates the motion of the operator. It will be demonstrated on the TALOS humanoid robot of Inria Nancy – Grand Est / LORIA, a full-sized, state-of-the-art humanoid robot. The PhD will combine two highly dynamic scientific fields:

  • machine learning to learn trajectories of the humans and build predictors of future motions given current motion;
  • whole-body model predictive control [3] to use the predictions to improve the control of the robot.

The applicant must have strong knowledge of robotics and machine learning, as well as strong C++ skills (the robot is programmed in C++).

The PhD will use our robot TALOS, a state-of-the-art, full-size humanoid robot (1.7m, 95kg).

[1] Penco L, Clément B, Modugno V, Hoffman EM, Nava G, Pucci D, Tsagarakis NG, Mouret JB, Ivaldi S. Robust real-time whole-body motion retargeting from human to humanoid. In2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) 2018 Nov 6 (pp. 425-432). IEEE.

[2] Penco L, Clément B, Modugno V, Hoffman EM, Nava G, Pucci D, Tsagarakis NG, Mouret JB, Ivaldi S. Robust real-time whole-body motion retargeting from human to humanoid. In2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) 2018 Nov 6 (pp. 425-432). IEEE.

[3] Koenemann J, Del Prete A, Tassa Y, Todorov E, Stasse O, Bennewitz M, Mansard N. Whole-body model-predictive control applied to the HRP-2 humanoid. In2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015 Sep 28 (pp. 3346-3351). IEEE.

Logo d'Inria