Nicolas Mansard

Address: LAAS-CNRS
7, avenue du Colonel Roche
31077 Toulouse Cedex 4
Assistant: 05 61 33 64 69 (Marie-Agnès Bellieres)

Short bio

Since October 2008, I am working as a permanent researcher at LAAS/CNRS, Toulouse. I am involved in the Gepetto research group, with Philippe Souères, Florent Lamiraux, Olivier Stasse and Jean-Paul Laumond. I defended by Habilitation à Diriger des Recherches in Jully 2013.

In 2013, I was an invited researcher at Emo Todorov's lab, University of Washington (Seattle), USA.

I received the 2015 CNRS Bronze Medal for my research in robotics. The project ANR Entracte I am coordinating received the Grand Prix de l'ANR in 2016.

My research activities are concerned with sensor-based control, and more specifically the integration of sensor-based schemes into humanoid robot applications. It is an exciting research topic at the intersection of the fields of robotics, automatic control, signal processing and numerical mathematics. My main application field is currently the Humanoid robotics, as it causes serious challenges that are representative of many other robot domains.

Before being in the Gepetto group, I was a post-doc in ISRI Group, AIST Tsukuba, and a post-doc at the IA Lab of Stanford University, in the group led by Prof.O.Khatib. I have done my PhD. with Francois Chaumette in the Lagadic group, at INRIA Rennes. I was a student of the École Normale Supérieure de Bretagne and did both my MSc (Robotics) and BEng (CS) in the INP Grenoble ENSIMAG.


An overview of my research project is available below (in French only, sorry).

Hierarchical problems

The pseudo-inverse, used in most of the task-function-based works, gives the solution of an implicit least-square quadratic program. When considering only one task, it is then straight-forward to take into account some inequality constraints, such as an obstacle or joint position and velocity limits. The extension to a hierarchy of tasks is not immediate and was proposed by Kanoun in 2009, by using a set of quadratic programs in cascade. This formally defines the hierarchical quadratic problem and gives a naive resolution scheme that is very slow. This paper proposed to find an efficient resolution scheme to the cascade of quadratic problems.

The study first focuses on equality-only quadratic program and proposes an efficient resolution method based on a dedicated extension of the matrix complete decomposition classically used to compute the pseudo inverse. The method is five times faster than the classical concurrent solution to solve a hierarchical quadratic program. An active search dedicated to the hierarchical structure is then proposed to take into account a hierarchy of both equality and inequality. The solution avoids the problem of multiple activation and deactivation generally encountered with cascade of solvers. It is also much faster and straightforward to adapt to real-time constraints. Incidental results are also given to prove the complexity, the continuity and the stability.

This work is performed in collaboration with P-B. Wieber and A. Escande. It is sponsored by the project FP7 Robohow. See the page about the HQP paper. A MATLAB solver is available here (clean code with a lot of documentation, but the algorithm may not be very robust). A more efficient but less clean C++ is available on the LAAS github.

Inverse Dynamics

Task-based motion generation enables fast development of whole-body movements. When accounting only for the geometry of the system, the movements are very rough. This is especially true when generating human-plausible motions. The inverse-dynamics solver was developed to generate dynamically-consistent movement from a hierarchy of tasks.

The motion generator is based on our HQP solver. The variables are the contact forces and joint torques and accelerations. These variables must fulll the dynamics equation and satisfy the given contact model (only rigid contact was tested yet). When all the contact points are included in a horizontal plane, the contact constraint is equivalent to the classical ZMP constraint. The tasks are nally introduced in the solver by imposing the acceleration in the operational space to be equal to a specied value.

Sensor Based control

The camera is able to reconstruct the robot position with respect to previous positions. The map helps to consolidate the accuracy. However, the orientation is subject to drifts and the camera is not able to accurately track alone the robot vertical. Moreover, it is not the proper sensor to estimate the contact model (surface orientation and stiness). Two other sensors can be used to bring the missing data. The inertial measurement unit (IMU) measures the angular velocity on one hand and the linear accelerations and gravitational forces on the other hand of the body it is attached to. When the linear acceleration are perfectly known (for example, when the robot does not move), the weight force gives the gravity orientation. When the acceleration is not known, the six measurements are not sucient to make the gravity orientation observable. A classical association is to use the camera to observe the missing information. The coupling IMU/Camera is very interesting: the camera is slow (<100Hz) whereas the IMU is fast (>1kHz); the IMU quickly drifts whereas the camera does not drift while it stays in already-visited places. Moreover, the system camera+IMU is observable. A first fusion was performed in RT-SLAM.

We now tries to fuse the data coming from the force sensors in the ankles to estimate the elasticity in the robot feet and the ground characteristic. In parallel, Florent Lamiraux is working on the stabilization of the feet elasticity and Olivier Stasse is working on the rough estimation of the oor slope using vision. Coupled with the developments of fast dynamics-based control law, all these works are going toward safe outdoor walking on unknown grounds.

Motion signature

When replicating a motion, several spaces can be chosen to compute the distance to the original motion. However, in a given context, only one of them is relevant. The idea of the paper is to automatically recognize the relevant space, which would then be used as the reference task space to replicate.

The task recognition is performed only from the body trajectories without using any contextual information: the input is the configuration trajectory of the demonstrator. The motion is then projected in each task space of a candidate task pool and t with a reference trajectory. The residue of the fitting is used to select the best task among the candidates. The effect of this task is then cancelled from the original motion by projecting it in the space orthogonal to the task. The iterative selection finally ends when the residual motion is null.

Task for planning and performing

We look at a solution to automatically decide the data that are needed to specify a task sequence. When reasoning about a motion (for planning for example), the whole spatio-temporal trajectory should be considered1. The motion planning (both for searching for an initial solution (RRT, PRM) or to optimize a given trajectory) is a very complex problem, subject to the question of the existence of a solution and of many local minima. This is the good mathematical way to build a motion. However, the computational cost is yet much too expensive to hope to use it inside the robot control loop. Moreover, if it is used as a planner (i.e. to guide the controller by computing the desired motion as a much slower rate), it provides a trajectory in the configuration space, that is straight-forward to track with a fixed manipulator robot but can be more complex to follow with a mobile robot. The question behind this chapter is to know how to execute with a closed-loop controller a trajectory computed by a motion planner, or more generally how to bridge the gap between planning and control. In this objective, the task sequence can be used as a trajectory representation that makes the link between these two components: the task sequence can be used for planning, while our experience has shown that it is a very efficient way to control the execution.

Open-source motion generation

A inverse-kinematics based movement on the robot contains many basic routines that are reusable but have to be written on a very efficient manners, and topical implementations that does not require a high computational efficiency but is more difficult to reuse from an application to the other. The software is proposed to maximize the efficiency of the kernel routines and allow an easy integration of these routines to produce an actual robot movement.

The software is built as a mixed between C++ efficient routines, and a scripting language to construct the robotics scenarios. The control scheme is assembled using a graph structure (using a design pattern similar to MATLAB/SIMULINK) that links together the C++ routines. The basic features are then proposed to realize an inverse kinematics scheme: robot model computation (direct geometry and kinematics, Jacobians, generalized inertia matrix, etc), classical task functions (end-eector positioning, COM, gaze, etc) and a hierarchical inverse kinematics solver. The software is then distributed with a LGPL license. Everything is available through the Stack-of-Tasks Github.

Dance with a robot

The Dance with HRP-2 project is the opportunity to show the interest of the inverse dynamics approach for the generation of whole-body movements for humanoid robots.

The movements were first demonstrated by T. Benamara and recorded using a vision-based motion-capture system. The human posture is extracted by solving for each pose a non-linear least-square problem. The resulting posture is then dynamically retargeted using the inverse-dynamics solver. When necessary, tasks are added to obtain a more accurate replication of the movement (in particular, for fast motion of the COM). For walking, an inverse-pendulum based pattern generator is used to design the COM trajectory.

Motion capture together with inverse dynamics for reshaping allows a pretty fast development time. Nearly twenty minutes of motion were designed in less than two months by a single programmer. The show was performed live, without the security lifter, in front of more than 1000 in the music-hall of Toulouse city on October 2012.

Robust walk

We have proposed a first use of the inverse-dynamics scheme as a robust controller in simulation, to control the walk of HRP-2 on an unknown non-at terrain. The hypotheses of the simulation are the following. The simulated robot is tracking the given reference torques (and accelerations). It also perfectly perceives the collision points that are used in the force part of the dynamics equation. The floor is a random distribution of small objects (2cm) on a horizontal plane. However, the model of the floor is not known from the controller, which uses the approximation that the oor is horizontal. The COM trajectory is computed at each control cycle by MPC using a classical inverse-pendulum linearization. The hierarchy of tasks is composed of the COM task, one task to track the flying-foot placement and one task to emphasize the robot posture and avoid drifts of the chest attitude. Contact points are added in the SOT when a collision occurs between the flying foot and the ground during the foot landing. When three contact points are added, the foot landing ends because the foot movements are completely constrained. The contacts are relaxed when the walking pattern generator decides to take off.

The controller is then able to absorb the unknown rough terrain: the COM tracks the MPC trajectory even if the contact surfaces are not exactly those used by the inverse-pendulum model. The rough horizontal approximation is sufficient to ensure the proper regulation by the inverse-dynamics controller.

Selected Publications

All the publications are avaiblable here (with pdf files if available).

PhD Students

Two positions are currently open (in optimal control and simultaneous localization and mapping, and in motion planning for versatile factories). Contact me with vitae and reference letters if you are interested.


I also taught Computer Vision in Rennes (INSA and IFSIC), Numerical Optimisation at EDSYS, Toulouse, and assisted for practical works for the class of O. Khatib at Stanford and J-P. Laumond at Collège de France.


I am leading ANR project ENTRACTE (cooperation with INRIA Rennes, start Nov 2013). I am the CNRS coordinator and WP leader of the FP7 EuRoc project (led B. Siciliano, Univ. Napoli, start Jan 2014). I have a PI role in Oseo project Romeo-2 (led by Aldebaran and in the cooperation between LAAS and Airbus (led by O. Stasse). I participate to the ERC Actantrophe (led J-P. Laumond), FP7 Koroibot (led by K. Mombaur, CNRS coordinator P. Soueres) and marginally in FP7 Factory in a Day (led by M. Wisse, CNRS coordinator F. Lamiraux) and FP7 Robohow (led by M. Beetz, CNRS coordinator A. Khedar).

I was also part of the FP7 Robot@CWE, FUI Romeo and of various other projects (project leader of Novela project 2011, workpackage leader of ANR R-Blink 2009, Robea Egocentre 2003 and marginally in FP6 VISORBASE 2002).

Peer-review activities

I am Associate Editor in IEEE TRO since July 2013. I was also Associate Editor in IEEE ICRA from 2011 to 2014, in RSS from 2011 to 2013 and in IEEE ICARCV in 2011. I am a frequent reviewer for IJRR, RAS, IEEE ICRA, IEEE IROS, IEEE Humanoids, IEEE TAC and various other conferences.

I was reviewer for the call-for-proposal CHISTERA 2013, and in the several phases of ECHORD.