Robot awareness of human behaviors is an essential research problem in robotics with many important real-world applications, including human-robot collaboration and teaming. Over the past few years, depth sensors have become a standard device widely used by intelligent robots for 3D perception, which can also offer human skeletal data in 3D space. Several methods based on skeletal data were designed to enable robot awareness of human actions with satisfactory accuracy. However, previous methods treated all body parts and features equally important, without the capability to simultaneously identify discriminative body parts and features.


We propose a novel simultaneous Feature And Body-part Learning (FABL) approach that simultaneously identifies discriminative body parts and features, and efficiently integrates all available information together to enable real-time robot awareness of human behaviors. We formulate FABL as a regression-like optimization problem with structured sparsity-inducing norms to model interrelationships of body parts and features. We also develop an optimization algorithm to solve the formulated problem, which possesses a theoretical guarantee to find the optimal solution.

Code and Citation

Download the Matlab code for our FABL approach HERE. The code provided here is for non-commercial research/educational use only.

Please cite the following paper if you use our code or method. Detailed information of the FABL approach is also available in the paper.

      AUTHOR    = {Fei Han AND Xue Yang AND Christopher Reardon AND Yu Zhang AND Hao Zhang},
      TITLE     = {Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors},
      BOOKTITLE = {IEEE International Conference on Robotics and Automation (ICRA)},
      YEAR      = {2017}

Questions and Suggestions

Please contact Dr. Hao Zhang: hzhang -AT- mines -DOT- edu