Davide De Tommaso

Davide is a PhD student at the Department of Advanced Robotics, Italian Institute of Technology (IIT), Genova, Italy.
He is a computer engineer and his Phd topic is about Multimodal Interfaces Exploration in Human-Robot Interaction for Robot Skill Learning

Projects

A Projective Multimodal Interface in Human-Robot Interaction for Transferring Skills

Last update July 2011

Brief Description
We propose a method for enabling humans and robot to interact each other in the same environment for transferring skills using a structured light system combining a pico-projector and a RGB-Depth sensor. Instead of considering fixed tracking and screen or projection displays, a robot manipulator is endowed with perception and projection capabilities. This two-ways tangible interface enables to perceive the environment and to visualize augmented information in the workspace of the robot, helping the user to better understand what the robot has learned or to display information on the task.

Motivations
The most common way of transferring skills to a robot is computer programming and the use of teaching pendants. Since they require specific expertises in robotics programming, that the final user may not be familiar with, it becomes crucial to explore new modalities and communication channels of interactions. This project considers a compliant lightweight robotic arm as an interface that can move, actively perceive and act on its environment to facilitate task learning and visualization by enhancing the robotic tool with active perception and augmented reality capabilities. Indeed, our aim is to develop a system to share a common understanding of the whole scene able to get input from the humans gestures and to enrich the environment with augmented visual features according to the learning task. These issues remain largely unexplored in the context of learning by imitation, involving both human-robot interaction and human-computer interaction aspects.

Method
We explore scenarios where a pico-projector and a RGBD sensor (Microsoft Kinect) are mounted on top of a robotic arm ( Barrett WAM 7dof). The merging of these three technologies provides the robot with an active perception of the surrounding environment by detecting humans and objects but also by providing the user with a moving interface to visualize augmented digital information over real objects as HCI interfaces. In order to realize the system, some computer vision techniques must be exploited to represent spatial information, to recognize humans and objects and to track relevant movements. We consider a perception device such as RGBD cameras that can track both color and distance information, and that are small enough to be mounted on the end-effector of the robot. This allows the robot to actively track the features that are relevant for the particular task and also to process scene perception based on the extraction of the invariant characteristics of the task instead of relying on pre-defined configurations. The projective device is used to highlight objects of interest for the task or to visualize additional information that are relevant for the user, such as trajectories or landmarks directly in the robot's environment. While this issue has been studied in the context of mobile robotic platforms, it presents new challenges in the context of manipulation and human-robot collaborative skills learning.

Recent updates