Description

In recent years, there has been a tremendous effort in research toward novel devices and techniques to improve the naturalness of interaction with computers. Gesture recognition is among the most promising attempt in this direction. Gestures are expressive, meaningful body motions involving physical movements of the fingers, hands, arms, etc., with the intent of conveying meaningful information or interacting with the environment. Numerous approaches have been applied to the problem of visual interpretation of gestures for HCI. Images of hands, geometric moments, contours, silhouettes, and 3D hand skeleton models are a few examples. In recent years, however, there has been an interest in incorporating the dynamic characteristics of gestures, which carry as many information as postures. The major goal of this school is to brush a state-of-the-art in gesture recognition, both at the recognition (detection, tracking, recognition), dialog and application levels.

Speakers and Content

Dr. Rémi Ronfard, INRIA Grenoble

Rémi Ronfard, PhD, is a senior researcher in machine learning and computer vision at INRIA in Grenoble, France. After working for several years on multi-camera motion capture and action recognition in the MOVI team, he recently joined the LEAR team to investigate the topic of action recognition in movies.

In 2007-2008, he was the head of the Virtual Cinematography research team at Xtranormal Technology in Montreal, Quebec, where he developed automatic cinematography and movie editing tools for digital storytelling. Before that, he has worked in international research projects with IBM Research, Dassault Systèmes and INA as well as the MPEG expert group.

Machine Learning Methods in Visual Recognition of Gesture and Action

In this three-hour tutorial, we will review methods for learning models of human actions from visual examples, and recognizing them from video, using recent, state-of-the-art machine learning.

Gestures and actions are spatio-temporal patterns with internal structure and high complexity. The choice of suitable representations is often the most crucial aspect. In the spatial domain, actions and gestures can be represented with body models, with image models, or with bag of isolated features. In the temporal domain, they can be represented with templates, with grammars, or with bags of isolated features. By combining the spatial and temporal aspects of gesture and action, one is faced with a vast number of possible combinations.

The tutorial will survey the most useful and promising techniques for learning and recognition in each case, and discuss their advantages and limitations for real-life applications in face, gesture and full-body action recognition. In particular, we will examine how those different classes of methods can be adapted to achieve invariance with respect to viewing directions and performing styles.

François Bérard, LIG Grenoble

François Bérard, PhD, is an associate professor at Grenoble Institute of Technology and a member of the Engineering of Human-Computer Interaction group of the LIG research lab (Laboratoire d'Informatique de Grenoble). He has been doing research on large interactive surfaces since his PhD and during scientific visits to the Xerox Palo Alto Reseach Center (PARC) and the MIT Media Laboratory. Since 2008, he splits his time between Grenoble and McGill University in Montreal to cooperate on research in the field of interaction in 3D.

Gesture Design and Implementation for Interacting on Surfaces

The advent of multi-touch interactive surfaces, such as the Apple iPhone, is breaking the status quo in Graphical User Interfaces. It is an opportunity to bring a more direct, gesture based, interaction from research experimentations to broad usage. But the design space for 2D gestures is very large and not all gestures are suitable to Human-computer interaction (HCI). This tutorial will be centered around the question "what makes a good gesture for 2D interactive surfaces?" Based on an overview of the reseach produced in the HCI litterature, and on our personnal experimentations, we will provide elements of answer to this question. We will acknowledge that gesture design should consider the technical limitations of the gesture tracking technology. The tutorial will thus include an overview of the tracking approaches that are available to the interaction designers.

Frédéric Bevilacqua, IRCAM Paris

Frédéric Bevilacqua is the leader of the Real Time Musical Interactions team at IRCAM - Institute for Music/AcousticResearch and Coordination in Paris (www.ircam.fr). He holds a master degree in physics (1991) and aPh.D. in Biomedical Optics (1998) from the Ecole Polytechnique Fédérale de Lausanne (Swiss Federal Institute of Technolgy in Lausanne). He also studied music at the Berklee College of Music in Boston (1992–1993) and has participated in various music and media arts projects. From 1999 to 2003 he conducted various research projects at the Beckman Laser Institute at the University of California Irvine. He joined IRCAM in October 2003 as researcher on gesture analysis for music and performing arts.

Gesture controlled Musical Systems

Frédéric Bevilacqua is the leader of the Real Time Musical Interactions team at IRCAM - Institute for Music/AcousticResearch and Coordination in Paris (www.ircam.fr). He holds a master degree in physics (1991) and a Ph.D. in Biomedical Optics (1998) from the Ecole Polytechnique Fédérale de Lausanne (Swiss Federal Institute of Technolgy in Lausanne). He also studied music at the Berklee College of Music in Boston (1992-1993) and has participated in various music and media arts projects. From 1999 to 2003 he conducted various research projects at the Beckman Laser Institute at the University of California Irvine. He joined IRCAM in October 2003 as researcher on gesture analysis for music and performing arts.

Registration

Participation is free of charge for all members of the "Troisième Cycle" (i.e. EPFL, Universities of Fribourg, Geneva, Lausanne and Neuchâtel).

Access to the registration form

Program

Three internationally renowned speakers will present the state of the art in video based detecton and recognition of 3D gestures, including machine learning approaches. The school will also address techniques for recognizing 2D gestures (in particular pointing) on interactive surfaces, as well as the related HCI issues related to interaction with surfaces. Finally, the school will focus on applications of gesture recognition to musical applications.

9:30 - 12:30 a.m. 1:30 - 4:30 p.m.
June 21st Machine Learning Methods in Visual Recognition of Gesture and Action (R. Ronfard, INRIA) [presentation] Gesture Design and Implementation for Interacting on Surfaces (F. Berard, LIG) [presentation]
June 22nd Gesture controlled Musical Systems (F. Bevilacqua, IRCAM) [presentation] -

Organization and Contacts