next up previous contents
Next: Contents


MIT Media Laboratory, Perceptual Computing Technical Report #507
Appears in: Master's Thesis, MIT, 1998

Action-Reaction Learning:
Analysis and Synthesis of Human Behaviour



by

Tony Jebara



B.Eng., Electrical Engineering
McGill University, Montreal, Canada
June 1996


Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning,
in partial fulfillment of the requirements for the degree of
MASTER OF SCIENCE IN MEDIA ARTS AND SCIENCES
at the
Massachusetts Institute of Technology
May 1998


©Massachusetts Institute of Technology, 1998
All Rights Reserved







Signature of Author

Program in Media Arts and Sciences
8 May 1998



Certified by

Alex P. Pentland
Academic Head and Toshiba Professor of Media Arts and Sciences
Program in Media Arts and Sciences
Thesis Supervisor



Accepted by

Stephen A. Benton
Chair
Departmental Committee on Graduate Students
Program in Media Arts and Sciences

Action-Reaction Learning:
Analysis and Synthesis of Human Behaviour



by
Tony Jebara





Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning,
in partial fulfillment of the requirements for the degree of


Master of Science in Media Arts and Sciences







Abstract


I propose Action-Reaction Learning as an approach for analyzing and synthesizing human behaviour. This paradigm uncovers causal mappings between past and future events or between an action and its reaction by observing time sequences. I apply this method to analyze human interaction and to subsequently synthesize human behaviour. Using a time series of perceptual measurements, a system automatically uncovers a mapping between past gestures from one human participant (an action) and a subsequent gesture (a reaction) from another participant. A probabilistic model is trained from data of the human interaction using a novel estimation technique, Conditional Expectation Maximization (CEM). The estimation uses general bounding and maximization to find the maximum conditional likelihood solution. The learning system drives a graphical interactive character which probabilistically predicts the most likely response to a user's behaviour and performs it interactively. Thus, after analyzing human interaction in a pair of participants, the system is able to replace one of them and interact with a single remaining user.















Thesis Supervisor: Alex Pentland
Title: Academic Head and Toshiba Professor of Media Arts and Sciences, MIT Media Lab


This work was supported in part by British Telecom and Texas Instruments.

Action-Reaction Learning:
Analysis and Synthesis of Human Behaviour



by
Tony Jebara


The following people served as readers for this thesis:












Reader:

Bruce M. Blumberg
Asahi Broadcasting Corporation Career Development
Assistant Professor of Media Arts and Sciences
MIT Media Laboratory





Reader:

Aaron Bobick
Assistant Professor of Computational Vision
MIT Media Laboratory






Reader:

Michael I. Jordan
Professor of Brain and Cognitive Sciences
MIT Brain and Cognitive Sciences

Acknowledgments







I extend warm thanks to my advisor, Professor Alex Pentland for having given me to opportunity to come to MIT and for being a great source of inspirational ideas, insight and enthusiasm. Thanks, Sandy, for your support, knowledge, wisdom and patience, and for your faith in me.

I thank Professors Aaron Bobick and Bruce Blumberg for having been readers for this thesis. Thanks, Aaron, for your wit, intuition and in-your-face honesty. Thanks, Bruce, for your inspiring ideas and knowledge on animation and behaviour.

I thank Professor Michael Jordan for having read and commented on parts of the thesis. Thanks, Mike, for sharing your passion and knowledge of machine learning and statistics.

I also wish to thank my friends at the Media Laboratory for their support during the thesis. Thanks to Kris Popat who originally motivated me to think about conditional densities and for his inspirational work on decision-tree conditional density estimation. Thanks, Nuria Oliver, for helping edit this thesis and for your cherished support. Thanks, Nitin Sawhney, for working late-night and being there when everybody else was asleep. Thanks, Deb Roy, for showing me the ropes and reminding me to relax. Thanks to Brian Clarkson for hearing me whine about everything. Thanks to Bernt Schiele for reading the thesis and being the best German post-doc in VisMod. Thanks to Sumit Basu for letting me steal his cookies. Thanks, Tom Minka, for reading the thesis and for great conversations about statistics. Thanks, Ken Russell, for help with face modeling, excellent hacking and, of course, blitzing to FedEx. Thanks as well to Chris Wren and Barbara Rosario, office mates who had to deal with me taking so much space and having such a messy desk. Thanks to the VizModGirls: Tanzeem Choudry, Karen Navarro, Professor Rosalind Picard, Flavia Sparacino and Jen Healey. I'm running out of space so I'll speed up... Thanks to Thad Starner, Baback Moghaddam, Crazy Lee Campbell, Martin Zoomy Szummer, Andy Wilson, Claudio Pinhanez, Yuri Ivanov, Raul Fernandez, Chris Bentzel, Ifung Lu, Eric Trimble, Joey Berzowska, Jonathan Klein, Claudia Urrea, Marina Umaschi, Gert-Jan Zwart, Kai Wu, Avinash Sankhla, Dimitri Kountourogiannis, Laxmi Perori, Deepak Jain, Frank Bader and everybody else who I'm forgetting who made this place really fun.

Also, a heart-felt thanks to all my friends from back home in Montreal.

I would most like to thank my family: my father, mother and sister, Carine. They are the best part of my life.



 
next up previous contents
Next: Contents
Tony Jebara
1999-09-15