Anticipating Human Behavior

Munich, Germany, 8th September 2018, in conjunction with ECCV'18


In contrast to humans that are very good in anticipating the behavior of other objects, animals, or humans, developing methods that anticipate human behavior from video or other sensor data is very challenging and has just recently received an increase of interest. In the past, the features for analyzing, in particular, visual data like images or videos were too weak such that approaches that predict the future were unlikely to succeed.

This burden has been overcome due to recent progress in this field. The anticipation of human behavior, however, is not well defined in the literature and varies depending on the task in terms of granularity and time horizon. In the context of driver assistance systems, the prediction of the trajectory of a pedestrian needs to be within centimeter accuracy but only for a very short time horizon of one second.

For tracking applications or motion planning, the potential destination of a human and trajectories of several seconds or minutes to reach the destination need to be predicted. In order to prioritize several tasks for a service robot during a day, only the rough time and location of an activity is needed. For instance, when the robot anticipates that the owner wants to cook in one hour, the robot will be in the kitchen at the right time.
The purpose of this workshop is to discuss recent approaches that anticipate human behavior from video or other sensor data, to bring together researchers from multiple fields and perspectives, and to discuss major research problems and opportunities and how we should coordinate efforts to advance the field.

The topics of interest for this workshop include, but are not limited to:

  • Early activity recognition
  • Anticipation of trajectories
  • Anticipation of human poses
  • Anticipation of activities or events
  • Anticipation of group behavior
  • Predicting frames, features or semantic in videos or other sensor data
  • Predicting use of objects or affordances
  • Datasets, evaluation, and benchmarking
  • Applications including but not limited to robotics, autonomous systems, virtual/augmented reality
Prospective authors will be invited to submit a regular paper of previously unpublished work (ECCV workshop format) or an extended abstract of a published or ongoing work via the workshop webpage.

All the submissions will be reviewed by the workshop’s international program committee.

Accepted regular papers will be presented during the oral or poster sessions and included in the ECCV workshop proceedings. Accepted extended abstracts will be presented at the poster session.

We provide a limited number of travel grants for early career researchers, which present their work at the workshop.

The workshop will be located in room N1080ZG at TU München, Arcisstraße 21 The posters will be located in room 2607 and 3607 (upstairs gallery of 2607).


Submission Extended 12.07.2018 (7AM Pacific Time) Notification 31.07.2018 Camera-ready 30.08.2018 Workshop 08.09.2018

Submission and Registration

Regular Papers

Prospective authors are invited to submit a 14-page paper (ECCV format) via the workshop submission web-page. The papers will be reviewed by an international Program Committee. Reviews will be double-blind according to ECCV guidelines. The papers are expected to present novel work. Authors should take into account the following: All papers must be written in English and submitted in PDF format. The maximum paper length is 14 pages, excluding references. The workshop paper submission guidelines are the same as the Main Conference papers. Submissions will be rejected without review if they: contain more than 14 pages (excluding references); violate the double-blind policy; violate the dual-submission policy. Authors will have the opportunity to submit supplementary material. Accepted papers will be published by Springer together with all other ECCV workshop papers as post-proceedings. Regular workshop publications are intended to be treated just like conference publications, in that at least one of the authors is required to register for the conference (and hence pay the registration fee).

Extended Abstracts

Prospective authors are invited to submit a 2-page extended abstract (ECCV format) via the workshop submission web-page. The abstracts will be reviewed by an international Program Committee. Reviews will be single-blind according to ECCV guidelines. The extended abstracts might summarize already published work, work under review, or work in progress.
Authors should take into account the following: All papers must be written in English and submitted in PDF format. The maximum paper length is 2 pages, excluding references. In contrast to regular papers, the authors names should be added to the extended abstract.
Accepted extended abstract will not be published and dot not count as a publication. Instead, the authors will present their work either as poster or oral presentation at the workshop. Registration for the workshop will be required.

Paper Template

Paper Template for Camera-Ready Paper

Copyright Form

Travel Grants

If you wish to be considered for a travel grant, please mark 'yes' to the question 'Are you applying for a travel grant?' when submitting your paper or extended abstract on CMT.

Submission Web-Page

Travel Grants

We provide a limited number of travel grants for early career researchers, which present their work at the workshop. The travel grants will cover parts of the travel cost (conference fee, accommodation, transport). The amount (about 400 EUR) will be determined. The selection will be based on the quality of the submission and the current academic career stage. We aim to support in particular female researchers.
If you wish to be considered for a travel grant, please mark 'yes' to the question 'Are you applying for a travel grant?' when submitting your paper or extended abstract on CMT. The grant will be awarded to the presenter who is required to be co-author of the submission but not necessary first author. Eligible are students at Master or PhD level and PostDocs at universities and academic institutions. It is allowed to apply for other travel grants as well.
If you have been awarded a travel grant, you will get detailed instructions per email. Only actual expenses will be reimbursed up to the amount of the grant. For this, you have to send us the original receipts after the workshop. If you receive a travel grant from another source, you can use both as long as the travel grants reimburse different expenses. For instance, you can use one grant for the conference fee and the second for the accommodation.


Prof. Michael S. Ryoo is an Assistant Professor in the Department of Computer Science at Indiana University Bloomington. His research interest is within the areas of Computer Vision and Robotics, with a particular emphasis on human activity recognition/learning, first-person vision, and robot action learning. Before joining IU, Dr. Ryoo was a staff researcher at the Robotics Section of the NASA's Jet Propulsion Laboratory (JPL) from 2011 to 2015. Dr. Ryoo received the Ph.D. degree from the University of Texas at Austin in 2008, and the B.S. degree from the Korea Advanced Institute of Science and Technology (KAIST) in 2004. His paper on robot-centric activity recognition at ICRA 2016 won the Best Vision Paper award, and his HRI 2015 paper was one of the two nominees for its Best Enabling Technology award. In 2017, he founded a computer vision startup company, EgoVid Inc., focusing on privacy-preserving recognition of humans, objects, and their activities. Prof. Dariu M. Gavrila received the PhD degree in computer science from the University of Maryland at College Park, USA, in 1996. From 1997 till 2016 he has been with Daimler R&D in Ulm, Germany, where he eventually became a Distinguished Scientist. In 2010, he was appointed professor at the University of Amsterdam, chairing the area of Intelligent Perception Systems (part-time). Since 2016 he heads the Intelligent Vehicles and Cognitive Robotics section at the TU Delft as a Full Professor. Over the past 15 years, Prof. Gavrila has focused on visual systems for detecting humans and their activity, with application to intelligent vehicles, smart surveillance and social robotics. He led the multi-year pedestrian detection research effort at Daimler, which was incorporated in the Mercedes-Benz S-, E-, and C-Class models (2013-2014). He is frequently cited in the scientific literature and received the I/O 2007 Award from the Netherlands Organisation for Scientific Research (NWO) and the IEEE Intelligent Transportation Systems Application Award 2014.


13:15-13:30 Introduction 13:30-14:00 Invited Talk Robots Anticipating Future Scene Michael S. Ryoo, Indiana University 14:00-15:05 Oral Session Action Anticipation by Predicting Future Dynamic Images Cristian Rodriguez (Australian National University), Basura Fernando (Australian National University), Hongdong Li (Australian National University) Joint Future Semantic and Instance Segmentation Prediction Camille Couprie (Facebook AI Research), Pauline Luc (Facebook AI Research), Jakob Verbeek (INRIA) Context Graph based Video Frame Prediction using Locally Guided Objective Prateep Bhattacharjee (Indian Institute of Technology Madras), Sukhendu Das (Indian Institute of Technology Madras) Predicting Action Tubes Gurkirt Singh (Oxford Brookes University), Suman Saha (Oxford Brookes University), Fabio Cuzzolin (Oxford Brookes University) Forecasting Hands and Objects in Future Frames Chenyou Fan (Indiana University), Jangwon Lee (Indiana University), Michael S Ryoo (Indiana University) 15:05-15:40 Poster Session including Coffee When will you do what? - Anticipating Temporal Occurrences of Activities Yazan Abu Farha (University of Bonn), Alexander Richard (University of Bonn), Juergen Gall (University of Bonn) Motion Prediction with Gaussian Process Dynamical Models and Trajectory Optimization Philipp Kratzer (University of Stuttgart), Marc Toussaint (University of Stuttgart), Jim Mainprice (University of Stuttgart, MPI-IS) R2P2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting Nicholas Rhinehart (Carnegie Mellon University), Kris M. Kitani (Carnegie Mellon University), Paul Vernaza (Carnegie Mellon University) Action Anticipation with RBF Kernelized Feature Mapping RNN Yuge Shi (Australian National University), Basura Fernando (Australian National University), Richard Hartley (Australian National University) Predicting Future Instance Segmentations by Forecasting Convolutional Features Camille Couprie (Facebook AI Research), Pauline Luc (Facebook AI Research), Yann LeCun (Facebook AI Research), Jakob Verbeek (INRIA) 15:40-16:20 Poster Session including Coffee Learning to Forecast and Refine Residual Motion for Image-to-Video Generation Long Zhao (Rutgers University), Xi Peng (Binghamton University), Yu Tian (Rutgers University), Mubbasir Kapadia (Rutgers University), Dimitris Metaxas (Rutgers University) Deep Video Generation, Prediction and Completion of Human Action Sequences Haoye Cai (Hong Kong University of Science and Technology, Stanford University), Chunyan Bai (Hong Kong University of Science and Technology, Carnegie Mellon University), Yu-Wing Tai (Tencent Youtu), Chi-Keung Tang (Hong Kong University of Science and Technology) Adversarial Geometry-Aware Human Motion Prediction Liang-Yan Gui (Carnegie Mellon University), Yu-Xiong Wang (Carnegie Mellon University), Xiaodan Liang (Carnegie Mellon University), José M. F. Moura (Carnegie Mellon University) Embarrassingly Simple Model for Early Action Proposal Marcos Baptista-Ríos (University of Alcalá), Roberto Lopez-Sastre (University of Alcalá), Francisco J. Acevedo-Rodríguez (University of Alcalá), Saturnino Maldonado-Bascon (University of Alcalá) Am I done? Predicting Action Progress in Video Federico Becattini (University of Florence), Lorenzo Seidenari (University of Florence), Tiberio Uricchio (University of Florence), Alberto Del Bimbo (University of Florence), Lamberto Ballan (University of Padova) A Novel Semantic Framework for Anticipation of Manipulation Actions Fatemeh Ziaeetabar (University of Göttingen), Minija Tamosiunaite (University of Göttingen), Florentin Wörgötter (University of Göttingen) 16:20-16:50 Invited Talk Vulnerable Road User Path Prediction Dariu M. Gavrila, Delft University of Technology 16:50-17:45 Oral Session RED: A simple but effective Baseline Predictor for the TrajNet Benchmark Stefan Becker (Fraunhofer IOSB), Ronny Hug (Fraunhofer IOSB), Wolfgang Hübner (Fraunhofer IOSB), Michael Arens (Fraunhofer IOSB) Convolutional Neural Network for Trajectory Prediction Nishant Nikhil (Indian Institute of Technology Kharagpur), Brendan Morris (University of Nevada Las Vegas) Leader’s Gaze Behaviour and Alignment of the Action Planing from the Follower’s Gaze Cues in Human-Human and Human-Robot Interaction Nuno Duarte (Instituto Superior Técnico), Mirko Rakovic (Instituto Superior Técnico), Jorge Marques (Instituto Superior Técnico), José Santos-Victor (Instituto Superior Técnico) Group LSTM: Group Trajectory Prediction in Crowded Scenarios Niccolò Bisagno (Università di Trento), Bo Zhang (Dalian Maritime University), Nicola Conci (UNITN) 17:45-18:00 Closing Remarks & Discussion


Prof. Juergen Gall University of Bonn Prof. Jan van Gemert Delft University of Technology Prof. Kris Kitani Carnegie Mellon University

Program committee

Sergio Escalera University of Barcelona, Spain David Fouhey UC Berkeley, United States of America Hildegard Kühne University of Bonn, Germany Thomas B. Moeslund Aalborg University, Denmark Silvia Pintea Delft University of Technology, The Netherlands Nick Rhinehart Carnegie Mellon University, United States of America Michael S. Ryoo Indiana University Bloomington, United States of America Leonid Sigal University of British Columbia, Canada Gul Varol INRIA, France Carl Vondrick Google, United States of America Jiajun Wu Massachusetts Institute of Technology, United States of America Angela Yao University of Bonn, Germany


Prof. Dr. Jürgen Gall

University of Bonn

Institute of Computer Science

Computer Vision

Group Endenicher Allee 19a

53115 Bonn, Germany


Liability note

Despite careful content control we assume no liability for the content of external links. For the corresponding content of linked pages, their administrators are responsible.

Image source: Historisches Oktoberfest 2010 © digital cat,