Smooth Imitation Learning for Online Sequence Prediction

Authors

Hoang Le (California Institute of Technology)
Andrew Kang (California Institute of Technology)
Yisong Yue (California Institute of Technology)
Peter Carr (Disney Research Pittsburgh)

International Conference on Machine Learning (ICML) 2016

June 19, 2016

Smooth Imitation Learning for Online Sequence Prediction-Image2

We study the problem of smooth imitation learning for online sequence prediction, where the goal is to train a policy that can smoothly imitate demonstrated behavior in a dynamic and continuous environment in response to online, sequential context input. Since the mapping from context to behavior is often complex, we take a learning reduction approach to reduce smooth imitation learning to a regression problem using complex function classes that are regularized to ensure smoothness. We present a learning meta-algorithm that achieves fast and stable convergence to a good policy. Our approach enjoys several attractive properties, including being fully deterministic, employing an adaptive learning rate that can provably yield larger policy improvements compared to previous approaches, and the ability to ensure stable convergence. Our empirical results demonstrate significant performance gains over previous approaches.

 

 

Download Smooth Imitation Learning Dataset

The dataset contains 47 training sequences and 2 test sequences, encoded as tab-delimited text files. Columns 1-14 represent the feature elements at each time instant, and column 15 is the broadcast camera pan angle (in degrees).

Download File "Smooth Imitation Learning for Online Sequence Prediction-Paper"
[pdf, 942.40 KB]

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.