This course is an AI-hype free zone.

If robots are to move into human environments, such as homes, schools, workplaces, and public spaces, how can we design robotic systems for human interaction?

This course provides a deep dive into computational methods in human-robot interaction (HRI), focusing on probabilistic AI for robot reasoning and decision-making and reinforcement learning (RL), as they are used in the HRI literature.  Students also learn how to design experiments to evaluate HRI systems.

Lectures cover key algorithms in Probabilistic Robotics, including Bayesian Networks, Markov Models, HMMs, Kalman and Particle Filters, MDPs and POMDPs, Monte Carlo Reinforcement Learning, Q-Learning, and approximation methods.  Lectures on evaluating human-robot interaction systems include Experimental Design Methods, Measures and Metrics, and Planning and Running Experiments. Lectures emphasize the development of a strong intuition and deep understanding of underlying concepts, although practical issues will also be studied through programming assignments.

Throughout the semester, we will discuss seminal and recent papers in HRI that make use of the learned methods and techniques, covering the following topics: reasoning about human intentions, generating intentional and legible action, social navigation around humans,  nonverbal interaction including gestures and gaze, teamwork and collaboration, learning from humans, and formal methods for HRI. Students present papers in class and work in teams on an HRI research project.

Learning Outcomes

  • Find, read, and comprehend a technical HRI Research Paper
  • Develop a deep understanding of key probabilistic algorithms driving computational HRI
  • Implement a human-robot interaction system in ROS
  • Plan a human-subject study involving a human and a robot
  • Present a research paper in a 20 minute conference-style presentation
  • Critically review a paper and comment on its advantages and shortcomings
  • Critical thinking

Time and Location: MWF 12:20-1:10pm, Hollister Hall 362.

Prerequisites:  Graduate standing. Seniors need permission of instructor. Python programming experience. This seminar-style project-based course is intended for PhD, MS, MEng, and MPS students from a variety of disciplines, including MAE, CS, ECE, and IS.

Required Readings:

  • Computational Human-Robot Interaction. Thomaz, Hoffman, & Cakmak, Foundations and Trends in Robotics. Vol 4: No. 2-3. Now Publishers, 2016.
  •  Artificial Intelligence: A Modern Approach. 4th Ed. Russell & Norvig, Prentice Hall, 2020.
  • Reinforcement learning: An introduction (2nd ed.). Sutton & Barto. MIT Press, 2018. [Online]

Additional Readings:

  • Probabilistic Robotics. Thrun, Burgard, & Fox, MIT Press, 2005.
  • Programming Robots with ROS: A Practical Introduction to the Robot Operating System. Quigley, Gerkey, & Smart, O’Reilley, 2015.

Papers and other material are available on the Reading List page.


The grade will be determined based on the following:

  • Paper presentations (20% of the final grade): Several lectures will be devoted to paper presentations by students. These papers, representing seminal and current work in the field, will complement the material covered in class.   An online sign-up sheet will be sent out after the first class for students  and students need to sign up at the beginning of the second week. All substitutions must be cleared at least 10 days prior to the presentation.
  • Assignments (25% of the final grade): Quizzes and programming projects will be assigned during the time period prior to your project proposal.  These will review material learned in class and also include introductory exercises to familiarize students with programming a simulated robot in Python with the ROS programming environment.  They are meant as preparation for conducting a more complex project/experiment for the final project in the class.
  • Class participation (10% of the final grade): At each class meeting, students who are not presenting are expected to read papers and participate in the in-class discussion of the paper.  Both attendance and lively participation will be noted.
  • Paragraph + discussion question (15% of the final grade):  To facilitate class participation, students will submit short writing assignments. The week that you are presenting a paper you are excused from this writing assignments. There are no late assignments accepted. You need to submit one paragraph about one of the papers each week to a discussion forum, in addition to a discussion question about a paper presented in class.
  • Final project (30% of the final grade):  Students will develop a final research project. Students will adapt or extend a method or algorithm from one of the papers presented in class, implement it on a physical robot, and plan a study evaluating their algorithm. Students will work in pairs or groups of three.  Several robot platforms will be introduced in class and made available for your use in these projects.  You may use an alternate robot platform with permission of the instructor.  A project proposal is due on October 27 and will be reviewed by the instructor.  The project will conclude with an oral presentation to the class during the final week of classes, as well as a written report.

Late policy: For fairness to all students, neither late work, nor late arrival to classes will be accepted. In case of exceptional circumstances, contact the instructor immediately, at least 24 hours before class time or assignment due time.

Academic Integrity: Students are expected to follow Cornell’s Code of Academic Integrity which can be found at http://cuinfo.cornell.edu/aic.cfm. The purpose of this code is to provide for an honest and fair academic environment.

List of topics:

  • Introduction: Human-Robot Interaction (HRI)
  • AI Track
    • Key Concepts in Probability Theory
    • Bayesian Networks and Filtering
      • Bayesian Networks
      • Markov Models
      • Hidden Markov Models (HMMs)
      • Bayes Filtering
      • Kalman Filters
      • Particle Filters
    • Sequential Decision-Making
      • Markov Decision Processes (MDPs)
      • Dynamic Programming
      • Partially Observable Markov Decision Processes (POMDPs)
    • Reinforcement Learning
      • Monte Carlo Method
      • Temporal Difference Learning
      • Q-Learning
      • RL by Approximation
  • Paper Track
    • Intentional Action
      • Intention Recognition
      • Intention Expression
      • Legible Robot Motion
    • Nonverbal Behavior in HRI
      • Social Navigation
      • Gestures and Body Language
      • Gaze and Eye Contact
    • Collaboration
      • Theoretical Foundations of Collaboration
      • Human-Robot Collaboration
      • Handovers
    • Learning in HRI
      • Learning from Demonstration
      • Socially Guided Robot Learning
  • Experiments Track
    • Experimental Design
    • Metrics and Measurement
    • Planning and Running Experiments

Expectations: You can expect the instructor to start and end class on time, as well as grade your performance and send you feedback on your presentations in a timely manner.  In turn, we can expect you to come to class on time, be attentive and engaged in class, and refrain from using laptops, cell phones and other electronic devices during class. I also expect you to spend an adequate amount of time on the readings each week (estimated: 2–3 hours) and on your final assignment (estimated: 40–60 hours).