Overview

Robots are moving from factory floors, battlefields, and space into human environments such as homes, offices, schools, and hospitals. How can we design robotic systems for human interaction?

In this course, students study core engineering, computational, and experimental methods in human-robot interaction (HRI). We cover robot perception, reasoning, AI, and machine learning aspects. 

We discuss seminal and recent papers on social navigation, teamwork and collaboration, machine learning with humans in the loop, embodied verbal and nonverbal dialog, and more.

AI and Machine Learning topics include: Bayes Nets, Markov Models, HMMs, Kalman and Particle Filters, MDPs, Supervised, Unsupervised and Reinforcement Learning.

Students also learn how to design experiments to evaluate HRI systems, including Experiment Design, Inferential Statistics, Correlation, T-Test, ANOVA, and regression.

Students present papers in class and work in teams on an HRI research project development and evaluation.

Learning Outcomes

  • Find, read, and comprehend a technical HRI Research Paper
  • Familiarize yourselves with the main probabilistic algorithms driving computational HRI
  • Implement a human-robot interaction system in ROS
  • Plan and execute a human-subject study
  • Analyze the results of a study using inferential statistics
  • Present a research paper in a 20 minute conference-style presentation
  • Critically review a paper and comment on its advantages and shortcomings
  • Critical thinking!

 

As an academic experiment,  this course is run in  coordination between UT Austin (ECE 382V, Prof. Andrea Thomaz) and Cornell (MAE 6710, Prof. Guy Hoffman).  This website represents the joint information and resources shared between the two courses.   Additional details specific to each course will be made available by each instructor separately.

Cornell Time and Location: TR 10:10–11:25am, Upson 216.

UT Austin Time and Location: TR 12:30–2:00pm, ECJ 1.308.

Slack Team: hri17 – [signup link]

Prerequisites:  Python programming experience. This seminar style course is intended for PhD, M.S., and M.Eng. students from a variety of disciplines, including MAE/ME/ASE, ECE, CS, and IS.

Required Readings:

  • Computational Human-Robot Interaction. Thomaz, Hoffman, & Cakmak, Foundations and Trends in Robotics, Now Publishers, 2016.
  •  Artificial Intelligence: a Modern Approach. Russell & Norvig, Prentice Hall, 2009.
  • Discovering Statistics Using R. Field, Miles, & Field, Sage, 2012.

Additional Readings:

  • Probabilistic Robotics. Thrun, Burgard, & Fox, MIT Press, 2005.
  • Programming Robots with ROS: A Practical Introduction to the Robot Operating System. Quigley, Gerkey, & Smart, O’Reilley, 2015.

Papers and other material are available on the Reading List page.

Grading

The grade will be determined based on the following:

  • Paper presentations (25% of the final grade): Ten lectures will be devoted to paper presentations by students. These papers, representing seminal and current work in the field, will complement the material covered in class.   Due to size of the class, papers will be presented in teams.  A doodle poll will be sent out after the first class for students to sign up, and students need to sign up before the second meeting. Grading of this assignment will follow the rubrics posted on the course site.  Papers will be assigned according to the reading list; however, students may choose to present a different paper related to the same topic. All substitutions must be cleared at least 10 days prior to the presentation.
  • Programming assignments (10 % of the final grade): Three programming projects will be assigned during the time period prior to the midterm exam.  These will be introductory exercises to familiarize students with programming a simulated robot in Python with the ROS programming environment.  These are meant as preparation for conducting a more complex project/experiment for the final project in the class.
    Midterm Exam (25% of the final grade):  This will be an in-class exam covering both the foundation lectures and the papers presented by students.  You are permitted to bring any number of written pages of notes to use during the exam.
  • Class participation (10% of the final grade): At each class meeting, students who are not presenting are expected to read the paper and participate in the in-class discussion of the paper.  Both attendance and participation will be noted.
  • Paragraph + discussion question (Pass / Fail):  To facilitate class participation, students will submit writing assignments. The week that you are presenting a paper you are excused from this writing assignments. There are no late assignments accepted, and students may miss one submission without penalty. Cornell students: submit one paragraph about one paper each week to a discussion forum, in addition to a discussion question about a paper presented in class, except when they present a paper. UT Austin students: one paragraph and one discussion question will be due (on Canvas) the night before every class in which papers are being discussed.
  • Final project (30% of the final grade):  Development of a research project by students. Students have the choice of (a) extending an existing method or algorithm from one of the papers presented in class and implementing it; (b) directly implementing a method or algorithm from one of the papers and running a study evaluating it; or (c) proposing a novel algorithm to implement and experiment with.  Students will work in pairs.  A robot platform will be introduced in class and made available for your use in these projects.  You may use an alternate robot platform with permission of the instructor.  A project proposal is due on Nov 1 and will be reviewed by the instructor.  The project will conclude with an oral presentation to the class during the final exam period, as well as a written report.
  • Cornell Only: Joint projects with Formal Methods (MAE 6770): Students who are interested in taking Formal Methods for Robotics (MAE 6770) and Human-Robot Interaction: Algorithms & Experiments (MAE 6710) are encouraged to do one project for both classes on the emerging topic of “Verifiable HRI”.

Late policy: For fairness to all students, neither late work, nor late arrival to classes will be accepted. In case of exceptional circumstances, contact the professor or TA  immediately in due time.

List of topics:

  • Introduction: Human-Robot Interaction (HRI)
  • Key Concepts in Probability Theory
  • Intentional Action
    • Bayesian Networks
    • Markov Models
    • Hidden Markov Models (HMMs)
    • Intention Recognition
    • Intention Expression
    • Legible Robot Motion
  • Social Navigation
    • Kalman Filters
    • Particle Filters
    • Social Navigation
  • Nonverbal Behavior in HRI
    • Gestures and Body Language
    • Gaze and Eye Contact
  • Collaboration
    • Markov Decision Processes (MDPs)
    • Partially Observable Markov Decision Processes (POMDPs)
    • Theoretical Foundations of Collaboration
    • Human-Robot Collaboration
    • Handovers
    • Multimodal Dialog
  • Learning in HRI
    • Clustering
    • Supervised Learning
    • Reinforcement Learning
    • Learning from Demonstration
    • Socially Guided Robot Learning
  • HRI Experiments
    • Experimental Design
    • Metrics and Measurement
    • Inferential Statistics
    • T-test
    • General Linear Model (GLM)
    • Analysis of Variance (ANOVA)
  • Applications for HRI
    • Socially Assistive Robots
    • HRI Ethnographies
    • HRI in Commercial Spaces

Expectations: You can expect the instructor to start and end class on time, as well as grade your performance and send you feedback on your presentations in a timely manner.  In turn, we can expect you to come to class on time, be attentive and engaged in class, and refrain from using laptops, cell phones and other electronic devices during class. I also expect you to spend an adequate amount of time on the readings each week (estimated: 2–3 hours) and on your final assignment (estimated: 40–60 hours).