Course Project

The course project is your chance to gain hands on HRI research experience. You are expected to complete this project in a group of 2 to 4. This project must involve conducting a human-subjects study that answers a novel research question focused on the interaction between a physically embodied robot and 1 or more people. This page contains details on the project deliverables, due dates, as well as examples of prior projects (at the bottom of the page).

Project Deliverables & Due Dates

---

Project Timeline

Submission of Project Deliverables

Put your project pitches in this Google doc. All other project deliverables will be submitted on Canvas.

Project Pitches

Before pitching project ideas, read this entire web page thoroughly to get the full picture of what this project will entail.

Although the project will be completed in teams of 2-4, these pitches are made by individuals. The purpose of these project pitches is to aid in the formation of project teams (your project proposal can and will deviate from these project pitches, again, the project pitches are meant to serve as a starting point for team formation). In this Google document you’ll pitch 1-3 project ideas that you’d be interested in pursuing for the course project. It does not need to be fully formed, but enough to give your classmates a good idea of what the project will entail.

Draft Project Proposal

The draft proposal should be 1-2 pages in length and describe:

1. The research question(s) you are investigating and any related hypotheses
2. The methods you expect to use to answer your research question (e.g., experimental conditions, experiment protocol, questionnaire measures, measurements of human behavior)
3. How you plan to evaluate whether or not you've answered your research question (e.g., statistical analysis, thematic analysis, comparison against a state-of-the-art approach)

Finalized Project Proposal

The finalized project proposal, in addition to what is contained in the draft proposal, should detail a detailed timeline with corresponding intermediate deliverables. Before submitting your final project proposal, please set up a meeting with a member of the teaching team in the range of Wed-Fri, April 9-11. You'll receive an email from me on how to sign up for a time to meet with us to discuss your project proposal.

Progress Reports

The progress report should contain:

1. An update on your progress relative to your project proposal timeline (1 paragraph)
2. The accomplishments you've made since the last project deliverable (1 paragraph or a bullet point list)
3. Any unexpected roadblocks you've encountered (1-2 paragraphs or a bullet point list)
4. An updated timeline of how you plan to complete the project

Literature Review (For Students Taking 30630)

If you are taking the grad level of the course (30630), you will write and submit a literature review of prior work that is relevant to your project. If there are multiple students on your project team taking 30630, each of you must write and submit an independent literature review. This literature review should resemble the background sections of the papers we read in class. Your literature review should:

• Contain at least 15 references to prior related work,
• Summarize the prior work as it relates to the context of your project, and
• Highlight the gaps in prior work (that your work will then fill).

The literature review should be written in the style of an academic conference paper (like the papers we've been reading for our class discussions), feel free to use the ACM templates for conferences.

Project Presentation

You will present the work that you've done in your course project during finals period. You should aim for your presentation to take 10-12 minutes, leaving 3-5 minutes for Q&A and transitioning to the next group (I will plan to cut you off if your presentation goes beyond 12 minutes). All members of your team are expected to participate in the presentation. Your slides should be uploaded to Canvas by the start of the class meeting where you will present. Your presentation should highlight the following:

• Motivation: Detail the research question(s) you were interested in investigating and why it's important to explore.
• Background: Describe any relevant background literature and tell us what unique and novel contributions your study is making beyond what already exists in the literature.
• Methods: Describe your study design (experimental conditions), hypotheses, study protocol (videos are great for this), and the measures you used to assess your hypotheses.
• Video: Show video clip(s) giving example(s) of the interactions participants had with the robot in each of your experimental conditions.
• Results: Report on the results you found. Please include all of the main measures, regardless of whether or not you found significant differences.
• Discussion/Implications: Reflect on the broader implications and takeaways from your results.
• Learning Outcomes: Enumerate some "lessons learned" from this project, what you learned by going out and actually conducting HRI research, beyond what you've learned from reading the papers in this class.

Your presentation will also be graded for its design and style. Please ensure that your slides are NOT too text heavy. If the audience is spending their time trying to read a lot of text on a slide, it's hard for them to pay attention to you. Use visuals and limited text on your slides, describing the details verbally during your presentation.

Final Report

The project report should be written in the style of an academic conference paper (like the papers we've been reading for our class discussions), feel free to use the ACM templates for conferences. The report should have the following sections: abstract, introduction, background, methods, results, discussion, conclusion, and contributions. The abstract - conclusion sections should resemble the papers we have read and discussed in this course. The contributions section should detail the contributions of each team member, detailing which portions of the project were completed by each team member. The report should be approximately 6 pages long without references.

Teaming Survey

Due at the same time as your final report, ensure that you've filled out the course project teaming survey. Assuming that all team members make approximately equal contributions on the course project, all team members will receive the same grade. However, if certain team member(s) do not contribute a sufficient amount of work to the course project, each team member may receive a different grade that reflects their individual contributions.

Data Sharing

If you have not done so already, please share all of the data you've collected in your HRI project in a UChicago Box folder and add the teaching team to the shared folder. This includes questionnaire data, video data, and any other data you collected in your project.

Project Grading

---

The grading for the course project will be as follows:

• 15% intermediary deliverables - project pitches, proposals, progress reports
• 10% final project presentation
• 25% final report
• 50% project work - your grade based on the work that you accomplished during the class project is not dependent on your project being a 'success', your project work will be graded based on the project's scope, novelty, and the investigative work that you put into pursuing your research questions

For further details on project grading, please check out the final project grading rubric.

Helpful Information for Running Your Human-Subjects Study

---

Suggested Timeline

For running an in-person human subjects study, here's an example timeline of what your project work might look like:

• Weeks 1-3: Form project team, finalize study design (main research questions, experimental conditions, protocol, measures)
• Week 4: Develop study protocol, questionnaires, consent forms, etc. and submit your IRB application
• Week 5: Program the robot and go back-and-forth with the IRB if they request any changes to your IRB application
• Week 6: Pilot the study
• Weeks 7-8: Run human subjects
• Week 9: Conduct data analysis
• Week 10: Work on presentation and final report

Please note that the quarter is extremely short, you will have to be on top of it from day 1.

Human Subjects Protection Training

Since you will be conducting a human subjects study in your class project, you will need to submit an IRB (Internal Review Board) Application and have completed Human Subjects Protection Training (← that link contains all the info you need to complete the training). Before you submit your IRB application, you must have already completed the human subjects protection training.

UChicago Internal Review Board (IRB) Application

In order to conduct human subjects research in affiliation with UChicago, you must submit and get approved an Internal Review Board (IRB) application. This means that BEFORE you run any human subjects, you MUST have your IRB application approved. For HRI related research, we submit our IRB applications to UChicago's Social and Behavioral Sciences Institutional Review Board (SBS IRB). Their main website, https://sbsirb.uchicago.edu/, contains helpful information, templates, and guidance for completing an IRB application.

The purpose of the IRB is to ensure that UChicago research is being conducted ethically and with proper respect to human participants. Your IRB application will include:

• a high-level description of the research you plan to conduct
• a protocol specifying what each human participant will do in your research study
• a consent form that each participant will read and agree to before beginning your research study
• a description of how you plan to protect any identifiable data you collect from human participants and how you plan to store the data
• any questionnaires you plan to administer to the human participants
• any recruitment methods you plan to use (e.g., flyers, social media posts, emails)
• details on how participants are to be paid and how much they will be paid

To create, edit, submit, and amend your IRB application go to https://aura.uchicago.edu/ and click on the "IRB" button.

It is most helpful to model your IRB application after one that has already been approved. Please ask the teaching staff to provide you with model IRB application examples to help you as you create your own.

When you complete your own IRB, please put me (Sarah) as the PI and all other teaching team members as collaborators on the personnel list. Since I (Sarah) will be the PI for the IRB, this means that I must be the one to hit the "submit" button whenever you submit the IRB application or revisions to your IRB application. Please email me when you are ready to make an IRB submission, and I will hit that "submit" button.

HRI Measurement Scales

HRI researchers at the Naval Research Lab and George Mason University have compiled a database of scales HRI researchers use in their studies. I highly encourage you to visit their website Finding the "Perfect" Scale (especially their database page) where you can find questionnaire scales you can use for your study as well as a rating of the quality of each scale.

Qualtrics

If your study involves asking human participants to fill out a questionnaire, I recommended that you use Qualtrics. In order to get all the nice bells and whistles that comes with Qualtrics, we highly recommend that you request a pro account. You can do this by completing this User Agreement Form (for more context, see this UChicago ITS post about using Qualtrics at UChicago).

Recruitment

You will be responsible for recruiting your own participants. You can post on social media, send emails, post on UChicago organization sites, etc. Just be sure that you include all forms of recruitment in your IRB application that you plan to use. We recommend you try to recruit the following minimum number of participants, dependent on your group size:

• Group of 2: at least 16 participants
• Group of 3: at least 20 participants
• Group of 4: at least 24 participants

Robot

Payment

For in-person human participants, we will pay them at the rate of $12.00 per hour in the form of Amazon gift cards. To facilitate this payment, please email the teaching team daily when you are running your study with a list of email addresses that correspond with your study participants from that day. I will ensure that they are sent their Amazon gift card for participating in your study.

Piloting Your Study

It is SO important before you start running real human subjects that you practice, practice, practice executing your human subjects study. We recommend:

1. Running the study several times yourselves from top to bottom, AND
2. Running the study one one or two friends (including having them fill out the questionnaires) and asking them if anything was confusing or if they noticed any typos in your questionnaire, AND
3. Running the study on members of the teaching team and getting their feedback.

Piloting is essential to work out any kinks and ensure that your study is set up to adequately answer your research questions.

Examples Projects from Prior Topics in HRI Student Teams

---