Learning models of language, action and perception
for human-robot collaboration

Dr. Stefanie Tellex
Department of Computer Science, Brown University

4:00pm Monday, 7 March 2016, ITE325b

Robots can act as a force multiplier for people, whether a robot assisting an astronaut with a repair on the International Space station, a UAV taking flight over our cities, or an autonomous vehicle driving through our streets.  To achieve complex tasks, it is essential for robots to move beyond merely interacting with people and toward collaboration, so that one person can easily and flexibly work with many autonomous robots.  The aim of my research program is to create autonomous robots that collaborate with people to meet their needs by learning decision-theoretic models for communication, action, and perception.  Communication for collaboration requires models of language that map between sentences and aspects of the external world. My work enables a robot to learn compositional models for word meanings that allow a robot to explicitly reason and communicate about its own uncertainty, increasing the speed and accuracy of human-robot communication.  Action for collaboration requires models that match how people think and talk, because people communicate about all aspects of a robot’s behavior, from low-level motion preferences (e.g., “Please fly up a few feet”) to high-level requests (e.g., “Please inspect the building”).  I am creating new methods for learning how to plan in very large, uncertain state-action spaces by using hierarchical abstraction.  Perception for collaboration requires the robot to detect, localize, and manipulate the objects in its environment that are most important to its human collaborator.  I am creating new methods for autonomously acquiring perceptual models in situ so the robot can perceive the objects most relevant to the human’s goals. My unified decision-theoretic framework supports data-driven training and robust, feedback-driven human-robot collaboration.

Stefanie Tellex is an Assistant Professor of Computer Science and Assistant Professor of Engineering at Brown University.  Her group, the Humans To Robots Lab, creates robots that seamlessly collaborate with people to meet their needs using language, gesture, and probabilistic inference, aiming to empower every person with a collaborative robot.  She completed her Ph.D. at the MIT Media Lab in 2010, where she developed models for the meanings of spatial prepositions and motion verbs.  Her postdoctoral work at MIT CSAIL focused on creating robots that understand natural language.  She has published at SIGIR, HRI, RSS, AAAI, IROS, ICAPs and ICMI, winning Best Student Paper at SIGIR and ICMI, Best Paper at RSS, and an award from the CCC Blue Sky Ideas Initiative.  Her awards include being named one of IEEE Spectrum’s AI’s 10 to Watch in 2013, the Richard B. Salomon Faculty Research Award at Brown University, a DARPA Young Faculty Award in 2015, and a 2016 Sloan Research Fellowship.  Her work has been featured in the press on National Public Radio and MIT Technology Review; she was named one of Wired UK’s Women Who Changed Science In 2015 and listed as one of MIT Technology Review’s Ten Breakthrough Technologies in 2016.

Note: This story has been updated from the original February 3 version.

UMBC Mock Trial has earned its second consecutive bid to the American Mock Trial Association’s Opening Round Championships (ORCs). The major accomplishment comes after the team earned the second overall bid out of 24 teams at the Baltimore Regional Tournament held February 27-28 at the University of Maryland.

In addition to the team’s overall success, “A” Team attorney Kayla Smith and witness Zachary Garmoe earned All-Region Awards, and “B” Team witness Lily Felber also earned an All-Region Award for her portrayal of the defendant. The “B” Team also won or tied on multiple ballots and had several close rounds against strong national competition.

“What the UMBC students accomplished this past weekend is nothing short of extraordinary,” shares UMBC Mock Trial Coach Ben Garmoe ’13, political science. “The students worked as a team on a level I had not seen this year; simply put, they found another gear. I am beyond excited for the future of the team and program.”

The ORCs will be held in Washington, D.C. on March 19-20 at the E. Barrett Prettyman Federal District Courthouse. UMBC Mock Trial will join teams from around the region to compete for a bid to the National Championship Round.

Earlier this year, UMBC Mock Trial’s “A” team finished in 1st place at the inaugural Charm City Classic Mock Trial Invitational, held January 30-31 at Stevenson University. The tournament included teams from UMBC, Stevenson, Johns Hopkins University, and the University of Maryland.

“The Charm City Classic is exciting not just for the results – this was the first time that UMBC hosted a mock trial tournament. We co-hosted the Classic with Stevenson University, and the plan is to make this a yearly event. Hosting an invitational will continue to grow UMBC Mock Trial on a national level as we seek to improve and achieve,” shared UMBC Mock Trial President Cheyenne Smith ’16, biology and psychology after the tournament.

The UMBC Mock Trial “B” Team also competed in the tournament and won the Spirit of the American Mock Trial Association (AMTA) award, an honor given to the team that best exemplifies the spirit of collegiality and professionalism at a tournament.

Additional honors for the “B” team included an Outstanding Witness award given to witness Lily Felber, in her first year of mock trial, for her portrayal of the defendant. “A” team members Kayla Smith and Zachary Garmoe also won Outstanding Witness awards.

For more information on the Charm City Classic and to view additional photos, read “Stevenson, UMBC host inaugural Charm City Classic Invitational Tournament” on Stevenson University’s news site.

Header Image: UMBC Mock Trial “A” team at the Baltimore Regional Tournament. Photo courtesy Ben Garmoe. 

Is your trivia team neck and neck with your arch rivals? If members of your trivia team come to Flat Tuesdays Karaoke Night on March 1st from 6:00pm - 8:00pm in Upper Flat Tuesdays and sing a song, you will receive a free trivia question and answer. It could be the difference between first place champions and sub-par second place.

*Team members must sing a full song to receive the hint.

*If two different team members sing different songs, they can receive two different hints.

**All patrons must be 21 years or older to enter Upper Flat Tuesdays and must present a valid government issues ID and their UMBC ID card to be admitted. Guests are only permitted with the appropriate hosts.