In EASE we investigate how robots could perform everyday activities such as cooking as dextrous as humans do. As an interdisciplinary project, EASE touches many aspects of human and robot movement and task solving around mundane household tasks. Our lab is home to subprojects H02 and P01: We focus on knowledge representation through ontologies, learning through imitation and modeling human behavior with serious games, as well as building artificial mental simulations as a tool for robots to demonstrate they understand a task.
In close collaboration with other labs and subprojects, we aim to build a pipeline for understanding natural language instructions. This pipeline parses instructions such as “Cover the pot with the lid!” into semantic structures, which are annotated using an ontology. These annotations help to know that for example a pot is some sort of container and that Cover! is the action to perform. With this information, we can tap into a knowledge base we build from observing humans doing similar tasks in serious games. In this case, we might find information on how to cover cups or plates, but might not know how to cover a pot. With a few more reasoning steps it is possible to find similarities between cup covering and pot covering, pathing the way to simulate covering a pot in a mental simulation. By running the simulation, the robot finally demonstrates it understood the task and could execute it now.