Abstract: Current state-of-the-art CNNs localize rare object categories in internet photos, yet, they miss basic facts that a two-year-old has mastered: that objects have 3D extent, they persist over time despite changes in the camera view, they do not 3D intersect, and others. We will discuss models that learn to map 2D and 2.5D images and videos into amodal completed 3D feature maps of the scene and the objects in it by predicting views. We will show the proposed models learn object permanence, have objects emerge in 3D without human annotations, support grounding of language in 3D visual simulations, and learn object dynamics that generalize across scene arrangements and camera placements.

Biography: Katerina Fragkiadaki is an Assistant Professor in the Machine Learning Department in Carnegie Mellon University. She received her Ph.D. from University of Pennsylvania and was a postdoctoral fellow in UC Berkeley and Google research after that. Her work is on learning visual representations with little supervision and on combining spatial reasoning in deep visual learning. Her group develops algorithms for mobile computer vision, learning of physics and common sense for agents that move around and interact with the world. Her work has been awarded with a best Ph.D. thesis award, an NSF CAREER award, AFOSR YIP award, Google, TRI, Amazon and Sony faculty research awards.

Abstract: In the last few years, the ability for robots to understand and operate in the world around them has advanced considerably. Examples include the growing number of self-driving car systems, the considerable work in robot mapping, and the growing interest in home and service robots. However, one limitation is that robots most often reason and plan using very geometric models of the world, such as point features, dense occupancy grids and action cost maps. To be able to plan and reason over long length and timescales, as well as planning more complex missions, robots need to be able to reason about abstract concepts such as landmarks, segmented objects and tasks (among other representations). I will talk about recent work in joint reasoning about semantic representations and physical representations and what these joint representations mean for planning and decision making.

Biography: Nicholas Roy is the Bisplinghoff Professor of Aeronautics & Astronautics and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology. He received his B.Sc. in Physics and Cognitive Science and his M.Sc. in Computer Science from McGill University. He received his Ph. D. in Robotics from Carnegie Mellon University. He has made research contributions to planning under uncertainty, machine learning, human-computer interaction and aerial robotics. He is currently the director of the Bridge in MIT's Quest for Intelligence.

Abstract: A wide range of applications can benefit from robots acquiring manipulation skills by interaction with humans. In this presentation, I will discuss the challenges that such learning process encompasses, including representations for manipulation skills that can exploit the structure and geometry of the acquired data in an efficient way, the development of optimal control strategies that can exploit variations in manipulation skills, and the development of intuitive interfaces to acquire meaningful demonstrations. From a machine learning perspective, the core challenge is that robots can only rely on a small number of demonstrations. The good news is that we can exploit bidirectional human-robot interaction as a way to collect better data. We can also rely on various structures that remain the same within a wide range of robotic tasks. Such structures include geometrical aspects, by extending learning strategies that have been originally developed for standard Euclidean space to Riemannian manifolds. In robotics, these manifolds include orientation, manipulability ellipsoids, graphs and subspaces. Another type of structure that we study relates to the organization of data as multidimensional arrays (also called tensors). These data appear in various robotic tasks, either as the natural organization of sensorimotor data (tactile arrays, images, kinematic chains), or as the result of preprocessing steps (moving time windows, covariance features). Tensor factorization techniques (also called tensor methods or multilinear algebra) can be used to learn from only few tensor datapoints, by exploiting the multidimensional nature of the data. Another key challenge in robot skill acquisition is to link the learning aspects to the control aspects. Optimal control provides a framework that allows us to take into account the possible variations of a task, the uncertainty of sensorimotor information, and the movement coordination patterns, by relying on well grounded control techniques such as linear quadratic tracking, differential dynamic programming, and their extensions to model predictive controllers. The formulation draws explicit links with learning techniques, as we can recast these techniques as Gauss-Newton optimization problems formulated at trajectory level (in both control space and state space), which facilitates the links to probabilistic approaches.

Biography: Dr Sylvain Calinon is a Senior Researcher at the Idiap Research Institute (https://idiap.ch), heading the Robot Learning & Interaction group. He is also a Lecturer at the Ecole Polytechnique Federale de Lausanne (EPFL). From 2009 to 2014, he was a Team Leader at the Italian Institute of Technology. From 2007 to 2009, he was a Postdoc in the Learning Algorithms and Systems Laboratory, EPFL, where he obtained his PhD in 2007. His research interests cover robot learning, human-robot collaboration and optimal control.

Abstract: Robots for minimally-invasive surgery such as steerable needles and concentric-tube robots have the potential to dramatically alter the way common medical procedures are performed. They can decrease patient-recovery time, speed healing and reduce scarring. However, manually controlling such devices is highly unintuitive and automatic planning methods are in need. For the automation of such medical procedures to be clinically accepted, it is critical from a patient care, safety, and regulatory perspective to certify the correctness and effectiveness of the motion-planning algorithms involved in procedure automation. In this talk I will survey some of the recent work I have been involved in where, together with colleagues at UNC, UW and UoU, we developed efficient and effective planning capabilities for medical robots that provide provable guarantees on various planner attributes.

Biography: Oren Salzman is an assistant Professor at the Computer Science department at the Technion - Israel Institute of Technology. His research focuses on revisiting classical computer science algorithms, tools and paradigms to address the computational challenges that arise when planning motions for robots. Combining techniques from diverse domains such as computational geometry, graph theory and machine learning, he strives to provide efficient algorithms with rigorous analysis for robot systems with many degrees of freedom moving in tight quarters. Oren completed a PhD in the School of Computer Science at Tel Aviv University under the supervision of Prof. Dan Halperin. He then continued his studies as a postdoctoral researcher at Carnegie Mellon University working with Siddhartha Srinivasa and Maxim Likhachev and as a research scientist at the National Robotics Engineering Center (NREC). Oren has published over forty peer-reviewed conference and journal papers. He has received the best paper and best student paper in ICAPS 18 and ICAPS 19, respectively.

Abstract: In this talk I will describe how formal methods such as synthesis – automatically creating a system from a formal specification – can be leveraged to design robots, explain and provide guarantees for their behavior, and even identify skills they might be missing. I will discuss the benefits and challenges of synthesis techniques and will give examples of different robotic systems including modular robots, swarms and robots interacting with people.

Biography: Hadas Kress-Gazit is a Professor at the Sibley School of Mechanical and Aerospace Engineering at Cornell University. She received her Ph.D. in Electrical and Systems Engineering from the University of Pennsylvania in 2008 and has been at Cornell since 2009. Her research focuses on formal methods for robotics and automation and more specifically on synthesis for robotics – automatically creating verifiable robot controllers for complex high-level tasks. Her group explores different types of robotic systems including modular robots, soft robots and swarms and synthesizes (pun intended) ideas from different communities such as robotics, formal methods, control, hybrid systems and computational linguistics. She is an IEEE fellow and has received multiple awards for her research, teaching and advocacy for groups traditionally underrepresented in STEM. She lives in Ithaca with her partner and two kids.

Abstract: In this talk I will present two recent results from the Robomechanics Lab at CMU, one in the area of robot design and the other in control and state estimation. First, inspired by the incredible agility of cheetahs, we analyze the design and use of their tail. Unlike past robotic tails, the cheetah tail uses aerodynamic drag to help with high-acceleration behaviors. Based on this we built and tested an aerodynamic robot tail which can improve the agility of a legged robot. The second half will consider the question of how to estimate the state of a robot that is making and breaking contact with the world. Impact is inherently discontinuous, and so classical approaches like Kalman filtering do not work. In this work, we present the Salted Kalman Filter that can propagate state uncertainty through these transitions and provide state estimation for hybrid dynamical systems.

Biography: Aaron M. Johnson received the B.S. degree in electrical and computer engineering from Carnegie Mellon University in 2008 and the Ph.D. degree in electrical and systems engineering from the University of Pennsylvania in 2014. He is currently an Assistant Professor of Mechanical Engineering at Carnegie Mellon University, with appointments in the Robotics Institute and Electrical and Computer Engineering Department. He was previously a Postdoctoral Fellow at Carnegie Mellon University and the University of Pennsylvania. His research interests include legged locomotion, hybrid dynamical systems, robust control, and bioinspired robotics. Dr. Johnson received the NSF CAREER award in 2020 and the ARO Young Investigator Award in 2018.

Abstract: I will give an overview of our work on robotic object manipulation. First, I will talk about physics-based planning. This refers to robot motion planners that use predictions about the motion of contacted objects. We have particularly been interested in developing such planners for cluttered scenes, where multiple objects might simultaneously move as a result of robot contact. Second, and as much as I have time, I will talk about a more conventional grasping-based problem that we have recently been working on, where a robot must manipulate an object for the application of external forceful operations on it. Imagine a robot holding and moving a wooden board for you, while you drill holes into the board and cut parts of it. I will describe our efforts in developing a planner that addresses the geometric, force stability, and human-comfort constraints for such a system.

Biography: I am an Associate Professor at the School of Computing, University of Leeds, UK. My research focuses on robotic object manipulation and I lead a group of researchers at Leeds on this topic. I am a co-chair of the IEEE-RAS Technical Committee on Mobile Manipulation and an Associate Editor for the IEEE Robotics and Automation - Letters (RA-L). Previously I was a postdoctoral researcher at CSAIL, MIT. I received my PhD in 2013 from the Robotics Institute at CMU.

Abstract: Robots hold significant promise in their potential to assist humans with a wide variety of activities across social and domestic settings as well as critical domains such as manufacturing, healthcare, and space exploration. However, practical deployments involving human-robot teaming remain quite limited as humans and robots do not communicate well; people often find robots incomprehensible and have difficulties understanding what a robot can or will do, while robots lack computational models for reasoning about complex human behaviors. In this talk, I will discuss my research to address these issues by supporting more effective information exchange between humans and robots. Drawing on principles from cognitive engineering, I develop generalizable knowledge, methods, and techniques that enable robots to convey their actions and intentions to people in ways that are intuitive and comprehensible and introduce new approaches that enable robots to model and understand various forms of human input, with a focus on supporting natural interaction paradigms. By integrating these efforts, I will show how we can design new tools, systems, and technologies that make robots safer, more efficient, and even more enjoyable to work with.

Biography: Daniel Szafir is an Assistant Professor in the Department of Computer Science and the ATLAS Institute at the University of Colorado, Boulder. He holds courtesy appointments in the Ann and H.J. Smead Aerospace Engineering Sciences Department and the Department of Information Science at CU and is an affiliate of the CU Institute of Cognitive Science (ICS), the Research and Engineering Center for Unmanned Vehicles (RECUV), the Program in Culture, Language, and Social Practice (CLASP), and the Center for Neuroscience. Dr. Szafir works at the intersection of robotics and human-computer interaction to investigate the fundamental principles that underlie effective interactions between people and autonomous systems. As part of his research, he builds new algorithms, interfaces, and systems that mediate user interaction with robotic technologies, with a focus on the domains of collaborative work, education, emergency response, and space exploration. His work has won best paper awards at ACM/IEEE HRI and HCII and been featured in popular media outlets including IEEE Spectrum, New Scientist, and Tech Xplore. He completed his Ph.D. in Computer Science at the University of Wisconsin–Madison in 2015 and was named to the Forbes 30 Under 30: Science list in 2017. His research support includes NASA, the National Science Foundation, Google, Intel, and Mitsubishi Heavy Industries.

Abstract: Human-robot collaboration has the potential to transform the way people work and live. To be effective at collaboration, partners must be able to understand each other, for example by recognizing goals, predicting future actions, and identifying when the partner needs help. Much of the information about these internal mental states is revealed nonverbally, through eye gaze, gestures, and other behaviors that provide implicit signals. Therefore, collaborative and assistive robots must understand and produce nonverbal communication in order to be effective partners. Enabling this requires a multidisciplinary approach that involves robotics, psychology, machine learning, and computer vision. In this talk, I will describe my work on robots that assist humans on complex tasks, such as eating a meal or engaging in a conversation. I will show how natural, intuitive human behaviors can reveal human mental states that robots must respond to. Throughout the talk, I will describe how techniques and knowledge from cognitive science help us develop robot algorithms that lead to more effective interactions between people and their robot partners.

Biography: Henny Admoni is an Assistant Professor in the Robotics Institute at Carnegie Mellon University, where she leads the Human And Robot Partners (HARP) Lab. Dr. Admoni studies how to develop intelligent robots that can assist and collaborate with humans on complex tasks like preparing a meal. She is most interested in how natural human communication, like where someone is looking, can reveal underlying human intentions and can be used to improve human-robot interactions. Dr. Admoni's research has been supported by the US National Science Foundation, the US Office of Naval Research, the Paralyzed Veterans of America Foundation, and Sony Corporation. Her work has been featured by the media such as NPR's Science Friday, Voice of America News, and WESA radio. Previously, Dr. Admoni was a postdoctoral fellow at CMU with Siddhartha Srinivasa in the Personal Robotics Lab. She completed her PhD in Computer Science at Yale University with Professor Brian Scassellati in the Social Robotics Lab. Her PhD dissertation was about modeling the complex dynamics of nonverbal behavior for socially assistive human-robot interaction. Dr. Admoni also holds an MS in Computer Science from Yale University, and a BA/MA joint degree in Computer Science from Wesleyan University.