All Segments
Drago Anguelov (Waymo) - MIT Self-Driving Cars
18 segments available
Drago Anguelov is a Principal Scientist at Waymo, developing and applying machine learning methods for autonomous vehicle perception and, more generally, in computer vision and robotics. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo. INFO: Website: https://deeplearning.mit.edu GitHub: https://github.com/lexfridman/mit-deep-learning Playlist: http://bit.ly/2S1MVdy OUTLINE: 0:00 - Introduction 0:47 - Background 1:31 - Waymo story (2009 to today) 4:31 - Long tail of events 8:55 - Perception, prediction, and planning 14:54 - Machine learning at scale 26:43 - Addressing the limits of machine learning 29:38 - Large-scale testing 50:51 - Scaling to dozens and hundreds of cities 54:35 - Q&A CONNECT: - If you enjoyed this video, please subscribe to this channel. - Twitter: https://twitter.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/lexfridman - Instagram: https://www.instagram.com/lexfridman
Segments Timeline
Welcome to the Future of Self-Driving
Drago Anguelov introduces himself as a Principal Scientist at Waymo, highlighting his extensive experience in machine learning for autonomous vehicles. He shares Waymo's impressive achievement of over 10 million miles driven autonomously, setting the stage for a discussion on the challenges and innovations in self-driving technology.
"all right welcome back to 6sz ro9 for deep learning for self-driving cars today we have Drago and glial of principal scientists at way mo aside from having the coolest name in autonomous driving Drago..."
The Waymo Journey: A Decade of Innovation
Drago recounts the history of Waymo, from its inception as a Google moonshot project to its current status as a leader in autonomous driving. He emphasizes the milestones achieved, including the first fully autonomous ride on public roads and the launch of a fleet of self-driving vehicles in Phoenix.
"a bit about our work and the the exciting nature of self-driving and the problem and our solutions so my talk is called taming the long tail of autonomous driving challenges my background is in percep..."
Taming the Long Tail of Autonomous Driving
Drago discusses the complexities of autonomous driving, particularly the 'long tail' of rare scenarios that self-driving vehicles must handle. He explains the importance of developing systems that can manage diverse and challenging situations encountered on the road.
"so I want to tell you a little bit about Weimar when we start way more actually this month has its 10-year anniversary it started with Sebastian throng convinced the Google leadership to try an exciti..."
Perception, Prediction, and Planning
In this segment, Drago delves into the core AI tasks essential for self-driving cars: perception, prediction, and planning. He explains how these functions work together to ensure safe and effective navigation in complex environments, highlighting the challenges posed by unpredictable human behavior.
"tail of events this is all the things we need to handle to enable truly sub driver this future and I guess all the problems that come with this and offer some solutions and show you how has been think..."
Understanding the Environment: Perception Challenges
Drago elaborates on the perception aspect of autonomous driving, detailing how vehicles must interpret sensory inputs to understand their surroundings. He discusses the variability in environments and the need for robust systems to recognize and respond to diverse objects and situations.
"sub driving which is perception prediction and planning so I'll tell you a little bit about those right and perception these are the core AI aspects of the car usually this task there's others we can ..."
Anticipating Behavior: The Art of Prediction
This segment focuses on the prediction capabilities of self-driving cars. Drago explains how vehicles must anticipate the actions of other road users, particularly pedestrians and cyclists, to navigate safely. He discusses the importance of understanding past behaviors and environmental cues.
"do it well I gave it up I'm a machine learning person I think when you have this complicated models and systems machine learning is a really great tool to model complex actions complex mapping functio..."
Decision-Making in Complex Environments
Drago discusses the planning aspect of autonomous driving, which involves making real-time decisions based on the vehicle's perception and predictions. He emphasizes the need for safe, comfortable, and efficient driving behavior in crowded urban settings.
"finding new examples in the world and for some situations we have fairly few examples as well right and so there are cases where the models are uncertain or potentially can make mistakes and you need ..."
Scaling Machine Learning for Autonomous Driving
In this segment, Drago highlights the role of machine learning in enhancing the capabilities of self-driving cars. He discusses how modern machine learning techniques are applied to improve perception, prediction, and planning, enabling vehicles to handle the complexities of real-world driving.
"we deal with large-scale testing which is another key problem it's very important in in the pipeline and also in getting the vehicles on the road so how do you normally develop a self-driving algorith..."
Q&A: Insights from the Audience
Drago engages with the audience during a Q&A session, addressing questions about the future of autonomous driving, the challenges faced by Waymo, and the impact of machine learning on the development of self-driving technology.
"a mental exercise right when you think of a system that is tackling a complex AI challenge like self-driving what is the good properties of the system to have and how do you think a scalable system an..."
Synthetic vs. Real-World Data
Anguelov concludes by discussing the balance between synthetic and real-world data in training autonomous vehicle models. He explores the ongoing research in adapting simulation data to better reflect real-world scenarios, emphasizing the importance of achieving realism for effective training.
"cycle to happen thank you appearance trouble thank you so much for the talk really appreciate it so if you were to train off of image and lidar data a synthetic imaging lidar data is there would you w..."
Balancing Rules and Learning in AI
Anguelov reflects on the balance between automatic models and rule-based systems in autonomous driving. He highlights the need for extensive testing and analysis to track model performance and adapt to evolving challenges, emphasizing the importance of expert input in areas where models fall short.
"achieving realism in simulator is an open research problem right I assume no there is a lot of rules that you have to put into a system to mate to be able to trust it you know and so how you find the ..."
Quantifying Uncertainty in Predictions
This segment focuses on the significance of quantifying uncertainty in the predictions made by neural networks. Anguelov discusses various techniques for capturing uncertainty, including leveraging environmental constraints and using probabilistic models, which are essential for improving the reliability of autonomous systems.
"right so evolving your system as you go I mean generally you know the MLP growls is the capabilities in the data sets girl right so you stressed at the end of both the first half and the second half o..."
Adapting Models to Different Environments
Anguelov explores the adaptability of autonomous vehicle models to different driving environments. He discusses the ideal scenario of having a single model that can handle various situations while acknowledging the need for complementary models to address specific challenges in diverse urban settings.
"also provide the measure of uncertainty another way of doing uncertainty is to leverage constraints in the environment so if you have temporal sequences right you don't want for example objects to app..."
Simulating Pedestrian Behavior
In this segment, Anguelov delves into the complexities of simulating pedestrian behavior in autonomous driving scenarios. He highlights the challenges of accurately representing pedestrian actions and the importance of realism in simulations, particularly when considering the interactions between vehicles and pedestrians.
"and I was wondering if you could either talk about talk more about or maybe provide some insights into simulating pedestrians because as a pedestrian myself I feel like my behaviors a lot less constra..."
The Future of Self-Driving Cars
Anguelov shares his insights on the timeline for the widespread adoption of self-driving cars. He emphasizes that while significant progress has been made, the technology requires extensive logistics, algorithms, and testing to ensure safety before it can be rolled out at scale.
"models are paying attention thank you for the talk it was very interesting since you you know titled and talked about it long tail it makes me wonder is the bulk of the problem solved do you think wel..."
Contextual Awareness in Autonomous Systems
This segment discusses the importance of contextual awareness in autonomous driving. Anguelov explains how understanding the attention of pedestrians and other traffic participants can influence vehicle behavior, highlighting the need for models to incorporate these social cues for improved safety and interaction.
"when you were talking about prediction you mentioned looking at a context and saying if a person or if someone is looking at us we can assume that they will behave differently than if they're not payi..."
Reasoning in Deep Learning Models
Anguelov addresses the underexplored concept of reasoning in deep learning, particularly in the context of autonomous driving. He discusses the potential benefits of integrating reasoning capabilities into models and how this could enhance their performance in complex environments.
"checking to see if they were consist ready that's my own theory by the way right but I feel that the concept of reasoning is under explored in deep learning and what it means right so if you read for ..."
Hybrid Systems in Autonomous Driving
In this concluding segment, Anguelov reflects on the role of hybrid systems that combine machine learning with expert-designed algorithms in tackling the challenges of autonomous driving. He emphasizes the need for a balanced approach to ensure safety and reliability as machine learning continues to evolve.
"interesting you mentioned expert design algorithms and I was wondering from your perspective almost from Wayne was perspective how important are those say non machine learning type algorithms or non m..."