Elon Musk says losers use LiDAR. [Explanation video]

in the case of lidar the march of nines isn't there an example i want to just get to your slam on lidar because it's pretty clear you don't like lidar in this ladder's lane lighter is lame that isn't there like a case where at some point 999 down the road where actually lidar may be helpful and why not have it as some sort of a redundancy or backups that's my first question and the second so you can still have your focus on computer vision but just have it as a redundant and my second question is if that is true what happens to the rest of the industry that's building their autonomy solutions on lidar they're all going to dump lidar that's my prediction mark my words um i should point out that i don't actually super hate lidar as much as may sound um but at spacex uh spacex dragon uses lidar to navigate to the space station and dock not only that we the spacex developed its own lidar from scratch to do that and i've spearheaded that effort personally because in that scenario lidar makes sense and in cars it's freaking stupid it's expensive and unnecessary and as andrew was saying once you sell a vision it it's worthless so you have expensive hardware that's worthless on the car so you might be familiar that there are at least two sensors uh in the car one is vision cameras just getting pixels and the other is lidar that a lot of uh there's a lot of companies also use and lidar gives you these point measurements of distance around you um now one one thing i'd like to point out first of all is you all came here you drove here many of you and you used your your neural net and vision you were not shooting lasers out of your eyes and you still ended up here we might have something well so clearly the human neural net uh derives distance and all the measurements in the 3d understanding of the world just from vision it actually uses multiple cues to do so i'll just briefly go over some of them just to give you a sense of roughly what's going on in inside as an example we have two eyes pointed out so you get two independent measurements at every single time step of the world ahead of you and your brain stitches this information together to arrive at some depth estimation because you can triangulate any points across those two viewpoints a lot of animals instead have eyes that are positioned on the sides so they have very little overlap in their visual fields so they will typically use structure from motion and the idea is that they bob their heads and because of the movement they actually get multiple observations of the world and you can triangulate again depths and even with one eye closed and completely motionless you can still have some sense of depth perception if you did this i don't think you would notice me coming two meters towards you or 100 meters back and that's because there are a lot of very strong monocular cues that your brain also takes into account this is an example of a pretty common visual illusion where you have you know these two blue bars are identical but your brain the way it stitches up the scene is it just expects one of them to be larger than the other because of the vanishing lines of this image so your brain does a lot of this automatically and and neural nets artificial neural nets can as well so let me give you three examples of how you can arrive at depth perception from vision alone a classical approach and two that rely on neural networks so here's a video going down i think this is san francisco of a tesla so these are our cameras are sensing and we're looking at all i'm only showing the main camera but all the cameras are turned on the eight cameras of the autopilot and if you just have this six second clip what you can do is you can stitch up this environment in 3d using multi-view stereo techniques so oh it's up there there we go so this is the 3d reconstruction of those six seconds of that car driving through that path and you can see that this information is purely is very well recoverable from just videos and roughly that's through process of triangulation and as i mentioned multi-stereo and we've applied similar techniques slightly more sparse and approximate also in the car so it's remarkable all that information is really there in the sensor and just a matter of extracting it people drive with vision only no no lasers are involved this seems to work quite well the point that i'd like to make is that visual recognition and very powerful visual recognition is is absolutely necessary for autonomy it's not a nice to have like we must have neural networks that actually really understand the environment around you and uh and lidar points are much less information rich environment so vision really understands the full details just a few points around are much there's much less information in those in the construction sites what do those signs say how should i behave in this world the entire infrastructure that we have built up for roads is all designed for human visual consumption so all the size all the traffic lights everything is designed for vision and so that's where all that information is and so you need that ability is that person distracted and on their phone are they going to work walk into your lane those answers to all these questions are only found in vision and are necessary for level 4 level 5 autonomy and that is the capability that we are developing at tesla and through this is done through combination of large scale neural network training through data engine and getting that to work over time and using the power of the fleet and so in this sense lidar is really a shortcut it sidesteps the fundamental problems the important problem of visual recognition that is necessary for autonomy and so it gives a false sense of progress and is ultimately ultimately crutch it does give like really fast demos and the autonomous driving you just do with some sensors lidar radar et cetera this or do you need more uh we believe just uh just cameras aren't the way to go um we don't use lidar at all uh the entire road network is designed for passive optical essentially vision so um if you in order to make a car drive properly you have to solve uh vision and at the point which you saw vision you really don't need any other instruments like a careful driver human driver can can drive with an extremely good track record um and the unlike a computer unlike a human the computer does not get tired yeah it has a 360 degree surround cameras it's got three cameras pointing forward uh so it's like being able to see with eyes on the back of your head basically um so it's really um vision is the way to go uh there's some value to um active optical uh for a wavelength that's occlusion penetrating so it can see through fog or rain or dust but it has to be high resolution such that you can rely on like for example for radar at a roughly four millimeter wavelength uh this is good for occlusion penetration um and uh but it needs to have enough resolution to know that you're breaking for a real object and not just a bridge or a manhole cover or something like that

Channel: Theoxa

Views: 87,912

Keywords: elon musk interview, elon musk award, elon musk, Tesla, electric cars, ev, battery, 4680, electric airplane, full self driving, dhl ceo, ceo interview, german factory interview, tesla cars, model 3, waymo, LiDAR, cruise, Zoox, apple car, fsd, LiDAR is lame, LiDAR is a crutch, solve vision, autonomy, andrej karpathy

Id: BFdWsJs6z4c

Length: 6min 52sec (412 seconds)

Published: Tue Dec 22 2020