Third Row Tesla - Episode 17 - George Hotz - Autonomous Driving

[Music] welcome to the third row Tesla podcast this is Sathyan for ball this is a really special episode we've got a really special guest I think you're gonna really enjoy it so we've got our third row crew we got Omar Kazi hustle truth boom and we've got Kristen k10 okay and then we got our special guest the one and only george hotz how you doing George pretty good excellent man so yeah today we're just gonna cover a bit about George and autonomous driving maybe cover a bit of his background and some of the future stuff so I said George I've actually been a huge fan for a long time I actually was just saying that I had Lyme brain running on my on my iPhone I think was like a 3GS way back in the day and you know it's just like hey did you ever see yourself go from from there to where you are now I mean how'd you get started like how'd you get started programming and like hacking you know the same way everyone else did but with respect to like how you go from that to this it's the same thing what's really the difference between jailbreaking a phone and jailbreaking a car sure yeah true that's an interesting way to look at it it was it was the jailbreakers that really drove forward the people at jailbreak is that apps on the iPhone before the App Store where they're gonna be apps where they're gonna be installer apps that ran on the device whoever ActiveSync back in the day you don't with the app of the computer you have to download it all in place the sparsity what if it happened maybe oh no Steve Jobs was saying it was all gonna be html5 right a sweet solution yeah so this is I mean this is the same thing right and it probably would have happened no matter what but it's a question about where it happens first and you know who has control so your approach has been a little bit different I mean I'd love to hear about the interviews you said you had two interviews with Elon right for Tesla I'd love to hear more about that and then how it turned into what you're doing now so a friend of mine we were working at this company and he approached me to talk with Elon about building a vision system for autopilot building a replacement for the mobile eye chip and I mean it's it's the moment wasn't very good uh I saw like a UI that was bringing up like the outputs and I'm like that's yeah you can you can beat this pretty easily with what we already have to back so yeah I mean what was your background in computer vision before that had you done anything with computer vision or deep learning I saw that I was I was working today I start out um I think it's publicize working vicarious uh and yeah it was relatively new to the stuff but I guess I've done computer vision before my sophomore year science fair project was a robot that could drive around you type in an object as a Google I was like the physical instantiation of Google like drive around you could type in like basketball and go pick up a basketball bring a do once it very uh wasn't very good computer vision but uh yeah no it was it just deep learning just makes really simple intuitive sense so you just you know gather a data set of images data set of ground truths and your humanity it was just something country so yeah the idea was I was going to build that mobile egg replacement freely awesome and then so what happened why didn't it happen but why didn't you stay on like work for tousle so it's a long time ago now but um that was almost five years ago now well Elin and I had agreed to a contract which said I was going to get paid out upon completion of the when my thing was better than the mobilized by I said approvable metrics I was going to be paid twelve million dollars if I finished it tomorrow and then I lose a million dollars for every month I don't finish my favorite my favorite phrase of Avila's was a incentive incentives aligned so I like that I like that I thought it was deal so we went back and forth for weeks with with with a lawyer working on the details of that contract and then yeah he won calls me on my birthday uh and he's like look yeah well you know it's not gonna be a contract but I want you to build it and I'll still pay you for it like what do you mean like so you know we're not gonna have completion criteria one of the completion criteria is going to be whether I like it or not like well that's no but that's not a contract anymore right you can have a contract be gated on your subjective feelings about what the thing is right because then I'm working and you know I've been a contractor for a long time that's that's a quick way that'd be like well actually we just want these three more little features right so it's all I was like no and yeah by this point of like by this point I'd had actually a lot of progress made on the on the mobile I replacement mm-hmm that's interesting and you weren't you weren't even under contract at that point is that right so you were just doing it just just for fun because as research and preparation and stuff yeah yeah yeah so I had rigged up in my car I was matching like I was ground truthing it using the campus and I was matching the campus to the 2d images and you could know where the car actually went and I was predicting where the car should go I wouldn't know I could start to say okay now I'm predicting you should turn now I'm predicting you should go straight just predicted steering angle for images I said an email to you on saying look no I'm not gonna I'm not gonna do this if you want to invest you can invest the car the prototype car that I was gonna work on is like that so I went out and bought the story you got attention from from Ashley Vance you know why did I wanted to get back at he loved it so it was just that criteria the issue then not having a set firm set of like this is what it depends on yeah okay it's not a contract at that point right like a contract needs to be a a a neutral third party needs to determine whether the contract is completed or not you can't have one of the parties say the contract will be completed because I'd say it'll be completed right away Neil I'll be like oh you know actually I wanted to do full self driving I want a nap in the backseat and I want to be that good I'm like this you know so it was basically like a zero dollar contract if you think about it it's like kind of giving you an impossible task well being mobile I was a very possible task the mobile ionic youth Rose was quite bad and part of the reason it's so bad is because the chip is so low power I believe it's a it's like a it's like 28 or 45 nanometers and it draws like a water to so you know I could just you know I could throw 10x more computer that's who cares also wasn't the thing with mobile I that they kind of like fabbed the neural net into the chip or something like that so it wasn't that flexible yeah it's unclear if I q3 even really was a neural net I think it had some neural net capabilities it does have some processing capabilities but yeah there was a lot of hard coded computer vision I mean mobile I was less of a scam now mobile I was more of a scam back then mobile I was more like well we're gonna you know we're gonna make these chips that like do like the minimum requirement to meet these these safety features right and we're going to work with regulators to mandate the safety features and I was gonna have to buy a mobile eye right they've changed they've come around so did that work that you were initially doing for what you thought would be eons contract kind of get you thinking hey I could I could do this with with comma and kind of birth that company well I mean it was comma right there was no there comma originally was a single purpose LLC for the contract we dissolved it created the C Corp version and yeah it was like you know I'll get investment basically comma was kind of a spin-off of this original idea that computer vision can drive a car by itself yeah you were working at Google about right before that right I was an intern on project zero school right before that I was out of his own vicarious and I start up in East Bay yes is really interesting for those people who don't know about comma cuz there's probably some out there listening to the podcast the idea was basically you know it's not like autopilot or any of these other self-driving car systems where you know you take a Robo taxi ride George said you know let's build something that we can just add into any car you know any new Toyota Corolla you can go install this , AI device you can install open pilot and you'll be able to get autopilot like functionality on that car well that's what it is today but that's not started out how to start out well yeah yeah I bought an Acura ILX those I went around to the dealerships and I was like can you show me the car turning the wheel and hitting the brakes right because I need a car I'm not I'm not rigging up motors to the real ER so yeah I mean I didn't know much about it back then but once I saw that it could turn the wheel I'm like okay there's some software doing now right once I saw that get it to break some like this is software I just need to figure out how to how to intercept the the calls I mean it's it's it's wild what it became that we can now supports like I think it's like 80 different cars now oh yeah we support everything right we just we just reverse engineer to all the abstraction layers for all the cars that we have this like vehicle abstraction layer and open pilot which makes all the cars appear the same to the other thoughts offer so the vehicle the manufacturers aren't actually helping you you're just doing it on your own yeah manufactures know I haven't spoken with a single manufacturer but the regulators came and slapped you right they said no right oh not really so I mean Anna's story was played up by the media and we let it get laid up so you run into this problem where you know one of the big problems in startups is you want to separate the people who believe in the idea from the people who like are trying to get onboard hypetrain mm-hmm and one way to do that is you know a big you to regulators he's a great way to just lose all the hype train people right like you know they they they disappeared overnight the dick riders disappeared overnight oh your car are out of business they're out of business look we've been around five years we're a profitable company we support 80 car here thousands of daily active users like yes you can actually go to comas website right now and get an open pilot device and just list all the software on it you purchase hardware which that's all right yeah yeah I noticed that Tesla isn't officially supportive but I'd love to see like how it compares so can't can it actually work with a Tesla just cured oh yeah there's I think about a hundred and twenty Tesla users so open pallets open source this Forks there's there's thousands of Forks of the software only like ten really get use but I'll go to the forks support support Tesla's yeah there's a decent-sized community of people who are getting it to work on pre a p1 Model S's exactly because I had a pre auto pilot Model S 2013 Model S and it would have been great you know someone's gonna have that car and they can actually go out and get your system and add it to it and then get auto pilot and because the lane-keeping is is the most important thing there's some annoyances about it there's some annoyances about it so the the EPS is the same so the steering works the same it's using the same steering as in the new Model S's but the brake is different so the pre a p1 Model S's did not have adaptive cruise control actually didn't have a way to hit the brakes they only had traction control brakes and you could do region but you know regions limiting batteries for um it works it's not the best supported car there was a video that came out about three months ago comparing the Toyota Corolla to the model 3 on autopilot yeah yeah someone did that I saw that there's pros and cons I wouldn't say one is strictly better than the other it depends on what you'll use it for and just like how you feel the system should be I want to go on the racetrack with it just kidding it will engage autopilot probably won't engage yeah yes some of the features of autopilot do like some of the safety features but when you're on the track actually they have track mode which is a whole other thing which is you know another discussion but that's that's kind of exciting to me when the car actually do it the ultimate lap for you you know well I mean it'll do a really shitty lap but I think right now it well yeah I think open pilot will pilot will engage anywhere autopilot you know some time you tap it it says autopilot not available no we engage and we try our best no matter where we are so you can engage it on a track will it get around the track here probably at 25 miles an hour some of the stuff we ship last year - like it doesn't rely on really lane markers anymore it kind of uses them but it can drive without them - as well so let's your take on kind of the general state of the self-driving car market right now like you know we've seen players like you know way mo and Cruz and others like that for ten years how do you see this technology actually coming into the marketplace and being used by people so Zeus is looking for a buyer at investment costs Cruz Cruz yesterday just announced layoffs no let's talk about that I mean you you have layoffs why because you lost so much revenue because of coronavirus oh oh wait you didn't make revenue um yeah yeah you know what I it does kind of seem like this whole thing with the virus is kind of a smokescreen for people to kind of offload a lot of their problems absolutely yeah a lot of things are getting blamed on it I mean you know a lot of pre-revenue companies you know are laying off people and it's like okay this didn't really impact you at all so where you know it was probably something they were about to do anyway before and it's no secret that these projects haven't exactly been going as well as expected they just keep pushing the date out and out now it's standard its standard cult leader stuff you know when the world didn't end last year all but don't worry we got new stuff the ones that check out my crews allergen Yeah right I mean that's kind of crazy that you know they showed this concept but it just seems so far ahead of where the technology actually is I mean it seems like no-one's designing realistic process the cruise origins so it's so sad does anybody want that has anybody said a model three like their own model three I've been like you know what I really want a small golf car where we face each other yeah hey nobody wants this real question how do you feel about the bus well at least on the bus you don't have to face each other sounds like something worse than a bus yeah I mean what anything do they do these people think self-driving cars are immune from like acceleration sitting backwards and stuff is uncomfortable yeah I mean I would be kind of scared of sitting backwards in some you know programmers self-driving car that you know I don't know if it's gonna work or not right now you can't even see in front of you think about cruises even if like like the car is so stupid even if the software works which it doesn't but it's still so stupid right you know what why don't you put you ever see the Mechanical Turk they should put just a lying-down human at the bottom of the car to drive it you know people who can fit in the small compartment yeah all right all right no more no more done Ripa dunkers but you know it's kind of like even if they figured out autonomy would people use this would they give up their cars is this how people want to get around all the time that's how the Silicon Valley elite imagine the pleb they should be transported yeah you have to wonder if they would really want to use that themselves you know it's it's all seems kind of just like this fantasy worlds kind of detached from reality yeah I mean in a sort of way they have to and like I kind of do feel bad for these people like you know what do you do if you've raised billions of dollars and really have kind of nothing to show for it you have to promise stuff that's better and better and better and light and then eventually that the house cards collapses well the problem is they have a strategy where they essentially need to achieve level 5 before they generate their first dollar of revenue yeah there's no incremental work but that's that's different to what you're doing because you're actually you've got cars on the road people are using it and getting data back I mean you know what are your thoughts on the you know what's your vision for the company going forward for kamma I mean what do you want to achieve we have the second largest network right now of like economist vehicles Tesla has the largest obviously by by a huge margin but like people think like oh well Weibo has a lot of cars coming has more comments way more than Cruz and zooks um so yeah I mean we're number two right now in terms of in terms of that and like there's a question of kind of how many cars you're gonna need in order to get to level five like how much data collection you're gonna need we estimate that it's between ten and a hundred thousand just like to be able to kind of almost train the models in real time and to get like this statistical to be able to stay with statistical significance your safety factor over a human so yeah I mean you know Tesla's already way more than enough they're not limited by that we can talk about what they are limited by later but I think that 10,000 number applies to everybody so can you really imagine one of these lame-o's scaling up to ten thousand cars of zero revenue yeah where each car costs what four hundred plus thousand dollars when it costs like because like a million per car over like the lifespan of the car yeah yeah so I mean you know someone's got ten billion dollars to blow on that stuff but what's your take on what happened with Anthony Levin dowse key and uber you know they just sued him like I think he's going bankrupt now and maybe going too jail yeah so I read an article once about this guy who stole a piece of trading software from Bear Stearns and I think it was Bear Stearns and and they caught him they deleted the software there was no harm to anybody and he got eight years in jail oh yeah I think was goldman sachs yes it was goldman sachs yes and that just shows you i mean you know if you believe that all prosecutions are political well that's the julian assange growth um yeah i mean that's what happened there google got back at mr. levin dosti for leaving it yeah I mean like you see their videos and it's just kind of this happy-go-lucky like lame-o like everyone's gonna get around in way mo and it's just like this cheesy marketing stuff and everyone kind of considers them the leader and self-driving and it's just kind of amazing to me that it's like okay nobody is questioning the fact this is taking ten years and they're saying it's not gonna be done anytime soon well so it's interesting to say like like way Mo's the leader in the Special Olympics for self-driving cars here's the quote that I gave on Twitter right if Tesla's if the race is from from California to New York LA to LA to New York City Tesla is driving in a Model S they just drove by st. Louis we're sitting at LAX waiting why are our planes delayed but you know once the machine learning takes off like alphago will be good and way most the furthest along their cruise ship just docked in Japan yeah and then what about the tech cuz I mean they're they're mostly lidar right now I mean it's you know there's they got the tech disadvantage and well in my opinion and the the data disadvantage it's unclear what that even means that they are lidar right like those companies are using lidar for localization most more more than anything else they can do perception with the other stuff its localization that's hard to do doing doing precise localization like or accurate with cameras is no it's becoming possible but it was not possible a couple years ago and but that whole approach is so wrong I think I think this whole approach where you are tracking your car down to the centimeter and if like Elaine moves well is not the map sorry we go it's a line fault yeah it's a line following robot must be costly to pre map everything - sure and like you know what you can do the math and like you know people like all the unit economics the unit economics don't worry it works out once we scale which are hundred million cars and fifty percent of people are riding with remo the mapping cost becomes negligible yeah who's gonna ride Remo that's that's my question who really wants to ride waynebow on self-driving minivan I mean look I mean if you think if you think of really like like it is pretty it's weird like like the way there are they're marketing as they're marketing it like it's something special there's two ways of which a ride-sharing app can compete in my mind you can compete on cost and you can compete on speed so might make something that's cheaper than uber why do I do I care if I get an uber or a lift Oh which one's cheaper and which one's gonna get here first exactly right and then with the lame-o you also have - okay I assume that it's a wash - lyft drivers or uber drivers drive faster I don't know but you know those self-driving cars drive man slow so you know ya yield to everybody in this stop something that's what I want my uber driver to do yeah yeah it was a ten minute drive and you know it took 40 minutes to get there because the car had to keep detouring yeah I don't think it'll be that but like like fundamentally they talk about it like it's not a new experience whereas with things like like like autopilot and open pilot they're actually new experiences for people on their experiences that people who have an experience don't exactly understand what it is or why they want it but then once people use it they're like so our cohort analysis our retention rate more than 50% of people who purchased a common product are daily active users it's pretty good Wow the only thing that really beats us on those numbers is mobile phones mmm-hmm I think we even beat laptops well um when it comes to like consumer electronics you know people don't use their laptop everyday maybe you know maybe some people do but like how many laptops it around you know what VRS you know you know how many times the average VR headset has used the media once a year movie it's once it's once the average VR headset is bought and used once that's crazy yeah it sounds like mine all right and that's like the problem with some of these new I'm not sure if I'm PRI love VR but this this cohort analysis idea of like you know you're giving people something new what scar analysis for airports so it seems like you're kind of skeptical of Silicon Valley like you know Silicon Valley of course famous for you know microchips phones computers but it kind of seems like you know some of these products that have been hyped or just hype you know these next big things wait microchips which are now made by TSMC Samsung and kind of Intel up in Portland though you know that's that's that's faltering and glowing at Global Foundries in Taiwan toon way that doesn't look like Silicon Valley no actually none of those places or Silicon Valley yeah where's the silicon well yeah I mean and then like phones which are made by like well okay Apple but apples on Silicon Valley apples Apple is a 50 year old company right like what has come out of Silicon Valley in the last 10 years social networking what else well name something like like what else and these like weird social apps okay you have things like uber but like that's not that's not that's a platform air B&B these things are not technology I don't know why anyone thinks they are who's getting duped here so the platform Valley Valley and they're making these these these things are literally not technology well some would say you know that it's just kind of c-money subsidizing businesses with bad unit economics that you know Hoover's a cash burning business you know eats is losing money Grob UPS losing money they combine them and they're just kind of using VC money to deliver food to people because they're promising one day with everybody when nobody even thinks of eating in any way besides those reads like I'm not saying look I like Cooper right I'm not saying like like uber has definitely added value to my life but it isn't really technology right it's you you can imagine uber being done in the 80s with a switchboard what has actually made over possible today the mobile phones are pretty nice yeah I mean you need a way for people to actually call the ubers and you know you need a way to pay all these people and matchmaking but yeah I mean again not technology so you think autonomy could be that technology that really causes the shift well autonomy is technology right Tesla is technology SpaceX is technology these are technology companies right Google and Facebook are not technology companies uber is not a technology company Airbnb is definitely not a technology company you know what was really not a technology company we work with a shitty office real estate company that my god it's technology you know I hope everybody learned their lesson and I hope everybody can apply the we work lesson to all the rest of the stuff to look pandemics check it up for sure everyone's working from home now yeah so switching gears I'm just curious coming back to the Tesla stuff and the have you actually tried out the new software the the new stoplight I have not tried out stoplights yet I rented a car I rented a model three for a week and a half for a road trip and I did thousand miles on autopilot and it's very simple little pot is very similar uh some things I think it does better some things I think it does worse right well when I realized I didn't have to touch the wheel to reset the timer I just know it got WAY bad exactly yeah that's a really good feature yeah I'll show you the traffic control thing if you want oh I've certainly watched the videos oh cool nice yeah so your system has a driver monitoring as well so you're looking make sure the drivers paying attention I mean what are your thoughts on that because I know a Tesla doesn't have that well it may have that in the future with the model 3 says we have the selfie cam but now no I'll tell you why Tesla doesn't have it it's sad so yeah well the problem with what's the problem with the selfie cam why can't you use it for driver monitoring you tell us yeah well you can't see at night so our original re on the last generation , device had the same problem we couldn't do driver monitoring at night so if you can't rule out driver monitoring completely um yeah you know like do you want to roll it out at all right and it must be a huge task to like I feel like driver monitoring is you know almost as complex of a computer vision problem is you know say lane-keeping or something you don't think so jaramana you're so easy we built our prototype in like a week and the prototype is like good enough that we can want to depress the next week so you see if you're george hotz it's easy no but my point is it's easy because like all we did we downloaded this open-source face detector called pianet and we've moved past this a lot but then we like trained a model to kind of we we ramped here on an art of data and then we trained a model on top of that and it was it was quick and easy and like you know what no startups have good lane-keeping except for early us really and no there's 10 startups that have totally acceptable driver monitoring solutions it's much easier yeah now to get that last bit our driver monitoring has come a long way I mean I'm a full-time engineer on it he's great he's like just shows me the metrics on my bridge we're just just pushing it to the point that you know it's really hard to get caught it's really hard to to to not pay attention and not get caught and it's also you know if you are paying attention it's good so reading curbs with the camera with the is really hard for the Tesla but you're the con you're you're not having that issue then we don't try to do things like that so I'll preface this by saying I think Tesla is two years ahead of us in hardware um four years ahead of us in fleet size but I think we're ahead on machine learning I think that that because Tesla has resources what they've done is Tesla's whole thing is based around like a big segment they have a net which is labeling stuff in the image labeling curves labeling lanes labeling cars and you've seen it you've seen this you've seen these visualizations online so we don't really do that we look at a picture and we ask the question given this picture where would a human drive the car okay so we don't need to explicitly detect a curve in order to know not to drive into that curve we just look at the picture and say we're a human drive and the answer is not into the car as long as they're not drunk or something then you're okay most humans are good drivers and that's one of the reasons it works I mean humans actually like you know people always loyal machines they're gonna be so much better than humans they're not even close to humans are incredibly good drivers so do you feel it's uh it's easy enough to get the training data you need to train your models yeah yeah I got 30,000 miles coming at that Wow that's crazy that's pretty good yeah amazing so so if you don't see you guys don't do 3d labeling when you do something else but how does what's your approach that how is it different to Tesla we don't do lately so you're just basically depending on the driver completely so it's like self supervised or something yeah so we do we do we do small amounts of labeling for a few things in fact actually have a project of an open-source project now comma 10k where users are giving us labels we only have five categories Road Lane undriveable movable I feel like this one where all you have my car yeah so it's like pick pick that you could see like hood and stuff but actually those are more used for understanding the motion of the scene lanes are the only semantic category we have and it's because they're good for maps but all the other categories have to do more with semantics right Road you can assume is all in a plane I have let's do it cymatics road you can assume is all in a plane movable stuff moves in a way different from the ego motion of the vehicle my car stuff never moves so it's not about semantically labeling stuff because that becomes really hard you know some of these datasets it's like they label cars and trucks a different color well what do you label an f-150 what do you label an f250 what if it's pulling a trailer right where's the line it's not right yeah because there's so many edge cases it seems like Tesla they keep improving in their visualization like you can see that this is that now a pedestrian walking you know across the road before it was just phased front on and then what they were like you know shifting across like that so the latest software update but I mean you're saying that that's not as important as just understanding those few basic things yeah that's it makes for nice visualizations but I actually think this is one of the ways one of the the wrong directions Tesla's going in you're gonna get in like way mo does this too um we joke at comma about way Mo's cone guy right and he like gave like a long presentation about the taxonomy of cones all the different cones what they need what their placements might mean and we joke like as a self-driving car company once you've hired a cone guy you failed he's telling me they have a guy who his job is just cones his whole presentation is about when I was driving the when I did the road trip when I was driving the car at autopilot those it was detecting the cones of the cone show up in the Tesla visualizer were like I mean he's not full time yet but Tesla's on the cone guy trajectory yeah well they have this video where this woman dressed up as a cone and hilarious but yeah I mean I think the visualization is important to the human to know that computers actually doing something I mean that's that's really like it it's like yeah I trust this computer that it's gonna you know treat me right and do the right thing safely all that stuff it's a cool ad for the Tesla cars if we had an extra if we had we have a lot of stuff that's just on visualize because I don't have a you eye person if someone out there is looking for a job and wants to come right are sick visualizations we could totally do that so so let's talk about the hardware a little bit so in terms of like inference on these models you've trained do you have some kind of GPU and the hardware or how exactly are you doing that so we have a Snapdragon 820 one in the in the in the competent and the Snapdragon 820 one has 500 gigaflops on the GPU so you're 500 gigaflops of computer 500 gigaflops can run a lot our newest model is an efficient net beat to so efficient that is like the latest the latest paper and efficient epi two is like better than a resident 50 nestlé's based on ResNet 50 so you can run decent sized models in this as long as you're not obsessed with life running it on eight cameras if you're trying to add it on eight cameras full resolution yeah of course we can't do that if you don't need to if you're trying to track every car the scene we can't do that but you don't need to because you're not doing that as a human if you look at these visualizations like look at Zeus's where they're tracking like 30 different cars and 1700 pedestrians and Market Street humans don't do this humans look and say like you know there's a bunch of people over there adorn a juggling car engine well there's like a car in front of me all that car is like coming to cut me off right you pay attention to like two cars like two people right yeah kind of scanning for danger looking if someone's gonna walk down the street or anticipating it's it it's about knowing which ones yes it's about you can't just pick random no so being able to know which ones and being you would have your machine learning software pick that is this super work so how would ya I'm not familiar exactly with that Snapdragon processor you mentioned is it kind of like how would you compare to say you know the latest samsung phone or something like that in terms of processing power or even like the latest Nvidia you know they came out with a new chip as well oh oh god don't get me started videos another company that I could rip on a lot but uh no to compare it to it to compare it to a modern smartphone so it's the Snapdragon 820 was a 2016 era smartphone processor that's pretty amazing you're basically driving a car with smartphone oh it is a smartphone yeah yeah of course let's actually phone motherboard it's a phone motherboard in the device so do you find it you know challenging in terms of the resource constraints on that on that system or is it actually pretty easy so when you run a neural network on a GPU the CPU is like queuing up all these commands and that's doing tons of work in the GPU driver so that was using half of a core so we have a four core processor to run the whole thing on and it was using 50% of a core um but you know whenever we run into resource constraints we think how can we be more clever the resource constraints kind of help us out like I don't feel massively constrained in any kind of way we're running an efficient net B to at 20 FPS so with the model that's very good and then with like all the CPU I mean it's written in Python to a lot of uh is actually Python well so like you know car the the car control problem is low speed people think like oh it's all gotta be like real-time and perfect doesn't really matter humans like you always have at least 50 milliseconds to react to everything you're driving human reaction times to 50 like um so yeah are we resource constraints somewhat but not that not as much as you think so why did you choose not to run on say an Nvidia jet set or something like that so you know what I'm gonna get something I'm gonna get something I'll be right back this is one of the saddest things in the company's history oh yeah I never shown this to anyone yeah great this here is a kala 6 mm 6 cameras like the logical numbers Wow yeah right love it and in here is an Nvidia Jetson TX tail oh really huh yep there was there was a time I made it smaller this this this prototype is from 20 set this is from 2018 so this is this is 2 years old Mountain um basically what happened is Nvidia well the first people we spoke to from a video are very nice the first people who spoke to from an video were like yeah we can tell you tx2 chips 80 bucks a chip the documentation for the tx2 chip is much better than the documentation with a Qualcomm chip it's all available online to alike great we're gonna design it around this Nvidia chip they're gonna love us well the guy who was in charge of Nvidia autonomy division got fired put a new person in job in charge no person complete like that chip and an invidious strategy around this for the last two years has been completely nonsensical so what actually happened when we met with the new business development people from Nvidia to talk to them about buying chips they looked at me they're like we're not gonna sell you chips you can buy modules these look to $400 yep what I did I wasted hundreds of thousands of dollars designing this thing um it's it's it's sad it's sad and they're like we're just not gonna sell you chips you can't do anything you can't get the chips anywhere else oh when you at least Qualcomm has multiple different module manufacturers Qualcomm was honest with us to hear we approached Qualcomm to buy chips and quarkons like will sell you chips oh you got to buy a hundred thousand like thank you for being honest with me right and Vidya you know they may butter you up they send over the guys and this is where I really don't trust business development people and I'm like okay I actually wanna play in order for 2,000 chips they like get out of here calm but then what in video comes out with next is the Xavier the Xavier costs $1100 for the module yikes the cost to automakers 500 that's not right what's up with that who do they think they're selling to automakers will pay 50 bucks they just came out with another one they came out with an even more ridiculous one announced two days ago that's again these like these massive chips that cost $250 to fabricate there is no you can do incredible level two so this is a common tip this is this is this is our flagship product this is a common - it has a Snapdragon 820 in it you can do incredible level - autonomy on a Snapdragon 820 Snapdragon 820 now is a $50 check yeah with the 500 our chip you're not gonna get a robo taxi because Robo taxis aren't real no you're not even gonna get better level - everything that makes level 2 better is software bugs and open pilot we're not even really constrained much by this hardware this single camera guru like all you have one forward facing camera we're only using 12 percent of the box and we're only using 12 percent of the box not because of resource limitations or using 12% of the pixels not because they're resource limitations but because of our training architecture now we have to simulate and we need more box we're fixing this but um I I just I you know I can't believe Nvidia rent-seeking on the hole on the hole what happened with Nvidia I mean they were originally powering you know Tesla and autopilot and I know people people were thinking oh you know these guys are gonna be a big player in autonomy they have the GPUs and it seems like their whole kind of autonomy strivers you really went off the rails I'll say this I know I won't go into details on it but they had a similar business development experience that we did hmm and Vidya basically said screw you to Tesla good right that was so stupid and this is another layer why do you think Bonnie I think Tesla built that chip right I mean that's that's this that's typical Elon got mad it's a we're gonna build our own chip it's gonna be way better than it was no reason there's no reason Tesla should be building a chip unless Nvidia tried to rent seek to the moon which you know it's they've prohibited use of consumer GPUs and data center by the the EULA because they wanted everyone who runs a data center to buy their $10,000 scam GPUs instead of $1000 they just NVIDIA has no they're just like how can we put money in our pocket right now why should we build a $50 chip we can make our $500 check yeah I think they got burned really bad by the whole kind of Bitcoin bubble you know everyone was buying all these GPUs to mine like III um and then they you know ramped up production and then all of a sudden it's like okay wait III UM's not worth anything anymore and now they have all this excess production capacity and no one wants GPUs I mean yeah maybe maybe that's still why the 1080 Ti is the best price per dollar GPU Nvidia mate you know it's for your LG P you know um but yeah the coma six revealed right here a sad chapter and commas company and it really got us to consider like wait why do we think we need more than one camera why do we think we need more processor like we have the hardware division you know go to the arbiters and say build me the best thing you possibly can we get this we realize that all the bugs an open pilot are actually software and it has come such a long way in the last three years from the software so it seems like nobody is really considering like cost or unit economics or scalability at all besides comma and Tesla everyone else is just kind of like pie in the sky like buy all the hardware you need like no thought of like how we're actually gonna get this out and make people use it in a way that's affordable well because you're assuming that the users are their customer which they're not the the investment banks are their customers scarce right hmm build the fanciest thing ever make sure to stick your logo everywhere make sure make it look really legitimate it'sit'sit's let's practice it's practically cargo calls you know I think who's gonna come out looking great from this is Qualcomm hmm Qualcomm has gone the other way Qualcomm has gotten better with the business development over the years Qualcomm has made there's a great eight forty-five development kit from Thunder comm you can buy a 180 bucks for the whole the whole module just half the price of Nvidia for a module that's better many 45s better than px Dibble and I think that yeah I mean that's the right chip to put in the car it's 50 bucks it's incredibly flexible it's made already in quantities of millions the only there's no mobile phones using invidious chips the only thing that uses invidious chips is in their tennis which in any quantity and the only reason the Nintendo's which uses it is because Nvidia the old business development guys gave it to Nintendo it cost they're like we really want somebody to use our chips Nvidia if you want somebody to use your chips sell your chips on an open market don't rent seek charge 20% over what it costs you to make them and everyone will use the chips I hope they do did you think anyone's left using them in the autonomy space Nvidia really like the car manufacturers aren't the car manufacturers are the IQ 5 is decent chip mobile immobilise done better mobile a mobile eye has gotten uh you know so much better since their original fallout with Tesla there's some decent chips using there's some decent so supercruise is using mobile so Nike three and there's new IQ for cars that look good and the Yankee five looks pretty good like you know why why would I know in using video yeah especially if they're not even willing to sell it to you at a reasonable problem I'm sure I'm sure they'll sell it to out II in quantities of 10 million for $500 chip yeah it's not reasonable so you're back to Snapdragon then we're a full-on welcome mm-hmm okay yep we've moved to San Diego to Qualcomm's here uh yeah there you know at least with Qualcomm you know exactly what you're gonna get how do you see this all playing out with the autonomy players like the spatial approach players and the automaker's maybe in like the next five years how do you see it playing out with autonomy I mean maybe I'm biased but I think you're gonna have I think they're gonna have to I think they're not gonna have a choice now of course you know home pilots open-source this might have nothing to do with common AI and to be honest I really care I just want to win self-driving cars I don't wanna rinse econ self-driving cars as the gap widens between what manufacturers can offer and what open pilot can do it becomes more and more well so it's the same thing as phones right no this isn't Tesla Tesla is gonna continue to develop autopilot I think autopilots gonna stay a year to ahead of what we're doing I think we are will have traffic lights and well we're year behind Tesla but yeah I mean I think that you're gonna see a decoupling of the auto makers are gonna have to give up these stupid ambitions of trying to build these things in-house no GM you are not a connected cloud company you make metal boxes you know all these companies they have to stop thinking like leg leg like they get in some new like lunatic CEO is like we're gonna pivot into technology don't do that figure out who you can buy the technology from stick it in the car and continue to market cars that's what you've done your whole life why are you trying through something else stuff to change the culture yeah well nobody that's actually this isn't actually changing the culture they have to not change their culture what our automakers right well the makers don't make any of that stuff they buy it off in two months and from suppliers and they they do assemble they stick it all in mailbox and then they figure out you know the Honda days sales event so to get people to buy them yes Andy Monroe is on our podcast saying they you know they're their dream is just to take all the parts you know literally just stamp their brand on it and be done with it everything's just out sourced exactly and and Lincoln it's weird that they do GM why are you investing in cruise why are you investing in a company like that yeah it doesn't make sense figure out who builds it and stamp your brand on it so would you consider a flight sorry Omar oh it's just kind of would you consider like partnering with those guys and then you know just it's integrated that at Build time what what is what is partnering mean just man then you don't have to sell it after if the car is built and you do it while it's being built why do I have to partner with them it's open-source software they can run it yeah I don't need me take the car we're giving you you know commas or commas the people who buy these things I mean it's a lot of users but it's also pretty much every car company um I mean they're doing it right it doesn't they don't need to work with us you don't need to call me I'm crazy I just make software all right taking the software and use the software it's not about shady backroom development deals it's not about oh well you know we could all work with you here you don't need to just take it it only works in new cars you Toyota your Honda your GM this offers are you ready so then they would just license it from you I mean what would the arrangement be did its MIT license yeah you just take it okay so you just yeah so you just want to make you basically want to make cars better safer you know that's your I'm just trying to understand your no I want to I want to win something you want to win okay yeah well after Tesla wins who won Apple re-entrant is the way you see it playing out kind of like you know autopilot kind of gains popularity people start except expecting these driver assistance features and safety features and the other projects may not end up working out and then it turns to you the iPhone gains popularity every phone manufacturer have their OS 2.0 thing but the consumers demanded you know something good so they all kind of turn to Android and all the phones I've Android iOS and Android I think that's what happens cool so we will see auto pilot and Oakland pilot yeah pretty much how do you see yourself positioned competitively like why why comma rather than using a mobile eye chip in a car that's pre-built yeah so what why Linux rather than using the Windows operates Linux is better exactly I mean if you wanna you can make me you can make the the Apple versus Android and you can make the iOS versus Android analogy you can also make the Windows Mac Linux analogy right so so so you know Tesla is obviously Mac I they make an all integrated it only runs on their hardware our vertical where Linux we're free open-source software and then mobile I use Windows they're using business development deals to sell on a per unit basis to the manufacturers of PCs or cars or whatever right it's literally Windows um so you know who do you want to be today do you want to make Windows Mac OS or Linux well Linux has the most users by far mm-hmm so you're thinking in like the age of autonomy people could take an old car and then just take the open source software and make it autonomous with comma keeping all those old cars around does that kind of you don't need to you don't you can't just take the software and put it on the car you also need to of course buy a nice box from comma which you know doesn't come with any software you can run whatever software you'd like on it and maybe the boxes will continue to get nicer and the lab more cameras and they'll cost a bit more money and so on and so forth right so you know fundamentally common is a business and as the business work until electronics company we make we make consumer electronics we sell them comedy I slash shop you can put them on your car right so it's not like the car you put the software on it you need to first upgrade your car and then you can put this offer on and you upgrade your car by buying a by buying a little common device right and you know well the common one was cancelled we made a comma - so you all can guess what's coming next right you know it's it's it's up it's it's a classic it's a classic good business model we sell hardware for more than it costs us to manufacture and we make money and so you haven't been thinking of anything like services or you know installing for people or anything like that your businesses just sell the hardware you install it yourself it's so easy to install it takes five minutes well there's their steps they just have to push the button on the phone the device right and it downloads oh then this software suit yeah the software you just you just say custom software and you type in you know the URL of whatever custom software you want to install so if you know how to go to a webpage you know how to do that no but also there's a physical install process in your car too but it is people people get this and they think they're gonna have to like like you don't even need a screwdriver that's how simple is this all right like it's it's it's it's if you can stick a dashcam to your windshield you can install this so what's what's the the other challenges I mean so right now what are you guys working on in terms of your your biggest issue I mean that you can talk about like with full self-driving or you know some of the some of the software issues that you're having so last year we shipped end-to-end lateral control which is like what I was talking about before with the curbs we do the steering based on asking the question where do the human drive the car and so we look at the picture and say where did the human drive the car right but this doesn't tell you how fast the human went this only tells you you know where the human drove the car right um so the see how fast the the person went is a little bit harder but we're working on that as well and then once we have that well you're gonna get red lights and stop sites for free you see you don't actually need in order to stop at a red light you don't need to detect the light detecting the lights advanced strategy but realizing that in this scenario humans come to a stop that's vanished at so your model just looks at the whole image and is somehow able to just see from other cars and the lights and everything that it's time to stop it's gonna be the same way you do it just like looks at the image and it's like oh yeah I should be stepping on the brakes you don't have a tiny box in your head that isolates the traffic light and then runs a separate model to say what color the traffic light is or anything like that so we're working on it we have beta versions of it they kind of work kind of wow what if you had access to like you know Tesla's data they got a you know amazing amount of data what if you you somehow had you know if they provided that as a database for you guys wait now cuz it's all 3d model we have no layout Lego with respect to just test review labeling test those raw data we have more data right now than we know what to do with we can't I mean we can't we can't process all the data that comes in there's too much man right um so we just already we already are selecting certain data W ons so no more more data would not help us okay um more labels yeah what would I do with labels know what we're doing right now one of the big things we're pushing on it's just like like the machine learning that we're doing here is it's beyond it's beyond published date of the art so there's this thing called imitation learning imitation learning is like how do I learn to play a game by watching a human play game and there's problems this gets a real technical but this is all farm called behavioral cloning and we now have stuff which can defeat you have your plumbing oh I can explain something expressively if you ask the car draw the path that the human drove um that path always starts with the center of your car because obviously right you know it's that's what the Graduate says but that's not what you want right if somehow the human ended up in the left side of the lane you don't want the path to start at the center of your car you want the path to be I actually know you want to be here alright so how do you Crouch with that that's that's tricky so good I'm just trying to understand so without any any label data you're you know how exactly are you getting these you no morals trained is it just kind of supervising itself or how exactly are you doing that without labels I'm just trying to understand we have labeled it's just not pixelate so our label data is where the human actually drove the car hmm so basically your your you're labeling it with the driving data yeah I'm labeling it with how the human act the human policy is it way that makes sense so when people are driving manually those are your labels and that trains the system yeah and when people are driving with the system I mean we can use that data as well we just have to you know keep in mind that it's with the system and it's like if the system makes a mistake you have to make sure to say like this is actually not what you want to do well how do you detect the mistake is it like an intervention by the driver or yeah okay but can you detect it any other way or is that the only way it's the only way but that's that's the accidents are so infrequent that right well I mean you're mostly looking for driver or pedestrians or people doing crazy I mean it's just human beings through some unexpected things yeah yeah yeah but like most of the time the driver I'm sure almost all the time the driver does the right thing yeah right yeah the driver would but I just mean other other road users or like but those are fine that's part of the environment exactly all right well here's a fun question we have France where like if you you know wanted to go back and learn something or go back to school or something what what would you study now now that you have always like where you are in your career what do you mean what am i what am i studying right now well you know yesterday I learned all about OpenCL and GPU command buffers and like that kind of stuff right good good yeah I think that the people that like they look at schools I can put something on my resume so I can get a job whereas you know you're looking at what do I need to get this dust yeah what's what's useful knowledge useful knowledge is good what do you look for in people when you're hiring intelligence and motivation that's pretty much it okay so Yvonne's not big on advertising how how do you see advertising is it just you believe that the product itself is gonna sell course do you put a lot into advertising no I don't do advertising all so like so if unified advertising is paying for eyeballs no I'd never do that ever um like it's it's it's I mean it's it's it's it's gross it's uh you know it's not you know I have a better opinion of paying for sex than I do paying for paying for advertising right like like if your shits good you should be able to get it for free you know what if you can't I mean you can pay for it but you know really they don't love you they don't so you want to be loved in absolutely who are your open pilot users right now are they just kind of hackers or would you recommend it to kind of just the general public if you think you're ready for you're ready don't don't let don't let anything don't let anything hold you do it like ten people without much technical background figure out how to use it sure are you motivated and are you intelligent yeah that doesn't matter if you know things about computers so I had a read question you were looking at kovat and reverse engineering yeah the virus are you still working on that no I've after working on it for like five days and then yeah I was thinking in my life like how do I actually want to deal with this this coronavirus thing is a real thing I'm gonna have to deal with it I'm gonna have to come up with policies about whether I go zoom birthday party you're Korean way too many zoom birthday parties dude it's the same it's the same sort of procedure right the same procedure that I used for anything else I applied to coronavirus I went through it all the streams are online and my own conclusion kind of was wow I can't believe they shut down the economy for this he talked about like law and politics a little bit I was wondering you know what would you like to see on the regulation side when it comes to you know autonomy and other driver assistance type laws that affect comma how would you like to see that evolve I don't think you need much and I think that the reason you don't need much is because everyone's in are aligned you need regulation where incentives are not aligned like for example you know I think we should have I think that that we should have a lot more tax on fossil fuels because it's a negative externality right there's an externality you buy that gasoline it's not just the cost the gasoline that's also the cost of the the right and incentives aren't aligned to their right the incentive of the rest of society is well you know we don't really want you to burn that gasoline that's why there's a tax but when it comes to something like self-driving cars well you know crashes don't benefit people they don't benefit automakers and they don't benefit companies making autonomy software there is nobody who wants cars to crash so you don't really need you know I mean eventually you'll put some best practices regulation into effect once it's like I mean this is what's always happening on both the normal process of automotive regulation will work perfectly fine for autonomy do you mandate seatbelts well not with seatbelts are new unproven technology but once 50% of cars shipping have seat belts and now it's just like the lowest cutting cost manufacturers okay now see felt romantic airbags went the exact same way and I think eventually we should do the same thing with with level 2 autonomy systems with driver monitor and even - but yeah no sorry you want to drive a car you got to pay attention like so your drive ramanujan gonna like watch the eyes the pupils if they're dilated if people are high or if they're drinking I'm curious we've talked about it yeah we've talked about it and I'd be I don't know I mean I wouldn't want to say like someone's drunk based on how they are their pupils are dilated but you're driving the car by swerving yeah absolutely hmm right I mean you know at all we can do look the driver monitoring this is an this is a common policy I'm not trying to monitor you when you're not using the system all right we don't drive them on and doesn't work when the system is not engaged you can do whatever you want the system is not engaged but if you want to use common software well enough that's the trade-off you get the autonomy and we watch it all right we don't really watch you it's all local to the thing but yeah I mean that's kind of the trade-off if you want this convenience feature that kind of contract with devices you have to agree to pay attention and that's what level to us so what point do you see the softer actually good enough to remove the driver laundry ring not anytime soon no no and I think yeah I mean every time we hear oh yes autopilot is gonna be safer than people by the end of the year yeah well you know we were gonna drive across the country in 27 Tesla's gonna drive across the country New York to LA 2017 notice engagements still hasn't happened right so um it's gonna be a long time why do you think that is I mean what does the the the last you know 1% or whatever of the problem or or is it bigger than that yeah yeah it is kind of a the last 1% problem I mean it's exactly what people think it is so right now ah comma can go about a hundred miles between disengagement like a highway you'll probably able to drive 100 miles a lot of piles pretty similar yeah I mean okay so humans go about a hundred thousand miles between disengagement so off by a factor of thousand right so in a way that's not even 1% that's like Lincoln Lincoln saw like where 99% done no we're actually like point 1 percent up or off by a factor of thousand good thing is it will grow exponentially but you know that next good thing well cuz we're where let's say let's say the number of miles between dis engagements doubles every year right so if you want to get to a thousand whoa let's see that's one two two four that's like 10 doublings so it's gonna take any years Wow that's a good perspective I mean I guess that you know the like I think what George said is really true you know Silicon Valley doesn't work on technology anymore people don't work on technology they don't work on actually building new technologies that haven't existed before it's really just kind of like you know building software platforms that kind of you know transform these legacy businesses into you know kind of tech businesses because it's using software and there are a few people actually working on these really hard problems like autonomy but this is something that has the potential to really be transformative to the economy to the way we live life and everyone's just kind of expecting these self-driving car companies to finish one day and I don't think that's going to happen I don't think the legacy automakers are going to be able to deliver I think most of these VC funded Robo taxi companies are not going to end up being successful before they run out of cash so I think that leaves vision-based players like Mobileye tesla and kama to really try and deliver something incrementally that people can use and I think this is going to be the decade of autonomy and it's gonna be a decade-long project so I'm excited to see how that all plays out I think the key distinction actually isn't even vision versus non vision it's whether you've shipped products the biggest distinction in my mind is okay if you want to add a new car to your network how much money does this cost you well cost way more like you know for $2,000 it costs comma negative a thousand dollars and it cost Testament like negative ten thousand dollars so that's the big distinction because mobile line it cost mobile line negative 50 bucks right there's the companies who would cost negative money to grow and there's the companies who cost positive money to grow right the faster they burn the faster they grow the faster they burn cash which is why they haven't grown but with Tesla with Tesla no comma it's completely opposite that's 100 percent right I would say that and also usability like how many people are really using the product it's refreshing to talk to George I mean his critical thinking skills and feedback are sharp as we all know so it's really I enjoyed it but um as you were doing the reverse engineering for the the genetic the genome did you find that it was very similar to coding software the way that it was laid out because I mean you're doing it on a computer but oh so genomes aren't software there is no processor which in turn there are a lot more like cad models so it's it's like it's a camel it's a 3d model of a protein and you can't exactly read the reading the 3d model is pretty hard and then once you have a 3d model figuring out what that 3d model does is super hard as well so yeah there's some like big downsides to it I'm not that bullish on uh biology did you like it though was it enjoyable it's enjoyable until you realize it's hard to take this further until you realize that I don't have a simulator I can't test it in the avaya so what I'm like working on something that's code right either have like a simulator or I can test it right on the device and for bio you don't have a good simulator and testing it right of the device involves pipetting yeah it was fun to watch you like talk about though the codons and the protein in it yeah that was interesting anyways yeah I was like learning but you know no comma bio anytime soon well I actually have another question to kiss it this is like you know long-term big vision thing but you know past comedy I know you have you talked about with Lex Friedman one of our good friends on Twitter but you had an interview with him and you were talking about like you know your your vision for the future has that changed much or I mean you even talks about maybe a future a digital girlfriend or something like that in it absolutely that sounds fascinating look you already have like like digital girlfriend like digital companion apps they're like big in Japan and stuff that's not really what I'm talking about um the I think I said this on Lex I see it more like if I want to merge with a machine what does that merger actually look like and I mean I know some couples who they're practically one person so so I mean you know this brings up like newer link and stuff like that would you be willing to try something like that be like what do you want to be oh no no no no no no no no I'm also not getting on a rocket to Mars until there's five-star hotels there and there's billions of miles of safety data no transferring your consciousness super-machine of those you know I don't know if I'm like risk-averse or not you know what people are like all that so you know oh that's so that's such a risky thing like to like say that I'm like no I can say whatever I want there's no risk but you know to actually like sit my ass on a rocket oh no to stick some chip in my brain oh wait till a million people already do it and then you might I'm certainly not opposed to either who those things well you don't see unity using commas kind of risky no no well that's because you wrote that some of that goes no it's cuz I trust our safety Monica um so it's not I don't you so level - I don't ever trust the comma system to do anything I would definitely not trust it to like fall asleep or get in the backseat or anything no no no no but do I trust you using it the way it's intended yeah I mean you know money well money where my mouth is I use it every day like every day that I Drive I use the system I've used it on road trips the whole safety bottle is basically built around like the driver pays attention and they're always guaranteed by the system to have at least one second to react and then as soon as you step on either pedal it completely stops doing anything so like if you trust your own reaction time in one second you're gonna what to do so I mean I trust myself to try it it doesn't disengage if you turn the wheel right yeah we don't disengage on wheel no tell you why because the Tesla doesn't disengage on gas alright you can step on that you stub the gas and autopilot say engage but it will disengage in the wheel um see this seems weird to me because like if the car is applying too much brake the gas is not the opposite of the brake hmm and the brakes don't the opposite of the gas they're completely separate things but with the wheel if the car is applying too much torque to the right you can just apply torque to the left and torque to the left is actually the opposite of torque to the right right so do you use that to label the data train the system well grease corrections to train the system yeah of course disengagement corrections they're great yeah that makes sense right so so I know that you had described you you feel like it's not that optimal to use the disengagement on steering but typically when you disengage it's not because you just want to do something it's because it might be the system might not be doing the right thing so you're correcting the course right but and I mean there have been cases where people like oh I want to take over now it's going straight and you're like oh the simplest thing is just to you know disengage and then you kind of wobble the wheel which isn't isn't desirable so maybe that's what you were talking all right oh oh when you with the Tesla like if you if you're going straight and you're trying to dis again but then it's not desire but then you wouldn't do that you wouldn't have you have your car with open poet I actually want to do this yeah amy-jo feels it feels different from autopilot like autopilot holds the wheel very rigidly and you have to apply a lot of force to like get it out of the autopilot you get used to it after driving yeah well wouldn't work you're get used to it you know yeah you know exactly how to do it but like open pilots very RR to our policy is very different from that we will not fight you at all you can we when we detect any human to work on the wheel we ramp down our torque and proportion so you can like if it's going a little too much right you can just gently nudge the wheel to the left and it'll go back but it works if the steering is more cooperative cooperative than one person's in charge to the other person like that interesting yeah it's interesting to explore different kind of UI models for that yeah there's just kind of like this one you know product on the market but people haven't really explored different ways to kind of build products for people different user interfaces they're like yeah I prefer the open pilot one to the autopilot one but I think I think it's mostly a matter of opinion I also don't like my auto pilot will separate adaptive cruise from auto steer I'm like we don't do any of that it's either on or off so what are the other things that you think about autopilot like how it can be improved oh how autopilot how autopilot can be improved well so I I'll start with like what I think it does well it does better than it feels more like locked on Rails than our system it feels more like you're you're just like driving on a rail huh it also right now has a significantly better longitudinal policy than us like it's been our mergers and stuff we're not very good at merges we're not gonna like detecting the car over here I'm which autopilot is gotten a lot better at in like the last year uh but yeah I feel that it like the few times that I did have it mess up on my road trip and that's not pretty badly that the the flipside the trade off to that locked on feeling is it'll feel just as locked on as it drives you into a wall open pilot feels less confident when it's less confident because we we passed the model output you know more into the like oh we're closer to the model so like when it's a little unsure you know it feels a little jittery whereas autopilot feels rock-solid even when it's completely on shore even it's completely doing the wrong thing so I think that's a trade-off um I think that this is the driver monitoring is vastly superior to the wheel touch though yeah I mean the wheel touch at first the first time I used I used autopilot I thought that you have to touch the wheel and I hated it because I accidentally disengaged it like so many times and then when you took when you touch the knob it's like asthma it's hard for people to figure out how to hold it just right not too gentle not too soft right like you said I think we're still still a ways away know like getting good enough it'll never get good enough eventually Tesla's gonna put some infrared LEDs on their new model threes and those ones are gonna have that's definitely what's gonna happen they will eventually come around to driver monitoring it just may be a white so you don't think that full soft arm will be before that no way well which amends will you call full self-driving you mean self-driving where you don't have to pay attention no way no way I bet I bet ten grand right now that uh Tesla ships driver monitoring before they ship anything where you don't have to pay attention you know if you mean full self-driving like navigating on autopilot will make a right turn at a red light that much at first sure that's not that hard right but now make that right turn you know drew it a million times and don't mess up well yeah that's the thing is people confuse supervised autonomy with unsupervised autonomy ones vastly harder than the other well they're this I mean there's no categorical difference this is a question about how often it messes up reliability yeah so I had so yeah like I'd say autopilot messed up pretty badly once every hundred miles like once every hundred miles if I didn't correct it I would have crashed so no they weren't like like huge things and I could usually see them coming and and was that like involving other vehicles or it's just like on its own usually involving other vehicles usually involving other vehicles doing dumb stuff yeah maybe maybe we'll wrap it up and yeah it's been it's been such a good time haven't you you know having your honor show George has been real fun chatting with you I hope you've had a good time as well what's the website for people who want to you know get a comma or try it out comma dot a I come I didn't I you so there you go all right great all right thanks guys there were Tesla signing off [Music] you

Channel: Third Row Tesla

Views: 94,903

Rating: 4.8591857 out of 5

Keywords: Tesla, Electric Cars, Technology, Vehicles, autonomous vehicles, artificial intelligence, machine learning, openpilot, autopilot

Id: FIbvt4_InyU

Length: 80min 3sec (4803 seconds)

Published: Sun May 24 2020

Reddit Comments

[u/TomEnom]

A bit tangential, but I would add that there seems to be a bit of a battle going on between electrik.co and this third row podcast, with electrik taking the position that Elon can and should be criticized, and these guys on the fawning side, and attacking people who criticize Elon as being disloyal.

As someone who owns a Tesla and is a fan of what Tesla has accomplished I am firmly on the "Elon can and should be criticized" side of this and have always appreciated electrik's willingness to do so. The one time I watched TRT podcast I was immediately turned off.

I realize this is a over simplification, and my bias is showing through, but am I unfairly mischaracterizing what's going on?

[u/DoktorSleepless]

Pretty interesting. He shows off an old prototype from 2018 with six cameras running on an Nvidia Jetson TX2. He was in talks with Nvidia to be sold the chips at $80 a pop, but the head dude in the autonomy division was fired. When he started talking to the new guy, they wanted him to buy the entire module at $400 instead. So that project was cancelled even though it cost hundreds of thousands of dollars to design.

EDIT: 30 minutes to 1 hour mark is probably the most worthwhile part to listen to. Talks about how his approach depends a lot less on labeling than other companies. And about why the one camera and processing power of a smartphone is less of a bottleneck than people think it is.

EDIT 2: Bonus go brr meme

[u/plun9]

Interesting. At 31:07, George says:

We look at a picture and we ask the question, "Given this picture, where would a human drive the car?" We don't need to explicitly detect a curve in order to know not to drive into that curve, we just look at the picture and say we're a human driver and the answer is not into the curb.

This sounds like end-to-end machine learning, which is what Wayve is also doing. Amnon Shashua and Andrew Ng claim that semantic abstraction is more promising; because, with semantic abstraction, a lot less data is needed in order to achieve the same results.

[u/Keokuk37]

They seem so clueless. Couldn't watch it.

[u/bendandanben]

Damn, even more excited about this than the huge Elon episode!

[u/6ix10en]

Somehow they managed to make it a huge circle jerk even when having a tesla competitor on

[u/olaisk]

I like George but I don’t see his ideas ever working. I’ve seen him speak on at least 4 podcasts and it’s tough watching him for even 30 mins. It’s like he wants to become a brain surgeon without the training and seems to be missing a dimension of what’s required to 1) get a business up and running off the ground, 2) keep it alive.

I fully believe we should give people like him chances, but the reality is his oversell of practically everything he’s worked on is a massive turn off. At some point, at least one thing you’ve done has to be successful - in the market, not raised venture capital successful.

The coding sessions are pure cringe for any software developer. The fundamental cluelessness of what it takes makes these guys look like children.

Mods — you need to answer for this, stop hiding and muting. You banned my account after a deleted account personally attacked me. I blocked him, I’m assuming the one I blocked was another mod account which was deleted. Personally attacking someone IS against the rules of Reddit. Who did it? And why did you ban me. Muting me makes it appear like you have more to hide.

[u/lemonlemonade]

Super interesting how he's describing the more generalised approach, with what's driveable and what should I do with X type objects instead of trying to label every detail in the scene. I feel like it's closely related to false positives as well, which Tesla has so many of currently it's not even funny. Could watch George talk about this stuff for days.