Android Automotive

mmm all right so good morning good evening or good afternoon wherever you are hopefully everyone can hear me appropriately I am going to get started here in a few moments just going to make sure everything is operating properly we do have some private attendees who are joining us over either phone or some connect session and if you are one of those I'm just going to ask you to mute your mics unless you actually need to use it so give it a minute or two here just going to make sure that all the live-streaming indicators are a go and I'm going to get started all right if you are experiencing issues whichever way you have to talk to us give it a try we'll do our best so you should be able to type some stuff in the YouTube chat if you have any questions there or comments and if you are joining us through one of the private channels of you Adobe Connect or elsewhere you know feel free to voice your questions or comments or whatever else that you want to communicate and generally speaking wherever I get information or feedback I will send it back out via the live streams so that everybody has a record of it generally speaking making sure that everybody's on the same footing here okay so since it appears that nobody is joining my audio conference which I'm going to drop off of that if you're one of the participants who might have invited just give me let me know by chat on connect if you want me to join back generally speaking I'll wait you know I'll give people a heads up to join before and since nobody's joined at this point I'm just gonna go ahead and disconnect from that so I can still hear you off of Adobe Connect if you have any voice comments or questions or anything of that sort okay so what I'm gonna do today is I'm gonna go through what is publicly available regarding Android automotive and I want to highlight that right out of the door here because there is you know obviously this is an ongoing effort on Google's part there is still going to be things that are likely to be announced things are they're still in the works and so on so what you see here is just what's in the Android Open Source project you may obviously have seen or if you are one of the partners who are working on this have access to additional material or additional information but overall you know what I want to address here is just the things that are that you can find out just by perusing other sources or Google's websites okay so without further ado and by the way the slides will be up there after the presentation they are will be they will be on the usual terms that I use which is you know CC by-sa so have fun with that if you're not familiar with my work I wrote the embedded Linux book for Riley in the early 2000s the embedded Android book in the early 2010's and if you go looking in the kernel sources you'll find my name at least in one file and you know until somebody discovers that it's no longer of any use and obviously it'll it'll see you know it's in the days at that point in any case I've been involved in you know open source for a fairly long time I'm widely known for having contributed the Linux race toolkit which was one of the first racing tools if not the first racing tool for the last kernel and have been enjoying effectively helping people take open source software and putting it in all sorts of weird devices including cars over the years now despite my background as I usually tell people as a disclaimer I don't actually claimed to know everything I don't I wish I did so you may have questions for which I've absolutely no answer and you may actually know more about some of the topics that I'm about to discuss then myself and in that case you know either peel please fulfil her share additionally I'm you know an eager learner and if there's anything that I say here that just sounds too crazy for you then a by all means please do say so on the on the public chat on YouTube or elsewhere and you know will correct the course as as it as it needs to be if that has to happen why would you want to use Android in cars I think that's kind of like the first question to be asked and by the way some people have commented to me ahead of this presentation that how the heck are you going to fit you know all this material in in whatever session that you've been on YouTube and so generally speaking I timed those things to span about an hour an hour and a half and generally if you look at my other presentations on YouTube they've lasted about that long about 90 minutes so obviously this is not going to be the same sort of coverage that I would give you if if this was a class setting in a you know curriculum where we were doing hands-on or anything like that but in the case of this particular session my goal and I guess this is one of the first questions I want to answer before I even head into this question my goal here is to provide you sufficient level of information that you know where to start looking for things and that you understand generally speaking the architecture of what you're dealing with and how the parts kind of kind of fit together if you are familiar with the Android internals already that should be good enough to give you a good head start into exploring the additional things that have been added for car in the AOSP I was thinking this doesn't cover things that may not be there but if you want to get into the use of Android in cars and want to know what's in the USP this session will actually answer many of those questions okay why would I want to use Android in cars well that users have gotten accustomed to the you know pinchy zoomy swipey kind of experience on their phones and they increasingly expect this on many other devices so you know be it cars but be it also kiosks be it medical devices if they have to operate one anything else you know this this starting to get kind of like the experience that people are expecting from from their devices so that's I guess a primary motivator but a Nord Internet itself isn't the only operating system that can pull this off other operating systems can probably also do the same thing and and you know sometimes they do cut the the pushback say well you know I could do this with X or Y Z and yeah sure but then you go down list there and the other thing that the users are increasingly also expecting is an app and developer ecosystem right so they expect to be able to have apps that they can install for whatever you know service social network what have you that they're accustomed to using and yes some of these are questionable in a car a kind of environment do I actually want you know my you know friends pushing me videos that play will play on my or whatever else that's questionable but still you know this experience of okay I want to add this additional functionality go ahead and install it is something that Christa customers are users are actually increasingly you know minded to actually be able to to pull off and and the one thing that you really can't beat Android as is really the developer ecosystem so there are you know a large number of developers out there there are familiar with the inner ecosystem and developer AP is and that can pump out applications for those platforms this this platform if you go ahead and you use something else it's not that you won't find people that can't actually code for your API there are very talented people that know how to code for other operating systems for sure but it remains that as there's a huge Apple application developer eco eco system around Android and that is very difficult to beat that really is is is hard to replicate and people have tried right and some people have just offended that idea and just embraced Android and one with it obviously the other thing is that we're increasing expecting our our devices to be connected permanently in one way shape or form and moving forward other things like audio assistance and real-time data about what's happening in our world you know that too is something that is increasingly expected from users so for all these reasons I think that Android is you know a good candidate for these things because it's already doing it well on the handsets and it is open source right so it's it's available to you to download do whatever you want with it and we can talk about challenges a bit and you'll see here down there that I've got you know additional challenges I put you know a toward is controlled by Google that is certainly something that is always in the forefront of the the folks that that we've talked to for that are you trying to or have started putting in the cars okay what if Google pulls the plug what if you know Google does this what if Google does that since it's open source at some point in time you know you can always say okay well you know that propriety everything that you're giving us is no longer working or you know you you're not giving it to us in some acceptable terms hence we'll stop using it will go something else so that's always a possibility and I think that's always you know somewhere in the back of the mind of an eery decision-maker who's looking at at using a node now there are challenges irrespective this right that that present itself when you look at trying to integrate an operating system such as Android in an automotive use case scenario sorry typo there so integration in the existing car environment so how do you integrate Android into you know a car because it's mainly you know geared for you know phones and yes they we've stood over the years I have at Google has added support for other devices but this is where kind of like its its its natural environment so starting from this go in a car there's kind of like huge gap and how do you actually make this happen so one of the things that we'll see here is the various things that have been added to the AOSP to make that possible another concern obviously security and that actually has you know technical ramifications but also possibly some political or long-term ramifications that aren't necessarily just technical some of these are going to be business issues so you know security for this is one thing and if this stops being updated in two three years in terms of software updates it's not that bad because of the other day I'm probably changing this in three-four years or you know whatever at some point can I get bored of this and some new features gonna come out that's gonna make it such that that I won't update or just my you know contractor srinu's and they give me a few e free phone so I will actually probably replace it I'm unlikely to replace my car that rapidly or you know at least that's not kind of like at least for now the way people think about cars there are various changes in the industry where you know people thinking of pools of cars and that sort of stuff you know we'll see what goes but generally speaking their current pattern is person buys car you know expects it the last for a long time how do you do how to conduct security updates for operating systems that have been out there for 10-15 years that's a bit of a challenge especially as the root of this software since it's open source eventually stops being updated you know what do you do with kernels that fifteen years old challenging all right so that's it's not something specific to Android but it is something specific to trying to take an operating such as Android and bring it in the automotive use case scenario upgradability how do you continue upgrading something if at some point the future versions of this operating system start demanding more and more resources of the device and then the device that you issued at 10 15 years ago or whatever you know seven seven eight years ago no longer can actually technically on the hardware side be updatable what about power consumption the car industry is very sensitive about those things you know I've had numerous conversations with people in the car industry what it said you know there's no way that we want something eating up our battery all the time and and this is kind of like a sort of a non-negotiable mindset so how do you deal with that when Android doesn't presently at least in as far as I know provide us with any sort of hibernation capability the rear view camera right this is something that has to come up quite rapidly when you put the car on and Android generally speaking in terms of you know booting quickly we can have have some issues because there's a lot of things that happen at boot time um also storage we're leveling so if we're riding off into the storage device how long will this last you know if we start you know doing mobile phone style writes on to a storage device that's supposed to last us 10 years or 12 or 15 years how did how's that gonna work out so those are kind of like just basic technical you know challenges some of them more you know that require a bit a bit of business forethought or at least some sort of analysis as to okay we're thinking about bringing this operating system here into the into the device into our cars it has some software challenges but it held so has some business challenges how do we face those things so obviously the the other items I mentioned which is you know this is controlled by Google again what I said earlier applies and the fact that it's near toward replaceable devices so this is also that also something I touched on so generally speaking my current thoughts on error autumn or the following and this is again based on what's available out there Google may be working on some amazing stuff that they've shared with their partners that we don't know about but for for now this is what's what what I can see from looking at the USB so it seems that for now the the final architecture approach with regards to certain things in the USP is TBD I mean if some of the things I'll point those out as I go some of the architecture choices that they've made seem to point to the fact that they're not exactly sure how thing where things will land and sometimes they actually quite deliberately say so in their documentation they say we're not exactly sure how this is gonna be done in the end presently we're doing it this way we'll see later on how it actually works out this is something I mentioned earlier so there's a lot of stuff happening which is not public obviously it is in flux there are some key issues to resolve I've already kind of touched on some of them at on my previous slide but I will get into a bit more detail later on as as we talk about the the specifics of the of the SAC general purpose app development even according to their own documentation is effectively in alpha so this idea of being able to load on your head unit or your ivi some sort of application the third party is publishing is something that a lot of people are sensitive about and for a good reason right this is an actual car I'm driving this thing I don't want it to just start you know randomly interrupting my my driving with something that has absolutely nothing to do with the context of my concentration which could actually cause some some issues all right okay so let's talk a bit about what I want to cover in detail so we're gonna go over overall into an overall introduction just kind of like flushing out the nomenclature that they have out there regarding the use of alien cars then we'll start looking at the architecture what are the pieces that have been added to the Android architecture and we'll go through each one of those one by one and then I'll start talking about some specific subsystems that are documented in there either document or documentation or that have some sort of information about them in da OSP and to assist me and all that I've got here a into an emulator image of the of the car system effectively which I'm going to use just for oh and it actually just crashed that's awesome sorry about that let's just restart this guy so this is kind of like an easy way to get in to looking at what's what Android automotive gives us so you can see here doing it's a little you know boot animation and eventually it's going to get me to the launcher screen for automotive and this is what you get effectively for the emulator okay so if you have started looking at that you'll see that they're using kind of like two things and they're separate but for me they're confusing so when they refer to an auto versus an automotive so the Android auto really is an application app that's on your phone and it projects onto a device that is on Android auto capable so that means that the ivi is not running into it it might be learning something completely different it could be also running Android but whatever so if whatever it is it's running the application on the phone can then project down to this device some-some app content that is not the topic of this presentation what I'm interested in is the actual use of a toward itself in the car so that is it's running on my I VI or my head unit in some way shape or form okay now in terms of nomenclature it's not I mean at least reading in one of the Articles that were that appeared earlier this year it's a bit confusing not I'm not really sure what they well they're what they're they'll end up calling us so there was reference here by some of the Android folks to the use of embedded or native or PowerBar Android let's see where where that actually eventually ends up landing in terms of marketing terms so what I'm interested again is what has to do with the use of a Nord itself on the unit not the actual projection mode that you've got available in there so the interesting thing is that the for the car are an extension to what is already there in the AOSP architecture so if you've seen some of my other material elsewhere like the embedded courseware that I've got out there I mentioned there are different flavors of Android that are available and that in most cases the underlying architecture the same except there are some editions that are specific to use cases and this readily applies to the automotive use case where you've got some bits of pieces that are added throughout the stack to actually make the Android operating system fit for use in in cars some of the internals documentation that you see on source.com is someone add a sink out of date so or incomplete so you know your mileage will vary if you just go with the with the sources there is a lot of app developer documentation but from my perspective looking at it from the internals part it's a bit of work ill I mean there's there's a lot of stuff but then if you look at what you can do with it it's a very very tiny use case I don't know of any reference hardware that's for automotive if you do knows of such a thing I'm happy to to learn about it but what I've been using personally to explore this is the emulator target which I just showed you so if you just do lunch you can go see that there's a a car some car targets the one I'm using right now is a OSP car x86 64 and I'm actually running in as - and is not user debug - a difference and then you can effectively shell to it and go take a take a look at what's around now I'm doing this on 9 just because that was what would have it was available as I was preparing this presentation but 10 just came out actually yesterday and the sources have started trickling out so I do have a successful sink of 1000 r2 this morning I'll try to use that for part of the explanation if I can but you may find that my slides may not be in sync with what's in the with what's in the actual sources of 10 because some few things have changed so I might just fall back onto on 290x because that I know that actually works and of course I didn't get the chance to actually build ten and give it a run and try to run the the car emulator in there but that might be something that I'll attempt shortly okay the architecture of the car editions for Android okay this is the general in behind the blue boxes here is the general diagram I've been carrying over the years regarding androids architecture and it hasn't changed much for those of you who aren't familiar with this let me just give you really quick rundown in this so you've got the Linux kernel at the bottom which gives us everything that Linda's gives to other Linux distributions except it's got a few bells and whistles to allow this stack here to run on top of it and the color coding here in my slide is a gray is generally C code or native code yellow is system service or Core related framework related Java stuff and then you go up in the top here and the worst of his regular application so on top alone is Colonel we got a bunch of libraries Bionic being the main thing here with the C library am I gonna concentrate on that this this time in my explanations but what I want to show you here is that you've got these automotive hal stuff so behind this box that says you know hardware abstraction layer so the hardware abstraction layer part for automotive is called automotive and I'll show that to you later when I go through the code you've got a bunch of mini of diamonds nothing here particularly to point now we're learning automotive you've got in an in to tie box which of the shell this is the virtual machine that allows us to run java code you've got the system services and what's interesting for us is the car service so there's a car service that is running actually as a independent process you can see it actually if you type ES and inside the car service there are several car services so this unlike other system services seems to be a multiplexer so effectively you called the car service to get a handle on the service at the specific car service that you want to talk to and I'll show that you a bit later in more detail it's an interesting architectural choice that they've made there I guess it stems from the idea that since it's a work in progress we don't know what its gonna look like at the end then maybe let's just say that we want to have some you know calf holding that allows us to get into a car service or get a handle on car versus service but since we don't know the definitive list of the car services let's just make this TBD and we make a very simple interface that does it get her on the service that's behind it so I'll show that to you what this looks like a a an in detail but just remember that here unlike the other system services which you look at the car sir they're like a power manager or package manager or activity manager which has you know kind of like an interface and then you call straight in to that interface in this case you you call it service to get the handle of the car service specific car service you're looking for and then you call the interface of that thing to get the services that it provides there's a few libraries on top to give access to those to the service and then you've got some built-in applications inside the AOSP that are you know presumably example applications for you to look at how you can do a car launch or how you can do in hvac radio maps or whatever else that you've got and in terms of applications that would come from the store presently it seems that the only sort of application that you could publish this or for automotive is is media applications that's at least according to the documentation that I rechecked this morning seems to be what's what's available for you if you want to look at where things are in the sources in the actual AOSP so if you look at the house stuff it's under hardware interfaces automotive if you want to look at the service itself it's under package of services car service and then the libraries are also in the same substance kind of subdirectory the apps themselves are in packages apps car so that's where you'll find most of the pieces of the of the that are used are facing in terms of car application so as I was just saying the car service is effectively a multiplexing service so you've all got a very basic a IDL here that gives you a get car service which is the main thing that you're trying to call here and let me go show you here the the packages that will the implement the the the interface here that your you're going to be talking to so let me go grab my Android sources here so this is this is nine and will you know rapidly go check out on 10 see if it does the same so if I go under packages services car and then we're going under a service SRC come Android car and I'm gonna have a RI car imple here and so that's the actual car implementation and you'll see that you've got all these services which are actually being declared so I've got a car parameter car power management service car package manager service car input service driving state service you know and you name it and there's a whole bunch of these and all of these again are fetched by using the official I car AI DL and then you go and get a handle on that on that item underneath this and if I go checking here on say 10 decks just kind of like giving us a sneak peek at what's what's being done here so let me go in my sources here for 10 let's just do this if I by the way they'll you know this is my first look at 10 so there's some stuff here that I'm going to be discovering as as I'm talking to you but overall the first look this morning seems to me that at least the top level things are pretty identical and if I go under services I still got the car here and if I go under service itself going on I start seeing calm Android car I should still have a I car in pull which has about just at a glance if it seems to have 10 kilobytes more than last time and then you've got you know possibly additional services which I haven't compared at this point so you might want to chill you want to take a look at that alright so this is my car system service and if you want to see it on the device you can actually it's actually packaged as as an app so if I do a BB shell here and I do a PS and let's go looking for just car and you've got here the sorry become Android car right here and so that would be effectively the the package that is encompassing that service and you can go check the act the sorry not this but rather here if I go out to the and would manifest here this happens to be my package in would come car and you'll see down there that the application is marked as persistent so this is going to get started as soon as the system boots so it is packaged as a system service which is independent of the existing system services which are under the aegis of system server Java and so on and so forth so this isn't an independent app that registers a system service in that system service then exposes its services which are these 3e IDL calls on to the rest of the system and if you do a service list and again look for a car you're gonna find we've got car service here and you can just go ahead and do something like a dump cess car service and you get yourself effectively dump of this of the information of those various services underneath there so if I scroll all the way to the top of that you'll see that it's got the star kinda like things so a package manager package managed service which has these things then you go on to another one to input service what it sees and so on and so forth and by the way I just noticed actually earlier that you can actually do a car service - H done sis car service H and it's got a command line here to kind of like talk to some of the internals of it and toggle a few fields here and there so that's a quick look here at the car service again the first level a IDL is just together and then the actually ideals are underneath that so if you want to see what's specific about the car package manager or audio or whatever else then you have to take a look at those things I mean this is a snapshot I grabbed from last time I looked at the sources which is again nine dot X based so only a handful of these are actually publicly available in other words a lot of these things are hidden away from from app developers so the the API is not necessarily available to to to to general purpose use but more inside the AOSP so regarding the car api's so those that are available to or document it to application developers for use by play they're really just for media apps and it says so quite literally on the documentation and you can look at the Google Play services themselves as Google Play services and a lot of OS is still in alpha stage and that is accurate and that is still current as of as of this morning okay now the AOSP api's that are available to apps that are built within the AOSP are from my perspective a lot more interesting and so if you want to take a look at that stuff there's the support library and the car Lib which have those things inside of them so that will be effectively on your in your asp under let me go back here under a packaged services car so you've got here the car Lib and car support Lib those are the ones that you want to take a look at for for the for what's available for use in terms of api's in the AOSP for applications that are effectively built against inside the AOSP now those are similar in package nature as what you'd find a frame rich base core but they are specific to the use of effectively talking to if you go back to my diagram here we're talking about having libraries that allow me to talk to the actual system services which are housed in that separate car service okay so the car apps themselves now that is a bit interesting and let me take take you through some of this here so just kind of like give you a quick demo of what you gone there on the emulator so you've got a few applications some of them are just step placeholders so if you like on the actual navigation thing it says okay no Maps application is installed please contact your car manufacturer and in fact this placeholder is found as is if I go into packages apps if I go here under packages apps and I look under a car I'm gonna find the what is it again this guy is okay now let's try go this one it's thought it was here or is it in the services I'm not or think it's in the services I got confused so let me check here under packages services car car maps placeholder that's it so your placeholder here in fact if you go to the layout I think it has even here the actual same string so there this is the actual string here which says you know no Maps application is installed so is this just kind of like a you know stopgap thing ok if you go for the media so what do you want you want the radio so here's the radio application you know and effectively this allows you to you know I'm sorry the radio application going back here medium and effectively what you the radio app itself is under the packages apps this time so this one this packages apps car radio and then you'd have an example radio you've got a dialer right and effectively same thing here that one is I believe as well underneath here under dialer okay so just basic dialer application and you've got a few additional things here like the settings and so on and they also have a specific input keyboard so if you go here under let me try to see here if I go for a search and I start typing yeah so this input method here the keyboard that you see here which is effectively a specific input method for cars is also available as an app here into line IME now the one that's kind of interesting here is the is the HVAC application so if you click here on the right hand side here well it looks like this wavy can like windy thing this gives us the the HVAC user interface and obviously you know you can um you know increase decrease temperature and so on but the interesting thing here is if I start kind of clicking on other stuff the HVAC application quote-unquote remains on the top an HVAC here in order to pull this off does something special that I've seen used a handful of times in Android which effectively allows me to overlay some display so if you've got something that has to say overlay the display in a fashion such as this where you're actually you want to monopolize the what the user is is actually seeing and not allow anything else to run then what you can do is what the HVAC application does here and if you open the manifest by the way this doesn't actually have an activity it only has a service and if you look at the how the UI is implemented so if I go under sis SRC comm Android eight-track and if I open you the UI service here you'll see what it does is actually it talks straight to the window manager so it's going to go actually set up some parameters these ones are actually test parameters you can see here there is does not exist a way to get the current state of the system UI visibility from inside this is a service thus we place something that's fullscreen and check if it's its final measurements as a hack to get that information and the actual display actually happens and this function or method called look layout HVAC UI which is underneath here is going to say not focusable but it's going to say a type display overlay and this is going to make sure that it's on the top there and therefore you know whenever you you you bring that to the foreground it it should actually be able to be the first thing on the top and you can see here that in this case we're add view to window manager and track so they're just gonna add this this thing and effectively start receiving you know whatever input is is done on that on the interface so in other words and if I want to get something that overtakes the display in this fashion and again if like if I click on other stuff it's still happening in the background but the a track is still on top and if I can remove it it goes back to my other application so that's generally can like the what you've got and oh I kind of skipped the launcher but there is a car launcher obviously and elantra here is under just plain launcher and much like any other launcher if I open the manifest it's gonna have here an activity which is going to respond to intent action main you know category home default not kind of stuff so if you got something like that it'll it'll usually respond as as being to home or you know kind of a default launcher for that device so that's for effectively the applications and probably if I go here in in 10x they might have some more so if I go here under packages apps car they do seem to have this let me see you just kind of quickly comparing here visually those two things I'm sorry I wasn't here going up to apps car so you've got a cluster which has been added you've got one speaker still there if your local media players there you get a messenger okay that's - that - is there a notification overview notification is new sending this stream system updater yeah so this is fairly good similar already here link viewers new as well so you know I'll definitely explore this in more detail a bit later on so that gives me effectively the user interface that you know is going to come on top of the system service now in order for all of that to actually operate let me go back here to my architecture diagram so we've looked at that I've told you that you've got pretty much in terms of public API is the only thing you've got here is media and I've kind of looked at how this works and now let's take a look underneath the hood effectively in the automotive haidle or effectively hardware interfaces that are provided to us so if you look at the D haidle there's effectively three Hybels which are exposed audio control EVs and the actual vehicle itself so if I go out here all the way up to the top I'm sorry let's just grab this going under hardware interfaces and then automotive I've got here three sort of interfaces that are exposed and if you go here in ten it's effectively the same thing when I was looking at it this morning so I'm sorry hardware interfaces automotive we've got still audio control EVs in vehicle and if you look at the what's inside the EP eyes have been changed so audio control still 1-0 EVs is one zero and vehicle is two zero and that is the same thing here in nine we're still at those numbers so the Hal at least what's publicly available from what we can see the haidle interfaces haven't actually changed so let's go take a look at primarily the vehicle Hal so what do we have here what is the vehicle I'll actually give us and really this is a kind of like a property monitoring sort of API so get all prop configs you know provide a callback you know you got a set or here you got to get her you got a setter you got to subscribe you got an unsubscribe and that's pretty much it so what am I actually going to subscribe to that's where the types Hal becomes interesting so this is actually what's currently listed for the automotive Hal and I would presume that you know since we're the same ap level in ten that this should not have changed in the entries to the size did change so this is a bit interesting they kept the same version number but they actually seem to have increased the content of its let's just go back and check what they've got here in the house not yet four of that okay that's but that's pretty much sure what you want to read is the house are not yet frozen so I'm presuming effectively there that even though the numbers are the same that would explain why they've they've changed them so okay so if I go here and I look at the content of the the types Hal so you'll see see stuff as you know vehicle areas okay so window mirror seat dole wheel and then you go down you'll see stuff like the actual properties so info ven and Phil make model fuel engine engine oil level temperature sort of I think they have a sort of fuel remaining sort of fuel sorry later on and that sort of stuff so there's quite a few properties here that you can you know start poking or setting or you know monitor for for changes there's just kind of generally speaking here let's see just I'm just being curious mm-hmm so if I grab my shell and let me make it a bit bigger and I didn't the USP car x86 64 I'm just just keeping this same let's go under hardware interfaces car and let's go no automotive sorry and under automotive I went for vehicle so if I go to to oh and I do a let's just see how big this thing is types on hell so we got about three thousand lines here for the types Hal under nine and if I go this the same thing here under our interfaces automotive we are looking for vehicle and let's go checking the size of the types hell here sorry I'm going for - Oh Lord can dish L tips hell so we've got about a few 300 lines more that have been added here and you know I'll you know a lot of things I could possibly just just do for my own entertainment here's do a diff between this with the original and let's go take a look at the one from 10 just for fun so we're looking at Android AOSP it's in OH and an interfaces sorry hardware interfaces automotive vehicle - OH types hell and so we've got a few additional properties that have been added some bunch of comments a few more properties being added oh these are seems to be qualities and pressures okay yeah they change those yeah okay not too big but still so you know the the key thing to remember here from the the readme is they haven't frozen this API yet so in other words they will continue to evolve this and if nothing I mean this goes back to what I was saying it is a work in progress so it is likely that you know this will continue evolving until they are satisfied enough that this is going to be a general-purpose solution for you know most most units out there so that's where the actual hal definition itself for the vehicle and they've got additional interfaces here under say EBS this is the external view system so you got the enumerator and the display actually in the camera and I'll explain to you how this works later on this is the design they currently have which is effectively you've got an application that grabs range from the camera sends them to the display and that essentially with this application ought to be doing you just take the frames and send them in on the aside and get them to the actual display they've got an audio controller here and the other controller actually is layered on top of Android is existing audio control mechanisms so this is not a new audio stack it's just layered on top of the of the existing stack and there is a default implementation for some of the stuff so if I go here under vehicle to not oh default you're gonna find here a default implementation of the vehicle hal properties and and what do i mean by that well if you go here on the device itself so if i do an LS h l and let's go looking for vehicle right so we got a vehicle hal here which is actually being used i believe that we also have the corresponding process because it is running as actually a service sorry so yeah you got an android automotive vehicle service hal which is running over binders so if you're not familiar with that have a look at the treble presentation that I gave a while ago which also is available on YouTube so effectively this is a binder rised haidle implementation which is actually going to implement that API which I which I just mentioned is publicly available so that is for the idols okay now talking about the exterior view system so this is interesting because they tell you quite explicitly they don't want you to depend on the surface planner because the surface flinger is going to come in way too late in the actual boot process if you want to be displaying within two seconds what's going on in your rear-view mirror for example or your camera and then you have got to use something else and that's where they propose this sort of architecture and this is straight from their documentation seems again current as of still today so effectively you've got display driver camera driver and then you've got the hardware side which is effectively you know the camera implementation display Vermont ation and on top of those you've got the different Hal's you've got an enumerator which figures out you know what you've got underneath here in terms of cameras in this place and then you've got the manager whose job is to actually share these cameras among you know applications so effectively one camera could be probably shared by more than one application you'll notice that that manager itself has the same API the same Hal's right on top of it this is just to make it make sure that effectively the applications will see no difference if they're talking straight to the camera or if they're talking to the actual manager but generally speaking the idea is the application grabs frames from the camera sends them to the display and you know I'm not sure this is gonna be the final thing that that is going to to be to be used this this is one area that I you know I'm I'm iffy about the architecture being chosen I'm not exactly sure I like this architecture maybe there's you know plenty of wisdom inside of it that I'm just you know is evading me but I you know there's there's something here that I don't like you know is you know especially say going straight up to the Android side of things I you know I personally I'd be more comfortable doing it a bit more you know level maybe even at you know at an actual driver you know level itself but that's I don't know this is this is one part where you know question mark still TBD from my perspective as to you know what the final solution is going to be on that maybe this is it and we go with that so there is actually a reference implementation of an application and manager so if you go out to your sources under this the packages services directory that I was in earlier so if I gear on a package of services car you've got here the EVs directory which has got manager and application and sample driver so the if you want to take a look at you know how they propose doing it then then you can take a look at that and again so they're saying you know one stream multiple consumers that's the whole idea here I could have one camera and and and won multiple actual consumers of the stream and if you want to look at the actual haidle that they've got for this you go up here under hardware sorry hardware interfaces automotive I go for EVs one oh and then you've got the actual camera what they expect from the camera to get what they expect to get from the camera what they expect to get from the display and so on and so forth okay so their suggestion according to what they have in the documentation is take month the kernel drivers to support EVs you know I'm not the mind of you know why don't just if I'm gonna knock meant the driver why don't just cannot like a pipe the output from one driver to the next driver or something of that kind you know I'm not sure we'll see we'll see what happens with this and then have kind of like a state diagram of how this kind of like this should actually behave so get Kemalists a score you'll have for a notification and start mr. sloops see if the in reverse in that case you know start the image stream and so on and this would be kind of like what happens what happens on the actual application so get the buffer a copy image and then draw overlay graphic if you have to and then send the graphic out to to the target buffer where you can display it and go from there so it's kind of pretty straightforward okay so the properties which I showed you before get sub subscribe and you know if you want to see the list of things in there then you go in that in that haidle interface and they do have a few diagrams there to show you how they expect this to actually flow and this actually seems to originate from 8rx because some of this some of these parts are not there so you know HVAC application effectively wants to power on the HVAC talks to the car service which turns the HVAC power on through the vehicle network service not there okay you can find out HVAC power on so it goes to the VL which goes to the actual HVAC ECU the electron control unit of the ECU of the HVAC sorry and then turns it on and then whatever else has to happen there same thing for setting temperature so set the temperature to this yes the temperature has been set and so on and so forth okay so this is the property you know setting and getting stuff and you know possibly subscribing to notifications from that sort of thing okay so a roid is responsible for some of the audio but not all of the audio so it's responsible for things that you would expect of Android but it's not responsible for anything it's mission-critical so if it's like you know you're you forgot to put your seatbelt on or you're about to hit the car in front of you not in orders business sort of thing so that's there supposed to be some form of Hardware mixer underneath Android that deals with those mission-critical audio cues that might be happening in this diagram they have they're kind of like shows it to so there's expected to be an external audio mixer that gives you all the external audio signals plus what Guu what Android is going to give you and then this goes off to the care a car amplifier or whatever audio output that you might have to go out to the to the driver or you know passengers so you've got obviously the Android side of things so you've got a player going into audio flinger which is going to take multiple streams and then onto mixers and then effectively to logical buses which are now no longer identified as being specifically coming from one audio source or or another so one they have a few sequence diagrams of how audio playback is to be made and I kind of like them because they seem to be kind of at least can I give you mindset of how they're expecting how they expect you to to make this happen on and I think the that architecture makes sense to me can you compare some of the stuff I've seen so for example I want to play something on the radio I press power I say go to this station so the radio application tells the radio service it wants to do this so the effect of the tuner starts tuning into that to the channel then we ask for focus off from the audio manager it is granted then we go to the auto flinger tell the mixer to actually place a sort of splice between those two things here we can't remember the name of the actually the actual specific name patch I think they call it or so that effectively they're going to patch the output from this straight out to the mixer without it going actually through the Android hell so we're not going to take the audio back up to bring it back down we just tell the hardware send your audio out to the to the mixer if I'm doing this playing in the radio and then navigation event happens what am I going to do now so kind of like same numbering as before they're still keeping the same numbers but they're kind of adding more numbers in this case the navigation application is going to request a temporary focus transient focus or grab transients something like by the way the errors here are inverted 7 & 8 somebody who whoever maintains this diagram at Google I think you're 70 are inverted anyway so navigation application is gonna request focus it's gonna get the temporary focus it's going to go out here play something off to the mixer to say okay turn right turn left once it's done that it relinquishes the focus focus goes back to radio and radio continues playing whatever it was doing if we are playing an audio book it's slightly a bit different because now there has to be a way to actually start a phone to start playing the audio book sorry this that I was going to get into the next diagram if I'm straight playing an audio book in this case what's gonna happen is audiobook player is going to request a focus and this is gonna be a permanent focus and effectively we're going to tell the radio hey by the way you're about to lose focus entirely so radio says okay that's it I'm done stop taking my stuff and then effectively the audiobook application gets the focus it starts playing whatever has to go out to the mixer this one is interesting it's a it's kind of like the The Full Monty I guess or whatever it is is like the whole the whole thing kind of brought together I am playing an audio book navigation cue comes in how how am I going to deal with that so if you kind of follow the air the the numbers and they have an explanation there in their text what will happen effectively is that the the app audio played or the audio book application will get notified that it has to pause and then one after it has pas you know it'll pause there the navigation will play and it will stop you know I'll restart playback once the navigation instruction you know is done so when the system is about to say turn right if you were listening to an audio book the audio book will pause there and unpause automatically when the TURN RIGHT audio cue has been given so again I think this are the architecture and half of this audio stuff seems pretty sane and as far as I can see for for now okay power management so in the case of the power management so Android is running on some sort of sock and that board is assumed to be somehow controlled by the vehicle master control what they call the vehicle master controller the MCU or whatever is the core of your car there's actually handling you know some sort of centralized control or the various components so the idea is that the waking up and in the sleeping of the Android sock is going to be controlled by the VM see you and so the in automotive automotive is meant to be kind of like cognizant of this deep sleep state where it effectively suspends Teran android currently doesn't do hibernation so it doesn't actually store something to disk effectively you know being able to completely power itself off and then restore from that personally I think this is one thing that's gonna have to be solved at some point in the future I don't know where when how but somewhere somebody somewhere is gonna have to come up with a good story about Android hibernating because there are various use cases where you actually want and were to just you know okay stop you know suspend itself but not actually reboot for info I don't know who's gonna dress this in how but we're gonna have to figure out a solution for this at some point in the future I think the current mindset is to try to push for the suspend to RAM as far as possible and and try to kind of like fight the the classic mindset in the automotive industry that nothing should be taking power when the you know the bad batteries off but yeah yeah we'll see okay I'd like to see this being resolved for a variety of use cases not just a automotive one so there is a car power manager and there's a car power manager service and you know basic kind of sort of state diagram here the thing is off the application processor comes on there's a shutdown trigger sent by you know presumably the MCU as prepares itself the shutdown and goes into deep sleep and there's toggling between you know display on and display off so this is the interesting part here so the vehicle Hal here allowing us to talk to the VM see you with the car parents from service and then the car powers from service using lip suspend to actually go to the application processor and say okay time for you to go into a deep sleep mode alright and this is kind of like this is kind of like the communication diagram that they have on their website to document the the flow of things and this seems you know saying as I kind of like as a flow of things so initiate the suspend the V Hal goes up to the car manager service tells it that this is going on to kind of parent manager app API transfers this over to the application which would does whatever it has to do says finished and you know will start actually shutting the thing down eventually we say we say wish this but we're ready to suspend and then we suspend the actual device and obviously here presumably all the applications have been told okay you know that's it you you're supposed to you're supposed to be the sleep down waking up a similar sort of thing power up the powering up the device we're waking up from suspend effectively the kernel is kind of waking up as if it got hardware interrupts goes does this thing we can then pull the poke wake reason why did we cat wake up get reason we cruise and so on and eventually possibly return - to suspend if if necessary okay and then they have I just kind of replicated this on my on my in my side set but they have a few diagrams up there in their documentation that tell you the class year or a key of of all this stuff not super kind of like intuitive unless you're actually looking at this code in in a great level of detail which gets me to where leveling so alright so this is this is from their own documentation okay average data into phone per day is about ten gigabytes we expect Android automotive implementations to have more emmc rights than a phone so if there is one part where I think this the Android automotive story cannot hits the brick wall this is pretty much it there has to be some sort of forethought put into how do we deal with the fact that user wants to run you know applications which look like you know applications they have on their phone or maybe even the same app applications and when those applications don't care or are not made to actually take into account that they may be used for years on the storage that they have to operate on so this is this is something that has that doesn't for me at least have a good answer right now and honestly it may not be just on the I mean this is not just googles responsibility at the end of the day as I started off I said you know we're trying to use this operating system which is geared for mobile devices on to cars and and that has some challenges and I think that's that's that's one big one so I'm not sure that the solution is strictly a software solution so one of the things that they they suggest here is you know they they suggest using adoptable storage SD cards so in that case effectively when you're using in asti card you can say this is not able storage so effectively the the storage is encrypted in such a fashion that if you were to remove it it's completely useless but as long as this in the device you can operate on this SD card as if it was some internal data storage for for applications so but regardless I mean even if I go with this let's say I roll with that what does that really mean I mean does that mean that I have to go to my dealership in three years four years five years and have the SD card changed or this is this is an operation that user is supposed to be doing not entirely sure that's that's unclear to me okay so then they have some stats they're saying so for a 16 gigabyte MMC card with three kilobytes of erase cycles lifetime versus rights and they tell you know if you were writing 32 gigabyte today that's just five years of lifetime so what do I do with that you know do do I show up to dealership or service if you look at the dump sis of car service which I did earlier I didn't focus too much on that actually let me do it this way let's do adb shell dumb sis car service best service less and if I scroll down a bit here you'll see that in fact towards the end one of the big things that they've got here is effectively a tracking of the wear the wear of the storage device so let me just page down a few here and you'll see here that it's actually just printing out information about the amount of stuff that's being written onto onto the different of the storage so this is something that again I don't have there's no good story here for for this for now I'm not sure that the problem is strictly androids there is a there's a there's a there's a problem some of you in case commenting here that it it won't meet the temperature requirements I assume this is regarding the the number of Rights being being done on the device there so so again you know the this is something that yes there you go so the commercial SD cards won't actually fit the temperature requirements thank you sir so the the this is something that has to be talked about um will will have the India as an industry will have to figure out an answer for this and again not sure that this is strictly a problem Google has there's certainly something that Android can can help in that regard but there's there's another story to this that is Hardware specific okay boot time optimization so minimizing the boot time of Linux and so this is kind of just one of the things that often times when we talk to two car manufacturers comes up as kind of like a hot topic okay how do we minimize the boot time of our device so that when the person puts the car on they get to androids launch or whatever it is really really quickly so the if you just look at the kernel itself all right this this the trying to optimize the kernels boot on that's a topic that's been studied at length by a lot of people there are a lot of conference presentations that actually talk about that and I strongly encourage you to look around if nothing else the embedded Lotus conference has had several talks that covered that strongly encourage you to eel and to go to a Linux org and see what what kind of documentation there is to to do that so I'm being a bit drastic in my statement here saying that you know a little to no effort from Google to minimize interview times and boot times are you know unfortunately something that is still a sticking point for a lot of people you know if it was possible to hibernate Android for me that topic would actually effectively be taken off the list of things that needs to be worked on if we were able to hibernate then I could boot the device we put it in some sort of you know known to be good state and then every time I kind of power the thing on it comes back from hibernation but I'm not able to do that so if if a full boot is required that's where you know it generally takes a lot of time for Android to to get it to boot there are on that specific topic a few scattered presentations here and there if you bug me a lot by via you know email or social media I should be able to drag out some links about that but generally speaking that is a that is a problem any sorry I'm just grabbing a comment here or a question on the YouTube chat give me one second and in refrain work which is English and it's just a service KB so along any other frame accusing which vendor system service can be started alongside with any other system server which sources me sure yes if you've got a I'm not sure if I fully understand the question of the comment but John's speaking if you have a custom system service you can certainly start it parallel to whatever the framework already starts and I guess one of the things going to that comment is that that's effectively what the car service does it's effectively starting a system service or pack of system services aside from what the system server is is already starting so coming back to that so you know it generally speaking you can see proof of that in the architecture of the external exterior view system or the EBS if it's time critical try to avoid depending on Android I'm trying to make sure that you can get it done either in the kernel or fairly early in the boot process independent of the Android OS framework starting up because if you have to depend on that you have no idea what you're getting yourself into it could take a quote-unquote forever to get to two to that so you know last bullet point effectively kind of reinforcing what I'm saying at some point in the future it'll be interesting you see Android some are somehow addressed suspender flash I don't know how but it would be actually interesting to see that being being done so there are some built-in considerations in the AOSP itself just really quickly here going through if I go under packages services and I go to car you'll see here there's a car product and in here they have a few things like our C files build files boot animation and so on in this case the boot animations zip which were played which is effectively taken by the blue nation application and then is run on on the actual display an overlay of what premium packages has to be overlaid and some sec are specific SE policies that that are required for a car so if you want to look at how does this integration fall into the innovation of the car stuff in the AOSP for the build this is the rectory that you're looking at wouldn't suspend a flash make wearing much worse much much worse or are we talking about a single golden image so it could be a single golden image where effectively you boot into you do a full boot on an OTA and then you actually suspend it to flash with that OTA and then you continue booting with that suspend to flash image so this would be your you know good to good to go golden image and that you could use forever forward until you have to do another OTA so from that perspective it would effectively kind of minimize the issue of of we're leveling for for that sort of use case which is you know the automotive use case so it looks like the camera there is recommendation for not using surface wing to me to signal car does either provide any alternative solution to render frames other than surface flinger no does not so effectively androids display framework is surface flinger if you need to do if you need to do something underneath it then you're talking about effectively the you know talking directly to the drivers or whatever is whatever you want layer on top of the drivers directly and effectively if you look at the EVs documentation google has they recommend that you actually augment the drivers to actually satisfy your two second requirement okay so that is for the build requirements and one last thing just kind of by the way and this is something that we've you know we've seen in discussions with auto manufacturers over the years is initially even the Linux was something that people didn't want to runs like a weirdo normal in this so I can't you know trust this thing and now you're bringing all this Android thing it's like oh my god you know what the heck are we gonna do this so there are you know various hypervisor solutions container solutions that are being looked at used and actually rolled out so it's a completely different topic of its own have a look at that if you're interested in to kind of like jail Android into some sort of controllable unit that you don't want to have directly control your hardware whatever interface that you've got there this may be a viable path for for what you may be I may be trying to do and that's pretty much what I've got in terms of content here I'm happy to to take questions as they come either on the private side or the YouTube chat as some have posted I'll be waiting here for a few minutes to see if that some stuff comes out as I explained that the like go just going back to kinda like the the the kickstart of this of this presentation we are looking at a state of the industry where both the users and the industry itself wants to see these touch based devices in available in cars Android is a very good candidate for doing this because it's been actually successful very much in the consumer side and plus it's open source so you know Royal no royalties involved but there are challenges of taking such an operating system and make use of it in cars if nothing else because it is primarily geared at least in terms of development cycle on to being pushed to devices which are you know quote-unquote disposable you know handsets or whatever and and and so if you want to take that sort of operating system and bring it into a such a demanding environment as the current environment and then you have to you know have have a very soft conversation about certain topics and I think that in some regards Android does have the answers some in some cases as I pointed out as I was going through something still I think are you know TBD and I and and as I said also I don't think that this is the sole responsibility necessarily of Google or the Android development team there are things in there which are kind of like industry problems of that that come with the the you know as a sole responsibility of on to everyone when we start looking at taking an operating system such as Andrew and bring it into into cars some of these solutions will have to come from the industry itself not necessarily just just Google so that being said please feel free to ask any questions and I'm happy to pick them up as they come around is there any limitation terms on pouring Android auto motor frame with to a platform would it be possible to do without BSP or is a too heavy task in terms of code okay if you don't have a BSP you're a bit on your own here it's a significant amount of work to actually port Android to a new platform and usually what you want to do is start from a BSP that's given to you from a vendor for a reference board or reference design and you go from that so not impossible if you really want to do it on your own but it's not it's not and we've helped people do that in the past I mean it's totally doable but it's just not something that you usually want to do you want it you want to start with with a BSP that the vendors given you with functional how I'll hi hi sorry house and hi dolls so that you can then customize it to to your own tasks like knowing the air pressure or level so off of the private chat here I'm being asked does it support other features like and knowing the air pressure or level and so on yes the that's part of the actual vehicle property how those sorts of those sorts of data that you may be able to retrieve from your vehicle are would be in that Hal if you want to take a look at that who starts the car service is it a parallel to system server yes it is parallel to system server it is actually automatically started because the application is marked as persistent so if I go back here under package of services a car and then I go under service if you look at the manifest file here the application is marked as persistent so that will cause effectively this app to be started straightaway at boot up and usually that will be through the tivity manager and then the community manager will keep it alive all the time if I kill it off it will come back so if I go out here and let's let me do that just to show you as a demo if I do a PS and I go looking for again the car if I do a kill on Comm Android car which is this here let's do a kill on 1973 do a PS again so 1973 is gone but you can see here the car services back up as pit 1470 4372 so when you mark up an app it's persistent like that and build it inside the AOSP that's what you get as a behavior I mean just automotive framework in a platform with a standard BSP oh sorry okay I misunderstood your question so if you mean just the android framework or kinda like getting the car stuff to run on some BSP that you have that's not too bad okay that's actually feasible so in this case it's just a matter of creating a new build target in your build system that actually does the car products on the BSP that you've got whatever processor that you're running there and and so on so there's a build of there's a bit of legwork there of just a build system integration and flags and stuff like that that's that's that totally dual all right less chance at questions here so there is a usually between the time I talk and the time that actually goes out to YouTube a 10 30-second delay so I'm just gonna wait a minute here see if we get more will you make yourselves available yes absolutely once this is over either later today or tomorrow I will post the link to the slides as part of the YouTube video description so you'll be able to click there and and and get the slides No all right well this video be available only at a point of time to refer to yes yes it will I mean once livestream is over it effectively becomes available as a replayable video on YouTube so whatever link you use to join the live stream right now you can reuse that to look back at this presentation and in fact for those of you looking at this later on if you spot any errors in the presentation or anything that you like to add please by all means just post in the comments and we'll take it from there all right so thank you all for attending hopefully you know we've answered some questions and you know usually you pick up these topics based on what we see people kind of like asking questions about or what the conversation we have with customers so you know sometimes we see some conversations are recurring and we keep talking about the same thing with customers and usually we will you know I'll put some material together and share with everyone to help people in their work with Android so again think thank you all that for attending and I hope to see you soon in another YouTube livestream thank you folks take care bye

Channel: opersys

Views: 11,387

Keywords: Android automotive, Embedded Android, Android's car services, Android automotive APIs

Id: nNfX4sA2y4o

Length: 80min 15sec (4815 seconds)

Published: Wed Sep 04 2019