Deep Dive into Android IPC/Binder Framework

so welcome to the deep dive into Android IPC and specifically bind their framework my name is Alexander garganta and what I am what I'm talking about is basically a couple of things you know why IPC matters and specifically on Android and how is binder different than other forms of IPC and why is it well-suited for essentially what we need on Android your you know hopefully you're here because you're interested in binder itself but you may also be interested in how Android works and binder is basically one of those core pieces that glues everything together without by they're literally the entire system would fall apart so whether it's because like I said you want to take advantage of it and build your own custom apps and or low-level system services or you just want to kind of get get a sense of how it's all put together this is what we plan on covering my only concern is is this is a little disclaimer right up front is we only have about 50 minutes this talk when I originally put it together I was aiming for two hours so I'll have to condense quite a few things and skip over certain points and or even certain slides I know that's sometimes frustrating to do but it's the only way I'm going to get through it so we're going to talk about things like what binder is you know IPC advantages comparisons of binder to other frameworks binder versus other forms of IPC that you might have used so if you were developing apps on Android so the terminology high-caliber binder just communicates and the cause of discovery works a little bit about a IDL the by essentially the reference mapping in binder which is one of the interesting pieces of it we'll talk a little bit about kind of how you would use binder in a real application I'll talk about asynchronous binder and why it's important memory sharing and concerns about memory by their limitations and a little bit about security so I'm again my name is Alexander garganta I teach Android I'll work for Maracana we focus on Android on open source training and Android is a big part of it I also run the San Francisco Android user group as well as drug user group and involve with a bunch of other things relate to Android enter is basically what I do most of the time these days so what is binder binder is basically an IPC framework I like to call it framework because it's not just the binder as you will see later on we discuss the terminology behind there's a lot of things but you can think of it as a system for basically developing quote/unquote object-oriented OS but the idea is we're not developing the OS itself to be object-oriented which could do but rather we want to enable an operating system to behave as an operon object-oriented operating system so rather than thinking of your OS as a bunch of says you know system calls you think of us as a bunch of services that have states that behavior that you can invoke at any time I'm like microkernel to some degree but basically something that can be bolted on to any operating system like for example Linux so it is essential to Android this diagram you know it's kind of small so let me zoom in a little bit basically gives you an idea that you know pretty much all of the applications all the system services a lot of the be essentially underlying framework infrastructure or middleware as we call it all depend on Android binder especially at the binder and the driver level in terms of binder itself it comes from open binder it was first developed for BOS back in the day it was never really never really lived to see the day as part of us because VOD was acquired by palm it was then ported to the palm kernel for the cobalt and later on the Linux kernel and eventually was basically built into a full-fledged palm OS and served its purpose but that didn't really last too long because as we know Mia palm didn't really go as they hope they did so when when HP acquired I guess later on web was Google acquired the N hack burn and she was one of the key members of that be on the original B team of binder and so along with her came kind of the knowledge of how to do IPC in in essentially embedded environment so what is IPC and why do we care about IPC soon basically IPC is convenien lot of things but ultimately it's used for message passing synchronization sharing memory isolating but certain key components of the system and of course there's also something called RPC so promoting but in Android as you will see when we talk about it PC we do not assume any sort of remoting procedure calls it enables us to separate things it provides privilege separation data isolation information sharing and modularity that's kind of one of the key components of it so when it comes to IPC as you guys know you know being this is a Linux event there's many forms of IPC you know so we could be talking about files we call it talk it could be talked about signals sockets whether we're talking about UNIX or TCP sockets pipe semaphore shared memory you know message passing frameworks like for example queues message bus and then other forms of IPC like intense content providers messenger right and then there's the binder so why binder why specifically why do we need binder why is it so important so again this diagram on the right hand side kind of shows you the typical stack on Android you guys have seen this probably many times at the bottom we have the kernel at the top we have the applications and then in the middle we have this middleware well applications tend to be written in Java and in order to take advantage of duplication framework services which run in separate processes they actually consume them using IP C so this communication from applications down to application framework services is almost 99% done using binder framework go down from the application framework down into the lower layers we sometimes see the use of binder sometimes we use UNIX sockets and sometimes we directly load the drivers to you know essentially J&I and bunch of other things like house but that's beyond beyond this talk so in in androids binder specifically provides us with security so one of the features of it is that it enables us to run things in separate processes and as a result of that we get better security isolation ie sandboxing stability so if things crash because they're running separate process isn't because we have a good IPC framework in place you know they can be restarted without affecting other things in the system memory management so one of the key features of Android Men management is this ability to be able to kill off applications that are no longer needed either by activity manager service doing the job or in case it can't cope with memory demands having the low memory killer K can but for that to work you need to be able to cleanly shut down an entire process you cannot have applications share the underlying process as an infrastructure so you need IPC to enable that process to be able to exist and be able to still coexist you know with other processes in the system and then like I said even even basically androids own components and apps all writing separate processes so specifically what does what does binder essentially do for us it replaces all forms of my PC or I should say system five traditional justified based IPC on Android so there's no supporting Android for second so for sissified semaphores shared segments message queues and whatnot one of the main reasons for that is because androids you know essentially ability to kill out the processes without there having any sort of cleanup action scheduled means that if you were to do use traditional IPC it may not work very well because you may essentially end up with resource leakage and you may have malicious code essentially you know contribute to denial of service attacks binder has a built in IP reference counting mechanism which enables it to count objects if you will that are shared across process boundaries and when they're no longer needed because say the process that were sharing and Di's those objects can be automatically essentially well the users of those objects can be notified and those references can be automatically reclaimed without having any sort of lingering resources be left on the system this is very key to essentially why Android binder survives essentially that's cost the environment with low memory killer where other IPC frameworks don't and one of the other features of binder couldn't compare to other forms of IPC is the so-called thread migration programming model so the idea is with binder is you as the client of the binder framework you basically make an invocation a method invocation and on an object to you appears to be a local but in reality that object lives somewhere else and what will happen is your thread will by default block and on the other side will continue running if you will that method invoked invocation code of course it's no longer your thread it's somebody else's thread but what's interesting is that to you it appears like your thread just jumped on into another process and started executing within the you know those process boundary so that's really key because it enables very easy programming model to exist on Android we also have features like for example identifying senders of a request this is actually very key to Android security being able to know who's talking to whom at any time enables us to enforce permissions dynamically and this is how most Android services do it we also have unique object mapping across boundaries process boundaries so basically if I share a service object a binder object with another process and that process shares that same object with that yet another process that reference that object is unique across all processes on the system and the kernel driver ensures of that and this is actually key to how the discovery of bound services works for Android as we will talk about in a moment we also have ability to send file descriptors across whole process boundaries so for example if you want to play a media file in a media server you're not sending the entire file over you're sending just merely the file descriptor with that file to the other process underneath the hood the file descriptor strip descriptors get duped and basically copied over to the other side Android also comes with something called a IDL which is basically interface definition language that allows services to describe their capabilities to clients and moreover Android comes with a tool called the IDL that can extract proxies and stubs which do automatic marshalling and on marshalling of data as we will talk about in a moment all of that you get for free of course only if you do it in Java but nevertheless the idea is you get you as a consumer of the binding framework have actually very little to do to take advantage of it you have a fairly you know simplified transaction model sorry I should I skip born you know is very simple support for marshalling and marshalling of common data types all the custom types are also supported again this order generation of stubs and proxies is important the recursion across process boundaries is built-ins if a process a cost process being process because back into process a that works out of the box and if your process a happens to be talking to a binder object which is actually also inside of process a so it's not IPC anymore it's local you are essentially skipping the entire overhead of marshalling and marshalling and invoking things locally it is not meant for RPC it is very much oriented towards Claudia essentially client service or message passing it's not necessarily well-suited for streaming and is not part of any sort of standard so binder is not part of POSIX or anything like that so there's you know I mean in session its API and implementation or wiring the same as we will see pretty much everything you do in Android depends on barn they're drawing drawing on the screen depends on I drew my own binder handling touch events dependent binder you know playing stuff you know handling callbacks to your application dependent binder you know you sending intents receiving intents and whatnot then all depends on the binder so it's extremely important now that said a lot of Android low-level system services also take advantage of the UNIX domain sockets as a form of communication so binder is not the only game in town but it's the most common one so a lot of times when I talk about binder people ask well why why do I need to know about binder why don't I just use intents and content providers and certainly you can I'm not going to go into too much depth but the idea is that in Android if you have two applications they want to talk to each other they can literally just have for example their activities open up each other via intents the activity that it's been involved can respond back with some results to the caller via another intent you can also start services with intents you can send intense voice broadcast messages you can receive them and whatnot that also is IPC this down below which I'm going to kind of skip is an example of how you could do that many of you have ever with an Android application so probably done something like this but basically you could have an application create an intent start another application with that intent that intent contains think of it as the parameters of the request the other application can get that intent handle to get the data out of that intent do some work on the behalf of the caller another intent and basically send the result and then the calling application gets the result and processes that it works and it's well suited it's very little session slow coupling because these intents are not are well defined the data parameters are well defined but the problem is that this is all not really very o P it's asynchronous it's not very well-suited for situations where you want you know low latency and at the end of the day all of this still underneath the hood uses binder so yes in tensor a form of communication between applications and there's nothing wrong with that especially if you're just creating simple trivial things but if you want much more essentially you know up to the millisecond notifications between or communication between multiple apps and you don't want to depend on the asynchronous nature of intents well then you may want to go that to that bottom level and take a look at binder another thing that comes up a lot is messenger i pc now messenger i don't know how many of you know is basically a mechanism that allows a remote process to use a messenger to call back to a local handler they're bypassing you messages so basically idea is let's say you had an application is probably best in the studio just demo explai the code briefly let's say you have an application that wants to have some other service they'll do downloads of bunch of images or something what it could do is you create intent to put a bunch of your eyes that it once downloaded and they intend a messenger into that intent and send it over to the service that service on the other hand we can get that intent get the UI's from the from the intent and start acting on those you arise for example say initiated downloads but say wants to notify the original application of when every uri or we of URL or I guess resource gets downloaded what he could do is you can get create a message and then essentially stick that message into the message or send it to via the messenger to there's the other side the other side essentially passed the messenger which was you know link to a local handler that handler will just get a callback onto this handle message he received a message then act on it so that is a form of IPC it's a lot more much more it's much better you know to do it this way than to use frequent essentially intense and sending a bunch of intense because that actually is not even an option so it this particular use case it's it's pretty good but you know the media hood this still uses binder it's still asynchronous and it's still not the most well you know them as well suited for low latency because you still have to go to go through these handlers which are essentially message queues so before we talk about binder how binder works just want to kind of briefly mention a few things about binders as a terminology because binder means a lot of things so first of all there's the binder as a framework which is essentially the overall IP IP CRE architecture it's not just the driver there's live binder there's these proxies and stops and whatnot a lot of these things essentially come into play when we when we talk about binder so at the very low level like at the very bottom right here we have the binder driver so that is what facilitates the actual communication or exchange of messages across process boundaries yes I know this Tillich's conference this is what we're mostly interested in but then again this alone would be useless had it not been for the rest of the front in the infrastructure the protocol so the protocol is just how we communicate with a binder drain driver and most of the time we're essentially sending it I autos the eye binder interface is essentially a simple interface defines the the basic mechanic mechanisms of every binder object binder objects are those things that we actually want to use those are the things that are represent the services we want to interact with so as you will see later on these I bind their objects have capabilities like for example telling us whether they're alive or we can go and subscribe to be notified when they die so this is expressed using the default eye binder interface a IDL is I mentioned a language that enables us to define business operations that go on top of the eye binder interface and essentially that's what you know is what our clients are after without the you know the business operations was a point of using generic eye binder objects the actual binder object that is a basic implementation of that eye binder interface it provides a very basic infrastructure and that's what we end up extending from when it comes time to build building our own actual binder services binder tokens are basically think of them as candlestick or think of them as pointers but had to happen to be agnostic to the process therein so essentially a binder object can be referenced via a token or a handle that is unique across an entire operating system so essentially it's a reference that that ends up being unique and can be passed around by the service that's what we are actually consuming that's what we're interacting with so for example location you know the service is is a binder service or activity manager service or power manager service vibrator service you know surface flinger sensor service installments on these are all services they're essentially implement binder binder client that's what we use to essentially interact with the service the binder transaction that's essentially exchange of if you will data between the client and service through the binder driver and the transactions are as you will see fairly simple they involve just copy copy data back and forth a lot of a lot of what happens then is handled by the framework the binder parcel is basically a unit of data that essentially gets sent across process boundaries so when we send a message to the remote service we're actually sending it in form of a parcel so marshaling is a mechanism of converting our rich data types ie a request that contains for example objects or data types that mean have meaning to us into a parcel our marshaling is the reverse of that taking a parcel that was received through a binder driver essentially an unmarked and converting it back into rich Joe and say job or C++ or even see data types into something that we can work with proxies and stubs are the things that get auto-generated and essentially the what are the things that enable automatic marshalling and unn marshalling if you are if you happen to be coding things in java in which is they are truly auto-generated if you happen to be coding them in C++ then you have to go and write your own processing stops and finally the context manager that is the thing that enables discovery of binder services because you know having this remote process somewhere with these binder services available he's useless unless we get a reference to them to begin with and in Android that context manager is called service manager and that's a daemon that most of you probably seen if you've done PS so let's take a look at kind of how this communication works so at the end what we ultimately want as the users of binder is to have a client talking to service they just happen to be in different processes that's kind of what we are asking but in reality we cannot just go into another process and essentially invoke operations another process because that would violate the process isolation so we have to go to something that has ability to interact with what processes and that's the kernel so that's where the binder driver comes into play so how it typically works is that a service that wants to basically be on consume by clients will come to the driver and register by essentially spawning off threads they will block on the binder driver in these blocking I aku calls waiting for callbacks to do work so that happens preemptively so when a client wants to actually have service do some work on its behalf it sends a message to the binder driver the driver then finds the corrupt proprietary service to service that message the cert one of those service threads up blocks handles the requests and essentially sends the response back to the binder driver via another one of those threads that again ends up blocking once the response is sent that client at this point gets the message what's interesting is that to client this feels like an ACS or a very synchronous communication channel and the services you will see even though as these threads doesn't have to worry about them all of that low-level thread management or thread pooling is automatically handled by the same framework I'm not going to say the driver so how does the how how does this actually happen is that most of the time what we see happening is these local calls for readwrite calls we're basically a service will go to the by the driver and say hey I have some data for you or a certain size and then I have want to get some data back of sort up to certain size or I should say you know it'll basically get initially to this structure into which later on the data will be populated so basically what happens in the service god comes to the barn behind the dryer for the first time the right buffer is empty just going to sit there the and then the service then blocks when the client comes later on and sends them requested the service the binder driver takes the request copies it into basically the buffer that the service is going to then read from and then tells how much data is in the buffer and then at that point unblocks the service now now the service kinds of data when it produces a result it goes and basically takes the result writes it into again one of these buffers into this right buffer and basically gives gold goes and gives it back to the de Botton the binder driver and so essentially this happens on both the service side and the client side we're just exchanging these buffers back and forth what's in the buffers is something we'll talk about in a moment but basically it's the it are the parcels plus the handles to who we want to talk to now and there's in these buffers I guess I should also say there's these bookkeeping commands that basically tells us like for example is this service still alive is this you know if you wanted to ping something and whatnot we'll talk again about this in a little bit I have to speed up because we're going to run out of time um however one thing I wanted to mention is that most of the time all of this communication that I just mentioned is iocked ails it's very low-level we actually don't want to know anything about it as the consumers of this framework we just want to basically send messages back before I think right and act on messages so that's where the proxy stops coming to play so the proxies job is to basically take this very high-level say Java or C++ request you know I receive it and then convert it into one of those parcels and then submit an I auto transaction to the binder driver and block on the other hand the stubs job is to basically listen if you will to the binder driver callback and then upon receiving a callback D Marshall or Marshall that parcel into something that the service can understand and then call the appropriate callback method in the service the service is coded so that it only knows about these high-level method calls for example you know to do something that involves rich Java objects it doesn't know anything about the infrastructure except for the fact that the service has to extend from the binder object and implement this binder interface the rest of the implementation of the service is in no way you know tie to find their framework it doesn't care about it the only other thing this semantical thing that affected is threading because every time his service gets called it the invocation of the service method can happen in any one of these threads that was basically preemptively you know over here tie to the binder driver so except for the multi-threaded behavior and the fact that you have to extend for a particular class the rest of the infrastructure is completely seamless now one more thing I want to mention before we actually two more things before we move off of this slide one is um two clients I you guys you guys you know let's say you guys as application developers not as system integrators um even this is way too much knowledge for you you guys as the bail application developers don't even want to know anything about buying they're about proxies about stubs you just want to basically consume some service so what Android does for example for system services let's say locations manager service activity manager service power manager service and you name it there's like 60 of them plus what they do is they provide these so called managers so these things a manager's job is basically to be a proxy to proxy if you will so manager facilitates the discovery of the service you want to talk to and he hides the binder interaction away from you so for example when you're talking to a location manager you are actually talking to a local Java object which underneath the hood if you were to open up and take a look at it will convert your method calls into remote binder method calls via proxy and the reason why if the manager is there is to handle certain things like you know exceptions like for example let's say you get a remoting exception of the remote thing dies as well as the handle lookup mechanisms as well as the handle some threading issues for example if you're getting asynchronous callbacks the manager's job is to take those asynchronous callbacks and convert them into something you can handle the UI thread otherwise you you have these thread you know thread issues potentially so for those of you who have gone into the Velo application development you've been using binder all along you just have a necessary notice then because you most likely just been using the managers the last part is the discovery how do you actually find the thing you want to talk to you to begin with well when it comes to androids built-in services basically what we need is this thing called context manager so if any of you have done you know PS on an Android you know adb shell you would have seen something called service manager that is the context manager another name point so what the context manager does very early it's actually one of those Damon's that gets launched very early by in it it goes to the binder driver and says Kay bind the driver I want to be your context manager and thereby the driver only allows one context manager at a time to be registered with then what it does it submit essentially creates a bunch of threads and waits on the binder driver it itself is a service a context manager aka service manager is a service a binder service so what is it job its job is basically to sit there and allow other services to register themselves with a context manager so for example when you have a location service location service will rep be registered with the context manager as the location code quality code location service so when a client needs to find location service all you need to do is as the context manager so how does that work well basically the service well at some point will register with the binder driver the service will then use a service manager or context manager proxy because now the service is the client to the context manager then the service this props is going to go to buying the driver figure out where the context manager is I get a handle to the context manager and then submit a request the context manager say hey I want to register myself with you on the other side when the client wants to look up a service to talk to the client submits a request again via another one of these service manager proxies submits a request and right of the request basically gets a reference to the service manager once in has a service manager it asks where's the location service for example and it gets a reference back to it and now the client can go and use it the way we talked about before so if you were to do a DB shell service list you will see the list of bunch of these services so this is essentially for example the name of a service so all these services are registered by its simple string-based names and then this is the you know a IDL that it implements basically or the name of the IDL that implements say for example for location but there's you know 70-plus services you will find here this diagram kind of shows you another another way of picturing all of this communication let me see if I can quickly take you through it it's somewhat redundant but I don't know if it's probably so much easier to grasp so let's say for example you had a service that you want it to expose to clients how will that work and let's picture that service being part of the service manager right which is another daemon that we know exists in Android so the very beginning this is kind of started with a negative you know this service is going to run from a main and it will basically start a pool of threads that will then go to binder so this is step -3 and essentially block they will just sit there block waiting for requests so they will issue those a blocking I octal calls at that point the service will have a sip resume ibly another thread and that thread will then do a lookup of the service manager which we talked about and it will then register the service with the service manager or context manager again the names can be used interchangeably so now the client runs the client wants to use the service right how the client goes to the service manager I didn't put in this diagram just because it's again it would involve a lot more errors but basically the client ultimately over here wants to invoke an operation of the service that's what he wants to do invoke some service for some method foo and for you know pass some data let's call it some bar I don't know what that is um so this client will base invoke what it what it thinks you know is a a service reference and evoke a method foo for some bar but what a client doesn't know or care this is actually what's important is that it's actually talking to a proxy not a remote object like client is actually has a reference to local object that proxy will convert their request from you know some food that you know essentially method call foo to a transaction called foo whoops and into that transaction it will say while longer the transaction will basically invoke the transaction with these parcels these parcels will be auto created by the proxy and into the parcels will take this sub bar so we'll take a Sun bar and we basically shove it inside of this data parcel and then will invoke sometimes action that transaction goes violet binder through a blocking I auto call to the driver over here to the binder driver the binder driver now realizes that this transaction is referencing essentially this service because the service previously registered itself with it and basically it wakes up one of those threads that was previously blocked on the binder from the server side that thread takes the parcel that would that was basically received now to the binder driver on marshals it I should say gives it to a stub the stub on Marshalls it figures out that the client wants to invoke a transaction called you know foo and it actually invokes a method on the service called foo with that same bar so the this bar you know I call it some bar basically get recreated all on in another process it went through a marshaling stage then into essentially copy gets copied across the process boundaries Y divide the binder driver and there the other side gets recreated into its original state again the marshaling is built in for a lot of the common data types but you can create and roll out your own marshaling and marshaling strategies finally the service now invokes the operation the client requested which was the say the foo operation and produces some result the client or so I should say the service returns that result the result now goes back to the stub and wireless table gets remar sholde into now the reply parcel so it gets converted from some result which can be say some java data type into essentially a series of bytes and then violet binder submits get submitted back to the binder driver right this is again one other another one of those blocking calls on the other side we are blonde because the client had been waiting all this time the proxy receives the summary the reply parcel the proxy the proxy unmarshal the reply object from the reply parcel and is back there reply object or some result back to the client so the same some result that got created here and return is the very same object but now reconstructed in another process okay that's roughly how things work now this is kind of a diagram I showed a couple of years ago actually on when I talked about services in Android this is just it gives you an idea of what let's say a location stack on android looks like I'm not going to go over to the entire detail but just want to point out let's say you have wanted to consume a location service right what you would do is you would as a system for a system service called location what you would get is actually a location manager so what the location manager would implicitly do is do a call to the service manager which is down here to basically figure out where is the location service where is the location service location service was pre created earlier on inside of the system server and got registered with the service manager via this term called location so here when the client actually says I want to say get last-known coordinates what it's doing is it's talking to this but in reality this is first looking up the service and it is then going through a proxy via the binder driver into this stub and by the stub basically into a service and then from the service it goes and talks to these location providers which themselves or services and they talk to house and drivers and whatnot that's beside the point but basically the binder is key to this jumping from an application space to essentially this middleware space which itself is just another process again I wish I had more time to go over to more details but you know I have to speed up so what is a IDL so a IDL basically is the language for describing the services to their clients it looks and feels like Java it's not Java it's similar though this is an example of an a IDL basically you know service call it some food service generally we would call this save this file into a dot a IDL file with a name that matches the name of the service you will notice that this feels like namespace because it is it's same like Java notice we have this use of import and now our cereal we define our interface in this interface we just define the business methods that the service is supposed to implement and not supposed to have to implement and the decline has to you know can consume these methods are just prototypes they look and feel like Java so you're define the return type they need to find the name method name and you define one or more or zero or more parameters what's different is that unlike Java you can also specify this essentially flag the determines which way is the data copied for example if you say in let's say you're deleting some something of type bar that means that the data is copied from the client to the service but it's a service work to make some changes to bar let's say bar was immutable those changes will not be visible on the client side over here let's say you had this object bar do you want it to have the service save and maybe the service updates the idea of bar when it gets saved whatever bar may be that basically you know now you know excuse me means the data gets copied both ways so on what happens when you create a file like this Eclipse if you're doing this in Eclipse or if using the Android build system will automatically employ this tool called a IDL which is part of the SDK which will generate in the gen folder this essentially I foo service which is the same as the name of the service that of the file we created which implements essentially this interface or extends its I interface and inside of it defines this stub and then the proxy the stub basically has this on transact method that is how the service is going to receive transactions and the proxy on the other hand has the actual methods the clients are going to use and notice that what those methods do is they create these parcels convert somehow you know are data types for example this bar into a parcel so somehow they ask the bar to write itself into a parcel and then the stock or sorry the proxy submits three transactions you know that yes there's more than you know we can go much deeper than this but that's basically underneath the hood what's happening and then here is a definition of all the methods now written in Java if you were to do this in C++ you that you will end up having to basically write a lot of this code yourself so I'm not going to go into telepods that will be done right now so I deal out of the box or binder out of the box framework supports all of these data types especially in the Java land so you know all the primitives including plane barrier to raise character sequences and strings of course file descriptors which I said get copied as a file descriptor serializable objects which could get converted to bytes but Java civilization is not the most optimum optimal thing in the world and should be avoided maps and lists so basically if you have a bottom you know objects that are structured into maps or lists that will be automatically and Marshall than are marshaled bundles which are just specialized versions of maps object arrays sparse arrays and you know sparse boolean races are just more specialized and then I binder so these are the interesting ones so anything you pass to the other side as I binder gets passed as a reference and not as a copy so basically everything else gets copied including the file descriptor gets copied but I binder essentially gets passed as a reference which is how the client clicking for example submit a listener to the service so the service can notify the client of changes of some state via callbacks so in that case the roles change the client when it sends something that's of type I binder it's essentially becoming a service so the client becomes a service in the service that invokes that callback functionality becomes the client to the original file so however what if you have your own data type something that you know you could not basically you know doesn't match one of these so what you would do is you could then create your own custom data to class you would just in that class may have you know internally data structures of whatever so you can have whatever you want in there but the key to making it work with binder is to implement what's known as parcel level parcel was an interface that requires you to basically create a strategy for writing a fixed strategy for writing basically your object into a parcel right and that means both down to the parameters that are supported as well as a mechanism for converting a parcel back into that object so for example if you're selling bar to the service the proxy will call this method to convert your bar into a parcel and when the service receives this parcel you like the stub will call this method to convert the partial back into the original bar so as long as the the two mechanisms match that will work out of the box I'm not going to go into more details than that you can basically you know primary you know primitive values don't require for example directional flag and whatnot comments get copied and so on it's on that's beside the point you can read it on your own when we just mentioned this briefly binder basically is you know support these references like I said it can cross object boundaries and somehow remain unique this is key because for example if a service were to register itself with a service manager let's say a location service registers itself with local service manager well and now the client asks for that handle well in that handle needs to work in the client just like it did in the service manager and so these candles need to be able to be shared so what happens is that when you're creating these binary objects the kernel that binder does or anything the per kernel binder driver doesn't know anything about them because you know they're just local binder object but whenever you send a binder object across essentially the process boundary I through the bird kernel driver the kernel driver notices that that's a binary object and we essentially creates an internal mapping that remembers that that object points to your process and it then creates a handle that it sends to the other side if the other side ever references that handle actually the other side now gets essentially that super handle but the other side you need a local binder object represents the remote object so there's a local reference to what it's essentially you know your object so that the local reference essentially gets to some arbitrary you know not arbitrary but some sort of a pointer if you will in your own local local memory space but whenever you then later on invoke an operation where you're writing that memory address to the kernel driver you are referencing that essentially the kernel rewrites that local pointer back to that kind of generic pointer if you will or the handle and then maps it to what actual process is supposed to have that so there's basically state that is inside of the kernel driver that does the mapping of local pointers to these global if you will pointers and along with that it does reference counting so that it knows how many times a particular object is being used because he knows who knows who you know you gave it to so this is key because this way we can discover when something for example dies or if something is no longer needed the kernel driver can automatically tell the service that is no longer needed so for example service can take it out of service so now they are going to more details in that again we're just running out of time so let me get just briefly mentioned that there is an example here that I invite you to take a look at where basically I have a law application called Kiba dachi that has a simple UI that looks like this you punch in some number you select which Fibonacci algorithm you want to use whether it's going to be implemented in Java or C and whether it's going to be implement recursively or or iteratively and you click on this button and that button basically is supposed to return the result of the picu Bonacci calculation what's interesting is that the actual implementation of this Fibonacci algorithms it happens inside of this service where the UI lives inside of the client right so let me just briefly mention kind of how things work so this is again coming to the application space so if you're writing applications you'll really quickly realize that the services in clients that use binder directly are going to have to depend on a common interface so when you're sitting there forced to deal with shared code one of the ways to be able to do that in Android what you know essentially so you're not duplicating code is to create a library project if you don't know what that means you look it up but basically what I do here is I unfortunately don't have the time to go into that essentially it's a project that itself is deployable but it can be referenced by other projects of any art artifacts of a library projects getting corporate into your project at the build time okay so for example let's say I wanted to define a common interface and common data types right so these are the types that are referenced by the client and service and what I would do is I would create a common project I would make it a library project and inside of the project I will then define my a IDL interface and I would call create a file called I people have to service their a IDL is inside of it I will define the business operations and then basically you know that will be the first step unfortunately this uses custom data types so this will actually not generate anything yet until I do something else so I need to find this custom data types so discuss the data types in this case I have a record or class called Fibonacci request whose job is to basically encode the N which is what you want to do if you would actually on as well as the type of the algorithm you want to use it and do it on and in this case that's just a simple email well because this is a custom data type I can't just put it in a parcel because parcel isn't or anything about it would not request so in order for this to play well with a framework what I need to do is implement parcel Lobo so how do I do that I just add the implements possible which forces me to basically write this method which given some parcel I can convert my object into that partial which essentially means I'm just boiling it down to experiments notice that in ohms all I need to do is get its orbital value and basically I write it as a simple int on the flip side I basically need to recreate this object from a parcel in order to basically deploy it again so I'm not going to going to there's basically the Fibonacci response which is kind of similar I'm not going to go over it it's not important it's basically just has you know two Long's nothing but saying the same idea so I mean interest in time I'm just going to essentially move forward so basically when this is got it when you now calculated this this ail file you now want to create a service so how does that work well if this the ideal file was well because you know written the tool a IDL will automatically run by be run by eclipse and generate this stubborn proxy so what you do is you now create a new let's say a service project that service project extends the stub by extending a stub you're basically essentially you are in a binary object and that forces you to automatically implement all of those interface methods that does the stop essentially define so these are the methods that I had like for example you know feed bed you know just a bunch of food methods these are the business methods is not important so the implementation of the DS as you can see it's rather simple you're just we're in this case as long as you don't care about this part this part up here the rest of this implementation is pure Java if that's one of the key benefits the binder you don't have some sort of a loop you're not pulling on some file descriptor you just write in callback methods and what will happen is when a client submits a method and all a thread will be automatically picked from this thread pool and one of those threads will invoke one of these callback methods you just have to essentially do the work and so it makes your service extremely easy to write so of course you have to worry about protecting some shared state if there is any because there could be multiple concurrent requests happening at the same time so in terms of the clients I should at this part I'm going to skip this has to do with high exposed service to the clients it really depends on whether you're writing out an application or whether you're writing essentially a you know system service with a system service all you need to do is construct this object that we previously created this thing and you just go and say service manager would you be so kind to remember this but service manager will not do that for user services become for security reasons and so you end up having to essentially find another way of exposing it which this talks about and I get them we're not going to run out of time so we're going to go into details of it I just want to mention for the client part the client basically hi uses the service so this just talks about the client this is the whole UI you can literally go through all of this by the way all this code is on UI owned on github but the client basically just has a reference to the service by an interface but in order for the client to connect to it he needs to get somehow the reference to the remote service this is how he gets it as an eye binder object but the client don't can't use the eye binder object because it's way too generic so what the client needs is to convert that object into a proxy and this is the one line it basically does it for you this is auto-generated code behind the scenes this will generate a proxy and give it to the client and then when the client when it comes time for the client to use it this is basically all it takes and the client says service dot fib that's it it's a submit a request and it gets a response it doesn't know anything about binder at that point the only thing that makes this binder specific is the fact that this can fail with a remote exception that's basically if the service gets she gets killed in the minimum in the mean time so I'm not going to go into details of that I also want to invite you to take a look at the asynchronous binder which in this case basically what's different is that by adding this one-way keyword for example onto your ideal interface what you're saying to the binder framework is that the client should not block for the service to do the work so the client submits a story request and goes back to what it was doing so now how does the client get back the response in this case the client has a listener there it's sent to the service what happens is this listener gets passed as a reference and the client gets get so basically the data back that way so there's some threading issues that you have to worry about on the client side when you're using listeners because the callbacks happen on binder threads and if you were to try to update the UI from a binder thread you would basically get one of those exceptions so there's you know then you have to use handlers and whatnot that's where Co example messenger makes this easy for you so I'm not going to bring to the details of it because again we're out of time I want to mention for sharing memory binder has a limit as to how much data you can pass back and forth and that limit is one Meg if you want to share memory one way to do it is to use a shman so what you do is you basically create a national region of certain size you put your data into that region and then you send the file descriptor to that region to the other side if they want to get access to it again I'm not going to go into details you can read it on your own and other limitations binder has a limit up to 15 concurrent threads on their own in each process so that means that you're basically can up to 15 things happening to the binder framework or it for a given process so for example if the service happens to be doing some blocking work right so it's better not to do it in a binder thread but rather support its own thread to do that blocking work and allow the binder try to go back to the pool and service some other requests especially if that was written to be asynchronously we say synchronous to begin with in terms of security I'm just going to mention that there can only be one service manager at any given time for secure and that one service manager by default does not allow untrusted services to bind with it which is why when you're writing application level services you end up having to go to this kind of roundabout way of binding it there just talks about how this is done and finally I want to mention that one of the key features of binder is that it sends the information about the client to the service from every request which is then used by services to figure out who the clients are and then based on that information figure out whether those clients should be authorized to make exit or essentially invoke those operations so almost all of the applicable services in Android are enforced this way although like for example let's say vibrate let's say you want to vibrate a device well before the vibrate that happens the very first thing that happens is it checks for whether the caller has the permission to vibrate the device and the permission isn't granted you get an exception or for example if you're talking to location service if you don't have the location to access find location so if you don't have permission to access find location you get an exception how does this work well under the hood we just ask binder who are you you're talking to us once we know who you are by your UID we can go to figure out which application you are once we know which application you are we can ask the package manager which permissions have you have been granted to you so it's a it's a bits of mechanism that basically gets used for almost all permission enforcement on Android the rest of the actually only two more slides I think we're done one other feature by there is this death notification which is very important so for example let's say you go and ask location service to get you updates every time location changes and then you die because you get a low memory killed or yes'm something happens in your process and you forget to unregister well guess what we don't want to cap the GPS be you know pulling and sending these updates to nobody right or if you ask for a wakelock and you never return it and you die we don't want this wakelock to keep the device awake so what happens is these services basically track the clients basically through binder and they get notified when the clients die and they automatically do cleaner on the client side when that is detected there's this so-called you'll seal you can link to death of any binary object and then once you basically get notified as a debt recipient you can for example you know remove you know that object from for example you know being updated or something else like you know you can shut down whatever you were doing on the on behalf of that object it's actually very key to how Android manages resources had it not been for this we'd be really hard for Edgar to basically avoid these sort of you know especially bad applications you know creating denial of service upper antics and then finally reporting binder has a cold you know until the proc file system or to the debug file system basically reports on these reductions which I wish I had time to show you but you can kind of try it on your on your own and see what happens there's a whole bunch of other resources and want you to check out you know a lot of you know good slides and and essentially discussion of Android or of binder and open binder that's it's worth looking at and so again in summary we took a look at what binder is quite compares to others briefly how it works you know I briefly talked about how you could implement it I invite you to take a look at the code we talked about features like a IDL security death notification and so on and so on this record this talk will be recorded or actually is being recorded I'll save it and later and upload it to the same URL I'd like you to check it out and hopefully you guys got something out of it so thank you I'll be here if you have any questions

Channel: InfoQ

Views: 45,428

Rating: 4.9463601 out of 5

Keywords: android, ipc, binder, marakana, techtv, gargenta, android builders summit, android internals

Id: Jgampt1DOak

Length: 54min 57sec (3297 seconds)

Published: Thu Feb 21 2013