OS hacking: Better encapsulation of IPC service setup

well hello friends welcome back to the program today we're going to do some refactoring work and it's going to be about ipc services so let me show you a typical ipc service um the way we set it up so let's look at config server which is the configuration settings provider for applications and what we do in a typical ipc service is we create a core local server object which is an abstraction over a unix local domain socket server and then we do this thing called takeover from system server which means that we take the socket file descriptor that was actually created by the system server program uh and then passed to us and then we just kind of inherit that in this program um and then we set up an accept handler so basically the local server object will detect when there's an incoming connection it will automatically accept that connection and then it will invoke this on accept hook handler right here um and this is kind of what most of our ipcc services do in their main dot cpp uh and then if you look inside of the accept handler what we do is we just increment a client id so that every new client gets his own id and then we call new client connection passing in the specific type of client connection that we want to use in this program so what i'm thinking here is that it's it's kind of silly that we do this in every every ipc service main file so if you look at like launch server main you will see some stuff specific to launch server but then you have the same pattern here so you're creating a local server object taking over from system server and then setting up an accept handler that increments a client id and calls new client connection and so on and so forth if you look at other services so what i would like to do today is come up with some kind of abstraction where we can sort of encapsulate this whole thing into a simple pattern so that we don't have to do this everywhere but we can instead just say um well what would we say i guess instead of creating a server we would do something like i mean maybe we still want to call it server but so we'll have some kind of convenience class and live by pc and um and oh and something i didn't touch on is the fact that we have two types of ipc services we have multiple client servers and we have single client servers and it's basically what it sounds like multiple client servers can handle multiple clients connecting to them but single client servers only ever deal with a single client and so single client servers are launched on demand when somebody requests a connection then we spawn a new process and let that process talk to that client only so there's no multiplexing happening in those single client servers but this here is an example of a multi-client server so that's why we have an accept handler and um web content i guess would be a good example of a single client server so in web content um there's no accept handler but instead we just um we say take over accepted socket from system server so what that means is that system server has already got a ready-made accepted socket for us and if you look at the way these services are actually run by system server um you'll see how that works so um the web content process sort of set we set up this socket here system server sets up the socket with these permissions uh and then we say multi-instance true and that means that um whenever somebody connects to the socket we will spawn a new instance of the web content program and then allow it to take over this newly accepted socket and that happens here so that's sort of how a single client server is launched and i think we can abstract this as well although it's a little bit of a smaller savings because single client servers don't do as much setup [Music] but let's start with the multi-client server so i think we will call it um multi-client server sounds like it's a bit of a mouthful so let's just call it multi-server and um we're gonna need the type of client connection because that's unique to the program so we will say that we will serve this type of client connection and maybe then we just create one of these things um and we should also make this fallible in case we don't have memory for this business um but i think something like that okay and then how would that actually work let's just sketch it out here in the main cpp file we don't have to edit headers just yet this is just a tiny trick that i like to do in larger c plus bus projects like if you want to prototype some thing in a different name space that would normally go in a header you can save yourself a lot of rebuilds by by just prototyping it in the cpp file uh it's it's a simple trick but it has saved me so many hours of waiting for compiles so what does this thing actually do so we have a type name a client connection type i guess uh and then we will need a um static error or um error or nano own putter let's say to multi server create okay that seems kind of reasonable and then what would it do so it just has to do everything that we're doing here what if we just joint that in there um so first thing we need is a server and since we don't have a stack here we'll have to put it in a member variable reference core local server so we'll say oh shoot wait wait um we don't have a an object yet this is a static factory function so first thing we do is we just create this server on the stack so core local server try create and of course this can fail so we have to wrap it and try also let's just call this try for symmetry for now and then after we set up the server we want to take over from system server so that'll be our next step so we're going to try to do that and then if that works out all we need to do then is set up the accept handler and hmm so then we call new client connection with client connection type all right and i think then we don't need this to be a static end here but rather we can turn this into a member so um we'll do something like this and and and and at this point let's actually create the object so a new no throw multi server um and then we'll pass in the server we just created yes okay and i guess we can set up this lambda in the constructor there we go yeah so i'm still i'm still getting used to kind of these um this new way of user space programming that we're learning in serenity os where we actually try to handle errors so um now that we're writing something out here from scratch let's take care to actually use try and make sure that we propagate any errors that happen immediately when they happen so this looks pretty okay let's just use auto here just to be good boys um yeah this feels like it could work so at that point would this just work let's find out so build that's a good sign and let's see if launch server actually works so we can test that by i guess like double clicking on something in the terminal for example let's just double click on this readme and it opens the text editor so launch server appears to have launched that is fantastic okay so all right let's say that that's a good start so let's move it to a separate header then so lib ipc multiserver.h okay um wait how do i get there okay and then we need some headers so we need error because we're using errors we need libcore local server and of course this whole thing here has to be uh has to be in line we can't move it out to a cvp file because it is templated right so um we end up putting it in a header file i guess we could we could rearrange things a little bit and move some of it out of line but i don't think that's such a big deal in this case it's it's not a lot of it's not a lot of code that we're talking about if this were larger if it were to grow larger we can think about reorganizing it so that some of it would go in the cpp file instead but at this stage it's like just a handful of lines so okay and then that c line is nice enough to tell us that local server.h is no longer used here so that's a good thing um and what else do we need here i guess we need ipc connection as well client connection dot h i think we need that one that's where new client connection is right no just connection.h does that still build it does okay cool so let's try to apply the same pattern to other servers lib ipc multi-server alright and then let's create the multi server so try ipc multi-server in this case it's a config server client connection um try create and then we can just that's pretty rad hmm i'm digging this so let's see let's nuke that thing and where else do we have these take over from system server audio server let's just do all of them see what we can do here so instead of config server it's audio server and oh wait wait back up this guy was passing in something else to the client connection object so here it's not as easy because we are passing additional state to client connection okay so let's leave this one for later let's do the generalize the simpler ones first and then we'll come back to auto audio server okay so forget about that one for now let's do clipboard clipboard is nice and easy so again we just need live ipc multi server multi server clipboard client connection and try create cool okay and uh oh lookup server interesting that one it looks a little bit different because we're not in main we're actually in the lookup server object constructor so this program is a bit old it's not really written in the way i would write a a new server so i should probably refactor that at some point and um because like it it's sort of an anti-pattern in serenity to do fallible things in constructors because we have no way to propagate the errors so what we want to do is we want to move all the fallible things outside of a constructor into either the function that calls the constructor or into a factory function but let's just ignore that for now and see what we can do here so does anybody else use this m local server for something no they do not so that must mean that we can actually change the type of that to um to multi server and we will say that we do need to know the type of the client connection type here so oh wait what was the client connection type just client connection all right yeah lookup server feels a little bit like outdated in the way that it's written we have to give it a bit of a tlc at some point so what we'll do then is ipc multi server lookup server client connection is that not it is this thing not namespace it is namespace lookup server client connection and why don't you like that um oh i see so i was using the uh lookup i thought i was using lookup server namespace here but i'm actually in the lookup server class within the lookup server namespace and then lookup server here refers to the class not the namespace that we're in so this is an unfortunate state of things where we have a class with the same name as its containing namespace but of course the work around here is just to omit the namespace since we are in that namespace anyway still a little bit of an awkward contortion um so let's just call it m server is um well here we have to use must so in situations where we cannot use try because we have no way to propagate errors we do have a macro called must which is um a try it's like try but it just uh crashes the program if there's an error so it's sort of a stop gap at this point um that we'll use as you can see it was used here already to to do the socket takeover um multi-server client connection try create yeah that's still that's a that's a pretty nice change like all that stuff down to just a single line okay and then um take over from systems or let's just find the other ones uh so we have notification server let's do that one this is pretty mechanical but very enjoyable nonetheless i really like this type of um when you when you come up with a pattern to encapsulate something that you do repetitively in many places it's just kind of satisfying to just sit down and actually come up with that pattern and then apply it everywhere it never gets old not to me at least so here we'll just try to create a multi server for notification server client connection and we swallow all those lines so nice that is just nice i actually i wonder about um [Music] i wonder about this pledge here we're pledging unix thinking like why do we need to pledge unix but it's probably yeah it's because we have to connect to the windows server because a notification server actually connects to windows server in order to display notification windows and in order to establish that connection to windows server we do need the unix pledge promise but we do drop it here after we have everything set up so before we enter the main loop we do drop we do drop unix which means that we cannot create any new connections any new outgoing local socket connections which is pretty cool okay and who else sql server oh yes sql server i don't think i've ever done i haven't done any work on sql server it's just um all these other folks working on it but we do have a growing sql database server in serenity which is a pretty cool sub project um let's see multi-server yeah it's just a simple multi-server here as well so good man these are just so satisfying okay hmm and then we have some other ones so we have audio server that one was a little bit weird um inspector it takes a path here so that's new hmm take over from system server and it takes a path what do we do with that path socket path okay so socket path is non-null um oh i guess if we have um if we've taken over multiple sockets then we have to go and look in the set of overtaken sockets that we have um all right all right so i guess we need to actually employ our multi-server thingy here with an ability to pass in the socket path so socket path um and it's optional so actually let's write it in this way because we've been moving towards using optional string anyway so let's just take another step in that direction and then here we can just pass on socket path value or like that all right and then that should allow us to wait where was it sql server no no inspector server so here that will allow us to use multi-server here as well client connection try create and then the argument is temp portal inspector all right cool yeah and then um inspector server is a bit special because it actually has two servers uh so the second one is not managed in this kind of standard pattern way so we'll just ignore that and let's instead look at whoever is remaining we have windows server okay so windows server and audio servers left all right all right what does windows server do windows server sets up two local server objects sure and then it sets up an on accept handler in exactly the same way as these other things so we could do the same thing for these so let's do that um so here we'll say like wait what do we say exactly i guess client connection and wm client connection all right and multi-server this is going to be nice so that's our standard client connection manager and then we also have a window manager client connection manager okay so we have those two boys and then we just have to actually instantiate them so i think we'll just do it here um try create passing in these paths of course so there's that one and we'll wrap it in must since there's no propagation path here this is also really old code that we need to give some love but we can't do everything at once so we will just do this thing right now all right so that gets rid of that and we also get rid of these very very nice nice and tidy could we drop anything more from here i think the local server include we don't need anymore right yeah yes okay let's see if this works what are we missing multi-server.h is missing um new client connection wait why sql server main wait what is not a member of ipc why is that oh shoot wait hold on it's in clientconnection.h i thought it was in connection.h for some reason yeah there it is i don't know why i thought that i guess i i was looking at this and then i looked at this thinking oh this is the file name of the current file but that doesn't make any sense uh because it's an include statement so yeah that was the problem all right so let's see if the system runs with these changes it does let's see some fire very cool yeah so that's really sweet i i like this quite a lot um what kind of removal are we looking at here we're looking at 23 insertions 80 deletions of course we do have a new header so the new header um have to do that actually um yeah so 62 insertions 80 deletions so like net delta is not that large but i still think it's a very nice change because it does simplify all of those main functions and it makes it even easier to create a new multi-server if you want to hmm why do we have a windows server connection include here that's very suspicious i feel like we don't need that yeah okay so let's um let's go for it oh now i'm forgetting about audio server actually so let's take a look at that again so what the issue was so the issue that we spotted here was that when audio server creates a new client connection object we pass in the mixer [Music] so the audio server's mixer object we pass that into each new client connection and i guess it's right here yeah yeah so i wonder what we should do about that so the first thing that i notice here is that i don't think anybody else is creating um a mixer right yeah so it's only ever constructed once in the main entry function so we could also make it um a singleton for example and then have a getter for it and then we don't need to pass it here another option would be to make this more flexible so that you can you could provide construction arguments for your client connection somehow but guess the most straightforward thing to do at this point is to just ignore it because i mean we're certainly not going to change the way this works here in this and this commit at the very least because that would be kind of um violating the the this the atomic purpose of this commit so let's just do this one first so libya ipc mult um introduce add um ipc multi server um abstraction let's say or convenience class this um encapsulates um what our multi-client ibc servers typically do on startup create a core local server take or a socket um fd a listening socket file descriptor from system server set up an accept handler for incoming connections ipc multi server does all this for you um all you need to do all you have to do is provide the type name provide the the client the relevant client connection type it's a template argument template parameter this is a template argument in this case because it's that instantiation i never thought about that much actually but i guess when you're instantiating a template then you're passing template arguments but when you're declaring a template then you're specifying template parameters well probably somebody's gonna tell me i'm wrong in some subtle way now but we can live with that still that turned out pretty good yeah so what do i feel about this i guess i guess i kind of feel like we can um like we can make the mixer singleton um i'm tempted to just ignore it actually and think more about it did i miss any other ones doesn't look like it okay and what about single server though so here for example web content takes over an already accepted socket let's see we have a couple of those right oh actually we have quite a few so we have websocket web content request server image decoder and file system access server they all do they're all single server single client servers hmm should we come up with something for that would this be meaningfully like basically it's about these two lines here would they be meaningfully better if written this way um i guess what you would you don't actually need a single server object rather you just want to accept a single client so um you have an already accepted socket coming your way so we don't have to accept we just have to wrap it in this thing right here so um takeover accepted client or something like that from system server web content client connection like you could you could do this um and i suppose it is it is nicer actually because it does it does sort of encapsulate and hide the fact that that you're creating this like local socket object and blah blah blah and you don't have to care about that part because like it is it is busy work at the end of the day even if it's just a single line of busy work and this does feel a bit more descriptive so i do i do like it let's do it uh and even though we don't have a single server uh single server object or class i feel like it's still fine to call it single server.h because it kind of it still kind of mirrors um conceptually the multi-server concept right so like multi-servers include multi-server.h single servers include single server.h so um and then what would it do it would just have namespace ipc uh template type name client connection type um and i guess we want to have an error or in this case so error or non-null ref putter to connection type i guess something like that and then wait does that thing take a string it does take a string does anybody pass a string to that thing nobody passes a string to it but it takes a string so it has that functionality but nobody uses it interesting well let's just not um take a string for now and then if we end up needing to accept um from somebody who has multiple takeover channels then we can deal with that then but for now let's do it the easy way so first thing we do is we just create a socket and we do a takeover and then we create a client and return it so return uh client connection type right yeah select that and then we put that in single server.h okay something like that um is that everything we need wait do we need local socket or does this thing bring that in uh ipc connection brings in local socket yes we're good we're good okay yeah that's actually kind of neat though okay and then we lose that include it's actually pretty this is actually kind of kind of pretty in a way okay so take over accepted um oh there's more of them yeah here our language servers also use this type okay so let's get rid of some of those and replace them with single server try uh ipc takeover accepted client cpp client connection look at that that's rather nice that is rather nice we'll do the same thing for this guy right here okay okay local server we don't need that yeah these are really nice just uh i was skeptical at first but like the more i see it now the more i like it because it just like now we just hide away the fact that you are creating a socket um socket object like you don't need to look at that it has no importance to you what we really do care about is the fact that we create a client um yeah yeah delicious okay oh here we are doing the right thing yeah just a couple more and request server finally so client is try ipc take oh we need to include it nice and nice okay will everything just work oh we have trouble okay i probably got a little carried away i forgot to wrap um things in a non-old ref putter i think or did i wait what did i do wrong let's let's try to build that again okay so what are you telling me we cannot create a client connection passing in a socket like that file system access server client connection why not what do you want you want a socket and a client id sure and what did i provide for you oh i need to also provide an um a client id so the client id is meaningless in a single client server or single server uh because it only ever has one client so uh it's kind of silly that we have to have a number at all this is just um the ipc system assigns a client id to every client connection so um we just use one for single servers and i think that's okay okay now what more don't you like wait what's the problem here we are in an image decoder image decoder main [Music] image current client connection wait so this thing was not taking hmm this thing was not taking an id actually yeah why do i have to pass the id that's still stupid because these client connections can just hide that detail internally so this guy right here he shouldn't have to care about this yeah that should be handled internally by the client connection subclass so this guy right here shouldn't be taking a client id because there's only ever one client anyway since it's a single server so we'll just drop that right there pass one and yeah and then this thing is is written in a way it has a hash map of client connections but it's a single server so it only ever has one client connection anyway and we should get rid of that map but that's outside of the scope of what we're doing right now so let's just ignore it and probably the same we're going to see the same problem here in shell plane connection yeah it takes a client id as well so um do that you don't need a client id my little friend it's okay you can just pass uno also the c plus plus language server of course we have to fix that one as well no big deal request server okay so it seems like this this like silliness was actually in quite a few places request server client connection also take a client id there we go we should also fix that up um okay even more all right all right no big deal all right will this be the last one yes let's give it a whirl everything comes up let's go to the online web in cyberspace and it seems like everything is working handy dandy so kick ass let's make a commit out of this so what kind of delta are we looking at here very humble you know 38 insertions 45 deletions uh which makes sense not a not a very big change but a slight gain in expressiveness i still definitely believe i think it turned out pretty good so let's commit that um yeah so um this can be this is an abstraction or encapsulation of the common work done by all of our single client ipc servers on startup um take over a core create a core local socket take over um taking over accepted file descriptor that's just fd everybody knows fd come on um create a an application specific or a a service specific no application specific client connection object wrapping the socket um this uh yeah it's not a huge change in terms of lines saved but i do feel that it improves expressiveness cool all right so i think these turned out pretty good and yeah i'm gonna keep ignoring the audio server one um we'll just ignore that because i don't know what i feel about passing the mixer in yet should we make it a singleton or should we have a mechanism to pass custom arguments to client connections of glass constructors i don't know what i like better so i will have to meditate on that one but in terms of where we started like config server and launch server and those guys i think this turned out really really sweet a single line everything you now need to set up a multiplexing multi-client ipc server very very handy so i think this will be the end of today's video this was a heavy refactoring video and i am very pleased with the outcome if anybody has suggestions for how we can improve this further um then i'm of course very interested so please let me know uh other than that i hope that you saw something interesting today and um thank you very much for hanging out i'll see you all next time bye

Channel: Andreas Kling

Views: 6,089

Keywords: serenityos, c++, programming, osdev

Id: mOdoDert2uM

Length: 52min 56sec (3176 seconds)

Published: Mon Dec 06 2021