Destroying Client-Server Barriers using TypeScript - London Node User Group - May 2023

hi everyone my name is Akash today I'm going to talk about a little library that I've built called Rocket RPC it's good to see that you know we've filled most seats today and it's good to have a tight-knit community because we can get to know each other individually so yeah it's nice to have this crowd here today so yeah before I start with the meat of my talk I just wanted to give a little context on what rocket RPC is how it was built things like that so yeah context the idea for Rocket RPC started out all the way in 2020. me and a few friends during the covert period were thinking of building a restaurant ordering app where you would go into a restaurant or use a QR code to scan it and then instead of you having to speak with a waiter they just bring the food to you but when you pay it online so what you see everywhere nowadays so we started along building it we started with the user facing Pages first so basically like a ordering type app where you would have a different screens and components each hypothetically having their own API so for example if you wanted to display a list of all of the items in a restaurant you would make an API call if you wanted to get a an item's details you would make another API call and I mean of course if we were building an application for restaurants we needed to build an admin dashboard for them as well so this is I mean this is an example from the internet but this is sort of what we built so we had an admin dashboard which comes with another set of apis with itself but what's the common connecting theme whenever we I mean when we were building this application whenever you're building an application yeah connecting thread it's all crud in the end so whenever you're building something you're only I mean whenever you're building apis you're mostly dealing with crud you aren't doing anything too complicated with it and that's that's what we realized as well so how do we avoid this mess of apis in 2020 we had sort of two solutions One is using hosted platforms like super bass or Firebase Firebase but the cons of them were that we would have to write very framework specific code so how many of you guys have used Super Bass or Firebase uh how many guys how many of you continue to use it after a while not many do because you don't really enjoy working I mean within someone else's like developer ergonomics that was the case with us as well now there's serverless functions we know there is always a server in the end of serverless functions and unfortunately like if you have used these kinds of services like next Js Edge functions or Netflix functions or Lambda I mean they all run in AWS Lambda in the end it's still sort of like API code you're still reading the request object you're still like writing to the response object so it doesn't really get you away from the apis and it's vendor lock-in again because you're locked into whatever vendor you're running with so today I'm going to talk about the solution called Rocket RPC it's built using typescript hence the title destroying client-side barrier client server barriers with typescript and yeah my name is Akash I'm a software engineer at sigtec we are hiring by the way so reach out to me if you want to look for a new role and you can have a look at my website the writing.dev as well and yeah enough talk let's actually get into the meat of how it works and what it looks like so we are going to look at a very simple application today basically to lose up the best way to experiment with anything and let me go into the app sorry I was working till late last night to make this work because it was just breaking I mean I've done this demo tons of times but it was breaking the one time that I want to do it in front of an audience so yeah this is what we are looking at I'll try to clear the console there yeah just a normal to-do's app you can add a few to Do's in here and then you can check them and check them remove them from the list whatever however I mean the basic to-do's example what does the code look like I have created a simple to Do's component here so we don't really have to worry about how it looks internally it take some notes Here it has it exposes a few callbacks so like add node to add a note on done so what happens when someone says that a note is done what happens when they actually delete it and then a notes object here I mean with a simple use state to handle notes so just giving you an idea of what the client side looks like now the server side this is where it starts to get interesting so this is what a server in rocket RPC actually looks like in short it's basically an API object that you're exposed to the client and any method and object that you define inside of this API object is automatically exposed to the client you don't really have to change anything else and because here comes the typescript part because of typescript I can use type inferences and all of the other magic we'll talk about that soon to expose that this type to the front end and then you get all of the benefits like type completion and every other thing which we'll have a look at as well this is this is basically the rocket RPC server you just pass in any HTTP module or a port number to it your API object and then the server option so in this case I'm just opening up the chords to everything so I can ex I can X access these objects from a react app so yeah we just had a look at the react app now we'll try to add some cool bits to it via rocket RPC so firstly let me access my rocket RPC client here using use client this is just a simple hook that I created which deals with like whoops that's my timeout I should actually close that in case it pops up again sorry so yeah this just accepts port number so I have my Prisma server running on Port 8080 I think yeah 8080 here so I just pass an http localhost 8080 which connects the client to the server and now now is when we start to do interesting things so right now we just have Hello World here again the most basic API that you can think of whenever you start a new writing in a new programming language or a new framework so uh let's just call that con console log that value right now so console log let's say client and of course the magic happens when I press the dot operator on it because it gives me type completions from everything on the back end so here I have hello I call that method of course because we defined it as a method here and now if I go to my front end initially it generates a promise because uh since it since the RPC connects over a network everything needs to be returned as a promise and then it result it Returns the result as hello world and this is where the magic just begins because we can start writing our own custom methods so for example if I want to write a method to sum two numbers so number one let's say number num2 another number and this returns num1 plus num2 and of course typescript knows at this point that if you are summing two numbers and returning it it's going to be a number so if we try to reconstruct this so const this is equal to client and again since this is completely running over typescript this will just yeah this will give us Auto completions here and this sum again like we can call this method anywhere so if I say console log sum 10 comma 20 or whatever and then check that out here then it will have that sum there so okay so we understand that we Define a method in here we access it on the front end that's all right so what else can this do so I was talking about exposing objects before and this is where sort of the cool part comes in so I can actually expose my whole Prisma client via rocket RPC so you can access Prisma from the front end which is sort of exciting I mean if you are into security it's not exciting it's exciting in a different way but yeah for some reason it doesn't give me auto completions for the Prisma client itself but I'll just write the temporal import down here quickly and nice so I exported a prismap client but what does it actually have right now so I just have a simple to do schema in my back end which maps to the to-do object that I have created here which is here yeah this is to do which is just text string status Boolean and ID string I have to use an ID to uniquely identify each element and react As We Know and oops I think I opened a new thing so it closed the whole thing yeah so yeah this is the model for to do it has the same structure ID status text ideally I should just have used this type while creating the to do object there but yeah we'll just we'll get to that soon enough and yeah I've exposed this right now so what I am going to do right now is try to change this application as much as I can like just add persistence to it by accessing my Prisma client from the front end directly and yeah this is going to be fun so of course type complete completions by this point you know what's going to happen we are going to get the Prisma client all of its type completions here as well that includes like all of the schemas that you have defined in Prisma dot schema as well of course and yeah what's the first step the first step is that we need to render our to-do's here we have no set notes to do so to handle effects of course on load we use a use effect and then what do we do we use Prisma Dot Prisma dot to dot find many and then again all of this is happening over the network none of this computation actually takes place on the front end and this is what rocket RPC is supposed to be about you just I mean you just expose your objects and then the library takes care of everything so because it's using the same type I can just pass in the set notes callback to dot then and it will automatically generate the initial set of nodes here obviously this doesn't really do anything right now because we haven't set any data in the table yet so we're gonna start doing that now so the first callback that we should probably fix is add note so I mean I can add notes here but this isn't this doesn't actually store the data to the Prisma rdb yet this just yeah it's just on the front end so how do I add persistence here I'll just add Prisma dot to do again I feel like I'm doing this so many times I can probably destructure on this here as well so const to do is equal to Prisma Maybe I don't know you guys can tell me if this is safe to do but yeah probably do something like to do dot create and then pass in the data here so that would be the new note which we have here so I mean the old callback it was it was just modifying the existing notes object and then appending a new note at the end via the spread operator everyone does it this way nowadays because everyone's used to safety in arrays immutability sorry but yeah yeah so we can create a new object and so earlier how I was generating the IDE was by using the date object which is a very happy way of doing it you could use uuids or whatever I guess but in this case since we are using Prisma now Prisma Auto generates IDs for us so that's nice I can just remove the ID part from here and then inside of dot then I'll just do the same thing no new note and then this can go in there and why is this complaining Let's ignore it for a bit new note yep that does it so now let's try to add some new notes in here and if I refresh the page it works so yeah we have persistence now the Prisma layer thank you and yeah I'm just gonna continue to do this but I guess you get the idea like I can use like to do dot update here and to do dot delete here and then it will just use the same Prisma methods exposing it over the client and yeah that's pretty much the functionality of Rocket RPC but there's a lot that you can do with it to be honest because one of the main things that I Envision it in helping out with is sort of graphql resolvers or like micro service a micro service architecture without actually writing any micro services so you write servers which expose whatever method you want to write over rocket or PC and like here for example this could be a Prisma client I mean a Prisma client which resides on a different server than your actual graphql resolver and this resolver could connect to the DB through Prisma over the network and then return I mean to lose or like do whatever you want to through the graphql resolver another thing you can do is also basic access other things I mean it just it doesn't just work with Prisma it works with sqlize it works right with Mongoose anything that you can think of and expose those objects over the client as well so I mean I can define an API like this like get users user.find all whatever and then use that within the resolver as well so you get all of the benefits of working with multiple orms and it's not just like different orms doesn't like sqlies and Prisma it also covers certain fall what can I say like cons of orms like Prisma as well so for example you know that you can't have multiple Prisma clients which are connecting to different databases within the same app but here you can expose a different Prisma database from different servers and then use all of that within the same graphql resolver or within the same app so you are not limited by whatever those applications can do either and before I go to the next slide there's also Apple I mean I've tried this with GPU JS as well so for example you can if you if you're doing some complex calculation on the front and you can offload all of that to the back end instead by just passing in the correct you know input types to gpu.js or whatever and then making the computations on your node server or somewhere else and yeah I hope by the time you saw this it was mind blowing for you but and you might have the question how does it work so there's a few things going behind it one of which is typescript of course we were just talking about how great typescript is and yeah let me go to the source code of docket RPC to show you how that works so basically what am I using typescript for typescript is providing all of the type inference here so whenever we are exposing a type from the back end wait let me close a few things so here how I'm exposing this type API here this is this is using all of the inferred types of the API object and then you know you can use this as a generic on your client so this is just an example client that I have here but this is using the actual rocket RPC client and what this does is it passes the type that you just exported from your server to the client generic and so this is what results in all of the type completions that you are getting here because it's just using the same type from the back end and let me quickly go to the client and server here feel free to ask me any question about this at any point because we are a small group I think we can do like random q and A's as well so let where should I start from here because there's a few things going on I guess let's have a look at the server server first there's not really a lot going on in terms of typescript and the server it's mostly socket i o as you can see here but we are going to get get into that in a bit I'm just trying to figure out yeah the socket the server doesn't really have to worry about the types here if you look at the client it it accepts a generic which can extend the record of string symbol or number which is basically what a record accepts everything accepts and then unknown as the value because we don't know what the value is going to be and how do I what do I do with this type I run this through a custom type that I have written called promise if I promise fire record which is this complex thing which HR GPT helped me write a little bit as well and this basically adds promise return type extends promise of any like it just adds promise to the return type wherever it is absent and that's what gives you the correct like away Types on the front end and yeah this is how typescript helps us because the client proxy Returns on let me have a look at what this returns yeah it basically returns promise if I record API so we get the type inferences in the client as well and what else does it work on it works on object proxies so object proxies are the fun bit I feel like object proxies are quite underexplored Prisma uses object proxies trpc another Library you might have heard of it uses object proxies under the hood as well and I am using it as well because they're really cool in what they do so basically what they do is they allow you to create a fake object and then it gives you certain traps on that object so whenever a user is accessing a certain property or trying to call a certain method then you can sort of execute your own custom code instead of returning whatever value that the user actually wanted to wanted to receive so in this case I'm using two traps for this object proxy this syntax this little syntax here new proxy and then a method inside of it this is basic basically the initializer method and then these are the traps that you can write for it yeah this should just come from JavaScript this isn't a custom library and then the get and apply trap what does the get trap do the get trap allows you to trap on the dot operator so whenever someone is using the dot operator you can help them do something else instead so how does this help me in this case so in two ways I have a special context called Rocket RPC context where I expose certain values like what the current socket client is and a bunch of other things through with it and if a user is traversing through a path so for example prisma.tudo dot find many so when when a user is saying Prisma what does that actually mean so I have to generate that path and then send that to the back end so I think it should be logged here as well yeah so a path like this is generated based on what the user was selecting on the front end and then it's passed to the back end for execution and where is it actually passed to the back end this happens on the apply trap the apply trap is called whenever you are calling a method inside of an object proxy on object proxy so that is why it's sort of geared towards methods right now and not towards objects rocket RPC I mean so when you are doing doing these things like calling the I mean basically calling a method on it it sends it to I mean it traps on it on here on the apply trap and object proxies are really fun because I mean using typescript I can basically destructure or destructure it at any point and when I'm calling the get or the path trap I mean whenever user is calling one of those operators it actually generates that prop that path correctly even though you have like de-structured on it like 10 times or whatever typescript takes typescript and JavaScript they take care of everything and yeah so that's where object proxies come in and right before I get into how this actually communicates with the back end I'll tell you about the last part that is Socket i o so for the network layer I choose I chose socket IO here because it is the most I would say reliable library to do server side communication it starts with web sockets first because web sockets are really fast and I mean there's benchmarks on it as well and then if a server if a client doesn't support web sockets then it falls back to http long polling which is basically it keeps on pinging the server for updates and how am I using it so on the client side this bit is a bit complex because I'm using promise queues and things like that if anyone has worked with multiple promises which resolve at random times you know what a promise queue is I guess so initially I just create a new socket appointed to my server connected and then log like rocket or PC client connected so that the developer knows that it's now working and whenever the apply trap is called I generate an identifier to identify my unique function call in the within the promise queue and then it generates a list of parameters that the backend uses being the ID what the path was which basically which procedure to call the parameters which parameters should I pass to it and again I mean this isn't like science fiction code I guess this just comes from JavaScript so I'm not even using any library and then what I'm actually using the library for is Socket IO so I emit all of this all of these parameters to the back end and then what I do is I generate a new promise which I then pass to the promise queue here if I'm not mistaken wait for result yeah so it then generates generates I guess this is more like a hash map like a promise hash map I guess so it passes that the promise whatever the method call with the ID that we just generated and passes it to the queue and how do I use this so firstly let's look at the Q object it's basically like a mapping between a string and whatever method response that we were waiting for and whenever the back end replies with a function response what we do is we destructure a bunch of things from the from that result so like the result the ID the status the error and this is all the error handling that I have right now but I'm hoping to improve it and maybe I get some contributors to help me with it of course and then I use the promise in this way so key of ID of results since we were storing the resolve of the promise on the promise queue so I can call it with the result and then it shall result it shall return that result to the particular method which it was called from and then I can delete it from the queue on the back end what this does is it just opens a new websocket for clients to connect to and it waits for function call and I think that's all it listens to yeah that's all it listens to so function call is all it listens to basically it's a server which is responding to function calls from the client and what it does is it generates the procedure path I mean basically procedure split is the path but separated so if I'm passing Prisma dot to do dot find many then I get Prisma comma to do comma find many which I can then use to you might have guessed by now basically I can use it to wire square brackets I can access the existing API object that I have and then using that I can apply that function call on it and since I get the parameter parameters from my front end as well I can just use this so I can get to my procedure by iterating over the procedure path and then calling procedure params and then I just return this result to the front end and that's it that's really how simple it is right now sometimes it I mean if you might get errors While You're Building Things what I'm doing right now is pretty simple I'm just printing those errors to the front end like in passing them over the network and just printing them to the front end instead it could be better I'm open for suggestions for this as well and so yeah that was my talk if anyone wants to contribute to this you could go to rocketrpc.com if anyone wants to try this you could try it out as well it's good for inter inter-process communication like client server communication anything that you can think of and thank you that was it yeah before I forget I'm at the writing Dev on Twitter or the writing.dev does anyone have any questions hey great talk why does it use websockets rather than HTTP for communication yeah I guess I mean there's advantages to both HTTP requests and websockets so HTTP requests the main advantage that they have is that you can cash on their responses so if you were looking for like caching I guess HTTP would be a better way to go for it but in my case I think socket i o was a better solution because it allows you to send as many requests as you want over the network Without Really blocking anything and also because you don't have to worry about batching and stuff in HTTP like because the payload and response headers are so large you might want to batch multiple requests together so I don't really have to worry about anything like that over web sockets nice thank you thank you for the talk because the initial socket IO so like if the data change from the server side can also can you can it push the change to the front end yeah you mean sort of like server side events right like passing data from the server to the client yeah I mean I guess you could do that I mean you could use socket i o for it you wouldn't really need a separate library for it then like I'm thinking I mean I've been asked this before as well but I'm not sure of a use case where you would want to pass a pass some data from the server to the client maybe while you're streaming maybe like subscriptions or something I guess or some dashboard like when the data change kind of in real time so you don't have to keep holding the server yeah yeah that makes sense cool thank you

Channel: Pusher

Views: 237

Keywords: developer language, language code, libraries, help, tips, forum, meet-up, meet up, learn code, coding education, coding hints and tips, lecture, coding lecture, learn about code, learn a developer language, amazon alexa skills, developer conference, node.js, javascript, backend

Id: wJyqa7COt2E

Length: 29min 8sec (1748 seconds)

Published: Thu Jun 15 2023