Decrusting the axum crate

Hi folks, welcome to another Decrusted stream. We haven't done one of these in a little while, we arguably haven't done any stream in a while. But the Decrusted series, for those who either haven't watched the previous video where we did the Surdy crate, or just it's been a long time, the idea of the Decrusted series is to take some crate that's popular in the ecosystem, and especially one that a lot of people use fairly directly. Like you actually have to interact with it and kind of need to understand how it works under the hood. And just sort of take it apart and figure out, why is the API structure this way? Why does it work the way that it does? What are the techniques that it uses? And really just form a more accurate and useful mental model of how all the pieces fit together so that when you are building your own stuff using that crate, you'll be in a better position to do so. You'll be in a better position to understand why you get the errors you get, understand why something goes wrong, understand how to fix it, or hopefully even just become more fluent with it. And when you want to build something using that framework in the future, you'll be more comfortable with what you have to do and potentially more efficient, more proficient and such. So the crate we're tackling today is one called Axum. And if you haven't heard of Axum, Axum very briefly is a web framework. Think, you know, like Rocket or like Actix or Actix Web rather. And Axum is one that is a little bit different in the sense that it only really provides routing and request handlers. And then all the middleware stuff, anything that you might wanna do in between a request comes in and a response comes back apart from the actual handler function, all of that stuff is handled through the tower crate, in particular, the tower service trait. We'll get into what exactly that means, but the essence of this is that Axiom is actually fairly small. It doesn't implement things like rate limiting, for example, that's not a part of Axum itself. Instead, you get that via these other middlewares that are built around the same trait, this tower service trait. And we'll dig into how all of this works and why they chose to do it this way. But sort of from a 10,000 foot view, it is really just a way to express application routes. So things like, you know, get to slash and what should happen when a request comes in like that. And then in addition, a sort of. mechanism for writing those handlers that receive the requests and extracting interesting information in those handlers. So let's go ahead and just grab the example and then start straight from running code. Let's do axum fun times. Okay, so we're gonna add, sure, we'll add Surdy, we'll add Tokyo and we'll add Axum. And then instead of this, we'll start with this right here. Right, and we'll do add tracing and we'll add tracing subscriber. And that should give us all the stuff we need to get started. Great, so if I now just, I just made this thing, I'm gonna cargo run and that'll build some stuff. And now I should be able to just curl localhost port 3000, and I get back Hello World. Okay, so we now have the sort of very rough idea of Crate. We'll actually walk through the code, but like this is what it does. It has code that defines the routes that you want to accept. It sets up a web server. It defines handlers for each of them. And then you can access that web server through something like curl. So that's the basic functionality of the crate. Okay, so let's then, let's see, does it have an HTTP parser or does it rely on Hyper? It uses Hyper under the hood. So Hyper takes care of all the HTTP parts, or at least all of the sort of protocol level HTTP parts. And then Axum builds on top of Hyper. So one of the things that's interesting about Axum's approach to the whole web framework thing is they lean a lot on the rest of the ecosystem. Um, so, you know, they use hyper for handling all the HTTP bits. They use tower for handling everything that has to do with, um, um, middleware and the like, um, they use tracing for anything that has to do with logs. Um, and, uh, they use the, what's it called? Oh, mini match, no, it is called match it. It uses the match it crate for doing route matching in an efficient way. So there's a bunch of bits in there in Axiom that are really just intended to be the glue between all of these to provide you the framework you need to write HTTP services. Okay, so let's now actually walk through the code here. So let's ignore the imports for now. We have a Tokyo main and async FN main. Okay, great, runs on the Tokyo runtime. Then we can mostly ignore that. We initialize the tracing subscriber. We'll almost certainly do a decrusted on tracing because there's so much to talk about there. So I'm not really going to talk a lot about the tracing parts of this as we go through Axiom because we'll just be handling that separately. There's nothing. or there's very little that's unique about the integration between axiom and tracing that we need to know about here. But we set it up just so that we get log output in this case. Okay, so the main thing that we, or where we start with the main actual axiom logic is we create a new router. And a router is the sort of primary entry point in your application. It is the thing that when a request comes in, the router determines what function ultimately gets called to handle that request. A router has a bunch of routes. And so in this case, you see you have a route slash. And when the request comes in for... for slash, you'll see this bit on the right, which says get and root. Root is an asynchronous function that we defined ourselves right here. So it's just an asynchronous function and it just returns a static string. And so what this implies is when a request comes in for root, then reply with this string right here. And in fact, if we do something like curl verbose, you'll see back that the content type here is just text plain. It's not HTML, it's nothing like that. It's generally just the plain text hello world that we get back. So that's if we do a get request to root. The get here, if we scroll back to the imports, comes from routing colon colon get. So this is a method matcher. And in fact, if you go back and look at axioms, if you look at routing and see the router submodule, no, the router submodule, You'll see that there are a bunch of these. So there's options, patch, post, put, get, delete, trace, head. So all of the sort of HTTP verbs are treated as separate method routers here. You can also use any, which if you want to say, I want to take any method under this path. And in addition, you can just create yourself. So you can create one of these method filters that allows you to say, you know, get and post, for example, all get passed in here. If we try to use one that there's not a route for, so if we try to do something like X host, Uh, and I still want verbose, please. Then we get back four or five method not allowed. So, so axiom tries to follow the HTTP standard here, where if a, if the route exists, but the method, the HTTP verb you tried to use does not, um, then you get four or five method not allowed. Uh, if we try to access a path that doesn't exist. So if we do foo, then instead we get a four or four not found because that path does not exist. You can customize this if you want, but the default behavior is, is basically to, um, to, to. follow the spec to the extent that it's reasonable. And we'll talk a lot about how exactly this sort of seemingly magic works. We can just have a random ASICFN and it just works when you pass it to get. You'll see there's another route here, which is to slash users that goes, requires the post method to be used to the post HTTP verb. And that goes to the create user function. And the create user function you see has a bit more heft to it rather than the root function. Create user takes arguments. And if you just read this naively, you can sort of guess what it does, right? It says that it expects that the body that's provided to this request is JSON and that the contents of that JSON deserializes into the create user type. And create user is just something that derives deserialize here. Um, and that's just an argument to the function. You might think this looks weird. Like why is there something on the left of the colon? Um, this is just a, this is just structure pattern matching. So you could do this, uh, and then payload is of type Jason. So we could do inside of the function payload equals payload dot zero. Um, if we look at the Jason type, it is just a, a tuple struct with a single field in it. So like this is the dot zero. Uh, so there's nothing to unwrap or anything. But rather than having to do this, let payload equals payload.zero inside of the function, we can use pattern matching right up here to deconstruct that tuple struct directly in the argument list. You can use this for any function. This is not an axiom feature. It just happens to be really useful in the axiom case. And then you see the create user returns a tuple of status code and we see this JSON object again with user as the parameterized type to JSON here. And as you would expect, this means that it returns a status code, which is the status code here comes from the HTTP module of Axiom. And it defines all of the standard HTTP status codes. In fact, I think it in fact is a re-export. not entirely wrong. Yeah, so the HTTP module is just a re-export of the HTTP crate, which just has definitions for all of the different verbs and methods and things in the HTTP standard. Um, so all the status codes are in there. So as you might expect, returning a tuple of these two means that you get to set both the status code for the response. And you say the body of the response should be the Jason encoding of a user struct. Um, and that just kind of works like you just give a tuple saying the status code should be created and the body should be this, this, but Jason serialized. Uh, if we go back now and we try to post, um, So we'll do data and what's the data it expects? It expects a username. And we do this, oops, users. It says unsupported media type, because we didn't declare that the content type of what we sent in was actually application slash JSON. So we do dash H. Content type, application, JSON. Uh, fail to parse the request body. That's because this is not valid. Jason, this is not a Jason. Uh, so you see now I passed in this Jason blob. I told the server that the content type is Jason. Uh, and you see the re the request we sent now has application. Jason's content type. The response we get is indeed two Oh one created. So the status code that's in the handler, you see the response content type is automatically set to application. Jason. because it understands that if you're encoding a JSON response as the body, you should also set the content type to JSON. Oh yeah, there's also "-json", I think, instead of data, but it doesn't really matter. And you see that we got a JSON body back that has the encoding of that user struct that it produced. Okay, so this is just roughly how you write axiom handlers. Questions about this before we dig into how this actually works, because it's pretty cool how this works. What happens when the payload doesn't match? Does axiom return specific HTTP error codes? I think it just gives bad request. So you can see this up here where I didn't correctly encode username with double quotes, like it basically wasn't valid JSON. This would also apply if it was valid JSON, but it wasn't a valid serialization of the struct that we try to deserialize it into, then you just get a 400 bad request. And you see here by default, it includes a description of like basically a debug print of the deserialization error. You can turn that off to tell it to not sort of echo that back to the user if you're not running in debug mode, for example. In fact, it might even do this by default if you run it in release mode, then I don't think it prints this error. Um, Is it mandatory to have the status code first and the return tuple? No, it's not. In fact, as you'll see soon, the return type here can be a lot of different things. So I can do this. Like so. Restart the server. And now if I post this, I get a 200 OK back instead, instead of a 201 created. So it's... polymorphic in the return type as well, if you want to think about it that way. Let me just add about the variadic generics design pattern they use for services. We'll get into all of that later. What happens when a route function panics? Routes aren't functions. Oh, you mean when the handler panics. We can try it. If I panic in here, What happens? I run this, send a request. I get empty reply from server and we got a panic. But what's important here is that these panics are isolated per Tokyo task. So if one Tokyo task panics, it does not panic the whole web server. It doesn't even panic that handler. So we can run other handlers. If we run that handler again, then it'll still run and it'll just panic again. So they're isolated from each other. So you can panic in there. It's just a panic message will be printed. Are the get or post functions returning functions themselves? Yes, we can go look. So this is generally what I recommend people do if you have questions like this. So the getter here is actually returning a router or a method router. And method router implements, if we go down here a little bit, method router. Oh, where is what I want? Now I can't find the thing I want. It basically gets, so the way the method router actually works is it, internally contains a sort of map based on the method of the incoming request. So it's basically a middleware. When it receives a request, it'll look at its internal map from method to handler, and it will look at the method of the request and it will just call the appropriate handler's handling of that request. So just think of it as like a, it has a mapping, let's do it this way. So the request comes in here and then it just gets routed to the right handler. So the request then goes into, say, this handler here for get requests. It produces a response and it goes back up and back out. And this is where we'll get into the tower service trait for how it does middleware like this. But the router is really just a thing that holds many handler functions and chooses which one to call depending on the properties of the request. Is there a way to return that JSON serialized object, but force the content type to be something else? Sure. So if you look at the kinds of things that you can return, which is in the response type here, you can return all sorts of things in response. You can return something like append headers, which lets you append additional headers to the response. You can fully customize the response if you want as well. So yes, you can add that here. And in fact, you can do this. So you can now make this a three tuple where one of the things that you're turning is that append headers. And again, we'll get into exactly how that works. I'm just trying to sort of get at all the top level questions before we dig into how this works under the hood. Can you explain the extractor syntax in the case of query parameters? Sure. So if you want to take a query parameter, you do this. Anything that implements deserialize here really, but hashMap string string is a common way to deserialize parameters. And now params is a hashMap from string to string. And inside here I can do params.get my param. to get the value for the query parameter myParam. And query here, just like JSON, is just a tuple struct that comes from the axiom extract module, which has extract query. JSON is also defined under extract. Does it throw an error when a route function has no arguments, but you provide a request body? No, so that's what root does, right? It has no arguments. It doesn't read any. The way to think about that is it doesn't read anything from the request, but it produces a response that is a body. There's no error. Oh yeah, someone pointed out that if you have a panic here, there is technically a middleware you can put in that catches panics in the inner handler and turns them into HTTP error responses rather than just getting like no body connection closed kind of thing. So that's something you can do with do. We're not gonna dig into that here. It's just worth knowing. Okay, great. So now we have a sense for how you structure these handlers and the kind of things you can do here. Let's now dig one level deeper. Oh, actually, there's one last thing I want to talk about, which is starting the actual server. So once you have a router, you have this thing, they call it an app in the example, you create a socket address, like where you want it to listen, and then you call axum server bind, and you provide that address. Axum server is really just a hyper server as you can see here. So this is just reusing the hyper service stack. And bind just, this just returns you back a hyper server. Specifically a hyper server builder. And then when you actually want to start serving requests, you call .serve on that builder. So if I go to the hyper docs, then go to server. Uh, and go to server, you see the bind returns a builder and builder. has a, where are you, has a serve method with lots and lots of trait bounds. But essentially, it takes a thing that can make a service. And what that means is every time a new connection comes in, it calls this thing to produce a service, a service being an implementer of the service trait. We'll get to the service trait in a second. And so each connection gets its own... instance of a service that gets called for every request on that connection. And so serve basically never returns. Serve is just like start the loop to listen for connections now. And anytime there's a new connection, create a new service. And anytime there's a new request on a connection, then pass it to that connections instance of a service and get the response back and then serialize that back over that same connection that the request came in on. And all of these bounds are all about like making sure that you actually have an HTTP connection that you can read and write from, that the errors are sendable across thread boundaries and all that. And then we'll get into exactly what a service is because that's sort of the natural next point to get to. And if we, oh, how did I end up so far down? And so you see here, when you call serve, you get back an actual server object, which is a future that you await. And when you await that, that await essentially never returns. It would return if the hyper server itself panicked, which basically it does not. There is a with graceful shutdown, which lets you take a future here. And when this future returns, this future, When this future here resolves, then the server is going to shut down gracefully, and this await will resolve, and this unwrap will finish with an okay. And so then you might be able to run additional commands after this. So commonly what you would do is do something like set up a one-shot channel where, in fact, we can just do this. Tokyo sync one-shot channel. And then this would be rx.await. And so now, right. So this will now, this graceful shutdown future will resolve the moment we send on shutdown. And so now you can pass the sender part of this channel to like a control C handler or whatever you might want to do. And so whenever you want the server to shut down, you call shutdown.send and you just send an empty thing. And then the server is going to shut down on its own. This future will resolve and you can continue. The other way in which it resolves is if it encounters an error. Like for example, it tries to call accept to get a new connection. And like. the socket itself is broken in the kernel and you get it, or like the, I don't know, the address has been, the network interface has been janked out so you can't accept any more connections. So the kernel returns an error to hyper's accept call. That error would also propagate up here. But let's get rid of that. That's just extra noise. Okay, so that's how you set up an Axiom server. That's sort of the glue that puts all the pieces together. Let's now talk about tower services. So it says pretty prominently in the tower, in the Axiom docs that, I don't know, where can I find the, maybe not here, but in Axiom over here, it says, I thought I saw the word unique here. Has it gone away? Ah, the last point is what sets Axiom apart from other frameworks. That is this point. It takes full advantage of the tower and tower HTTP ecosystem, middleware, services, and utilities. Axiom doesn't have its own middleware system, but instead uses tower service. This means Axiom gets timeouts, tracing, compression, authorization, and more for free. And by free here, they mean via other things that already implement this trait independently of Axiom. Like people don't implement the tower service trait because of Axiom, they implement it for all sorts of other reasons. But if you're using Axiom, you can make use of those implementations, which allows you to do things like share middleware with applications written using Hyper or Tonic. Okay, so let's now look at the service trait from the tower crate. So the tower crate mostly exists around this trait. Like it exists to define this trait and to allow an ecosystem to develop around this service trait. And the service trait fundamentally is a trait that takes in requests and asynchronously produces responses. And these aren't tied to HTTP. They can be requests of any kind, responses of any kind. And the idea is that when you call, what you get back is a future, where that future type's output is a result that is either a response or an error. So it's almost the most generic way you can think of expressing a service, which is the intent, right? That you have this trait that everyone can implement if they have anything that deals with requests and responses. You'll see there's a poll ready and a call here. We're not going to dig too much into why those are there. But the basic thing, in fact, there's a great article. Where can I find this? A blog. No, that's a lie. Where is that article? Maybe it's under Tokyo. Where's the article I'm thinking about? Google. Tower service trait asyncfn. Uh, where is that article? This one, it was from the tower Tokyo blog. Okay. So this, um, this blog post is really good. Um, it basically goes through how we arrived at the exact structure of the service trait. I'll send a link here in chat. Um, and, but the, the basic way to think about service is that a tower service is roughly equivalent to something that implements async FN request to results of response and some error type E. or let's in fact do this for some request and E. So it's sort of generic over request and E, like you might have a function that you might have a type that supports multiple different types of requests. For examples, you can implement the straight multiple times, but the very basic premise here is you have an asynchronous function from request to a response. So that's how to think about tower services. What's really nice about this is that they compose. So imagine that you have one function or one type that implements the tower trait. It has an asynchronous function from request to response. And then you have another function that does the same. So you have, you know, A and B. You can now define a C that takes a request and returns a result of response to E. And what it will do is it will call a of b of r. No, I'm lying. I'm lying. They don't compose in this way. They compose through the use of a separate trait, which we haven't talked about yet, called layer. Here. Where is the layer trait? Here. So this is a separate layer trait that lets you take services and sort of merge them. So you can, in this case, say a.layer b. And what that produces is a thing that is now, that implements service that applies both a and b to your request and returns the response back up. And so you can pass in here the request. And that's how layering kind of works. Again, we're not diving too deep into the mechanisms of tower here, but I more want to give you the mental model for what these services are. They are ways to express asynchronous mappings from requests to responses. And every Axiom handler is fundamentally one of these, or rather, to be more accurate, gets turned into one of these. Okay, so that's enough stuff you need to know about service in order to follow the rest of this, I think. So that then gets us to the question of, well, how, when a request comes in, how does it actually end up calling root and how does that get turned into response? Like we have this function, but how does that turn into like something that takes an HTTP request and turns it into an HTTP response. Like how does this transformation happen? Is somehow transformed into by axiom, right? Somehow that happens. And somehow that also happens for this. this function, which is constructed very differently, somehow also gets turned into that same tower service, that same kind of asynchronous mapping. Right? So that's the idea here, that you want all of your handlers to eventually turn into one of these things. And if we go back to something like a router, right? So we were talking earlier about what does GET really do? Well, really you can think of GET as, It gets a request in. And it's eventually going to return a response. And really, it has like a map somewhere of all of the inner handlers. In fact, maybe even let's reconstruct method handler. Is H, where H is some handler type. Let's do get and post for now. And so we're gonna implement service for method handler. And service takes a request and response is gonna be HTTP response. I'm not gonna write this accurately, but just accurately enough that it's fine. And so when a request comes in of type HTTP request, then we have to asynchronously produce a response. Well, what we can do is we can match on rec.method. And if it is get, then we call self.get of rec.await. And if it's post, So that is what a method router is. So when you call the get function, the get router here, what you're really doing is you're constructing one of these method handlers and saying that the handler for get is the function that was passed in and the handler for all the other methods is return the method not allowed error. That's the way to think about it. Like there's sort of a fallback here that's also a handler. And so all of these are really an option handler. Um, and so this is like, if self dot get is some, this is if self dot post is some, right? You get the idea. And if none of these really go anywhere, uh, so for anything else, then we call self dot fallback of rec dot await. And the fallback by default is return a four or five method not allowed. So that's what a method handler is. A method handler is really a service that wraps a bunch of other services. These Hs here are themselves asynchronous functions from requests to responses. And then now you can start to sort of see how this composes, right? So imagine that, imagine you have this, you know, the outer router, it needs to know, well, how do I route this path versus this path? Well, I might have something like a path handler. And it is just a paths, which is a hash map from string to handler. And it too has an input like this. And again, H here is just a, a service HTTP request, really. So we could even do, if you wanted to think about it this way, something like a box, din, service, HTTP request. And response is HTTP response. So in fact, if we want to be nice to ourselves here, type h equals that. And now this applies to the example further down as well. And again, of course, I haven't actually imported any of these, like this is going to produce a bunch of errors, but you can sort of glance at it and see that this is what makes sense. So if we now implement this for path handler, then it, instead of matching on the method, it's going to match on the path, or rather it's going to do self.paths.getRect.Path. And we're gonna, and if let some handler is equal to that, then handler of request.await, return that. Otherwise, you know, self.fallback, rec.await. There are lots of errors here, but hopefully you get the idea. Let me get rid of some of the errors that might actually make it easier to read. Right, so a path handler is also just a service that takes a request and returns a response, but internally has a bunch of services and it chooses which one to forward to. That's how routing works. And it's really nice that you can just compose them in this way. You can imagine constructing your own routing function that just has some other logic for how it chooses which handler to call. but this still gets us to this box didn't service bit. Like what this really means, if you think about it, is that the argument to get here has to implement handler. Like it has to implement service from request to response, at least kind of. right? But here we're calling it with one function that's just an asynchronous function from nothing to string. Here we're calling it with an asynchronous function that takes some like JSON object as an argument and returns a tuple that has a status code. Like, how does that mapping happen from the handler to a tower service? If we go further down, right, like this is somehow transformed into H, that type alias we had further up. or if we want to copy it so that we see it again, it's somehow transformed into this. How does that happen? And similarly down here, this one is also transformed into that same type. How? And this is where we get into what I think is the sort of, maybe the nugget of Axiom. So let's now dig ourselves back up a little bit to here and go to handlers. In Axiom, a handler is an asynchronous function that accepts zero more extractors. We'll talk about extractors later as arguments and returns something that can be converted into a response. So we already see the sort of shape of this, right? The thing that you return has to be possible to turn into a response. Fair enough. Handlers are where your application logic lives and Axiom applications are built by routing between handlers. That all makes sense to us. Okay, let's look at the handler module. Here they just give a bunch of examples of them, but let's go down and look at the handler trait. Okay, the handler trait is... kind of close, right? So a handler is something that, okay, it has a future. You can call it with a request and state, we don't know what state is yet, and returns one of these futures. And the output from that future has to be an axiom response. Okay. So it takes an axiom request or an HTTP request and returns an axiom response. This looks a lot like service. And in fact, we see here handler service. We look into handler service. Handler service, slow that down a little bit, implements... Tower service. Okay. So there's like some path here where you can start to sort of see the shape. That if we go back here to the method signature of the routing. If we go to... No, that's unhelpful. Routing. Look at get. So get takes a handler, h, where h implements handler. So what that means is this asynchronous function from root to static string implements handler. Otherwise, we wouldn't be able to call get with root. Right? Up here, we call get and we pass in root. So root must implement handler. Otherwise we wouldn't be able to call this function because that is the trait bound of get. And similarly, post is passing create user. Let's assume that post has the same trait bound, which it does. Then that means a create user. So this function also implements handler somehow. Okay, so this handler trait, this whole thing is somehow implemented for all of our random functions. And when we change this to have like query with params here, and hash map string to string, and we added in what append headers here, when we made all these modifications, this thing somehow still implements handler. How does that work? How is it that almost any function we write here, this is not quite true, but it feels like anything we wanna put here just kind of works. It somehow is a valid handler. Well, now we get to the key. So the handler trait is automatically implemented for a lot of types. And if you look carefully, you see there's a pattern here. Like it's not random types, it's for a bunch of tuple types with an increasing number of tuples. This sounds suspiciously as though there is a macro at use. And of course there is. So if we go into the source here and we go to handler, and we go to mod, Let me scroll down. Okay, we see some macro magic here. So you see there's an implementation of handler where the first thing here is a, the empty argument list, basically, and S. We don't know what S is yet. It's state, but we haven't talked about it yet. If you have something that is just like the empty handler, then you do nothing. If it takes no arguments, then you just call it with no arguments and you turn whatever comes back as a response. That's fine. And then we have this ugly macro business here. And we're actually going to read through this macro in a second. If we scroll down here, You see, here, all the tuples, ImpulHandler. So it's passing a macro to another macro. Okay, what's all the tuples? Let's go over here to all the tuples. Aha, all of the tuples indeed. So you see here, this calls the macro that's passed in with tuples of increasing length. Why? Why does it do all this? Well, if we go back to handler, this first type, this T that we see here, let's look at the... Where is my handler service? There's so many trade bounds. I want to find, right. So if we scroll down here, let's look for something that is a meaningful length, like this one MT one T two. So handler is implemented for any F, F is generic here, for any F, where F implements FN ones, FN ones is, we talked about this in the crust of rust on function pointer types. So a FN ones that takes two arguments, T one and T two, notice that this matches with what this first type is in the handler trait. So any function, this handler trait is implemented for any function F that takes two arguments, arguments and returns a FUTE. FUTE is a generic where FUTE is a future. And the output of that future, the thing that the future returns is a REST, which is a generic where REST implements inter-response. And inter-response, as you can guess from the name, is something that can be turned into an HTTP response. And then you see the last bit where T1 and T2 implement from request parts and from request. And let's now scroll down to something that's much longer to see if the pattern holds. Okay, so here we have something that takes, I guess, 13 arguments. Okay, so just handler's first generic parameter is M, and then all of these type parameters. And F implements handler if F is a function that takes 13 arguments and returns a future where the future's output is res, and where res implements inter-response. and where all of the types that are arguments to that function implement fromRequestParts. And here, too, you see the last one is special. The last one implements fromRequest and not fromRequestParts. We'll talk about that in a second, too. But see how the pattern holds. Like the handler trait is automatically implemented for functions of basically any argument length, as long as all of the arguments implement from request parts and the last one implements from request and the response implements into response. So let's do a sort of basic test here. Okay, here we have an asynchronous function. Okay, so does it return a where implements future? Yes, check. Does output implement inter-response? Well, let's see. What is inter-response? Inter-response. is just something that can be turned into a response where response here is HTTP response, and it's implemented for the unit type, okay? Whatever parts is, let's ignore that for now. It's also implemented for, and here we see this sort of macro magic again, where it's tuples of basically any length. So this trait is implemented for tuples of lots of different lengths where, all of the individual components of the tuple themselves implement inter-response or inter-response parts. And the last one implements inter-response. The separation between parts and the last one being different, we'll talk about in a second, I promise. And you see, there's a special case here for status code. So if status code is first, status code, I think does not implement inter-response or inter-response parts. Let's see. Yeah, so it does not implement it to response parts. So it's sort of handled specially where status code can be the first thing before the other arguments, but all of those have to implement into response parts or into response if they're the last one. Okay, so then the question becomes, well, here we only return one thing, which is the static reference to a string. So does static reference to a string implement... Implement interresponse? Yes, it does right here. So the interresponse trait is indeed implemented for that type. And if we look at it, it just is the same as a cow borrowed. Okay. And cow borrowed, we see the implementation for here. And it inserts into the headers that the content type should be text plain UTF-8, which is what we saw when we curled against that endpoint. So that's where this comes from. And full from, this is the body type. Full here is saying the entire body is available straight away. It's not like a streaming body or anything. And we turn that into a response. Great, so now we understand sort of where that response gets built up from. So does it implement response? Yes, it does, check. Does every argument except the last implement from request parts? Well, trivially check, because there are no arguments. Does the last argument, implement from request. Well, there is no argument, so yes. And so that's how this ends up working. This one ends up using the impl for handler when the argument list is empty. So right here, this implementation is what we use. And that implementation is just gonna call this function. That's all that implementation of the handler trait does when that is what the signature is. And then it's gonna call into response on the response here, which calls that function we just looked up with that implementation of into response that we just looked up, and that's how you get a response. Great. And there is, it's not actually very ad-rec generics here. Like it's not actually generic over functions of any argument length. It is macro expansion, right? There's the, all the tuples one that we saw. So it will only actually work for handlers with up to 16 arguments. If you have more than 16 arguments, it will not implement the handler trade automatically. Um, then you would need your own type that you implement handler for. It would be nice if this was genuine, sort of supported by the language directly, but for now, this is a trick where you don't really notice that. It's a trick. Um, oh yeah. So tuples also implement, uh, from request. So. This also means that any one of these can itself be a tuple. So you can actually construct larger ones if you're willing to do some nesting in there. Oops, that's not what I meant to do. Okay, so now we have looked at root, great. Let's now look at a more complicated one. So if we go back here, we do the same sort of checklist. Does it return a future? Yes, because it's an asynchronous function. Does the output implement into response? Well, this is a tuple where the first thing is status code. Okay, great. The second thing is append headers and the last thing is JSON user. Okay, append headers. Does that implement into response or into response parts? Right, so remember the rule here is that we need to that a tuple here implements into response. If the first thing is status code, the last thing implements into response and the middle things all implement into response parts. So does append headers implement into response parts? I'm gonna just tell you the answer is yes. Does the JSON type implement, oh, why did I scroll? Does the JSON type implement into response? Well, let's go look. So we look at the JSON type. No, not that one, but the response JSON type. Okay, so what does it implement? It implements inter-response if T implements serialize, which is what you would expect. That if you say that you return a JSON of some type, that thing is serializable so that it can be past the 30 JSON and produce the right thing. And so that might make you wonder, well, let's finish the analysis and then we'll do the wondering. Okay, so this response type does implement into response. Great. Does every argument except the last implement from request parts? Well, so that would mean that query needs to implement from request parts. Well, let's go look at query. So query is what's known as an extractor. We'll talk in a second about what extractors mean. It does indeed implement from request parts. If you want to look at it, I recommend going to look at this. The implementation of from request part for query is really just calling. Let's try from URI. It extracts the... um the URI from the request parts here is just like the request but split into its constituent parts so the URI from the query it extracts the query string so the things that follow question mark from there and then it passes that to surdy url encoded which decodes the stuff that comes after the question marks into a type using the deserialized trait right, deserialized owned here. And so this from request parts implementation of query just extracts a URI, extracts a query, deserializes that into a T, and then that's the thing that it extracted. So it does implement from request parts, great. And then the last part of our rule is does the last argument implement from request? So does JSON payload implement from request? Well, again, let's go look. So if we go back here, we go to JSON. It does implement from request as long as the type is deserialized. So that's fine. Let's look at the source here. Well, it checks the implementation of from request is the JSON content type set in the headers. Then read all the bytes out of the body. And then deserialize with serty JSON. And that's kind of it. Like that's all you really need to do in order to extract JSON from a request. And so at this point, we actually know what an extractor is. An extractor is something that implements from request parts or from request. And so it can take the parts of a request or the whole request and turn it into something that can appear in the argument list for a handler and have it still implement the overall handler trait. Okay. Hopefully that this journey we've been on so far makes sense. Now you hopefully see how the handler trait sort of fits together here. The last part that was really useful to me actually when I first found it is let's go look here at. This one. Okay, so this macro right here is what I would argue is the heart of Axiom. Let's go to like 2.23. I don't know why GitHub has started doing this, but it will only highlight lines if I change the URL, if I refresh. Okay, so the impl handler, trait is the thing that actually implements the handler trait for all of these different function types. So the argument list here is all of the ones that are in the square bracket. Square bracket is a repeated list of types. Remember, it's the list of arguments to the handler except for that last one. And then the last type, the type of the last argument is sort of kept separately in this last meta variable. variable. And so we implement handler for f, where f is an fn once, and this here is the macro expression for saying repeat all the types in the argument list. So it's implemented for, you know, fn once that takes two arguments, for example. These bits are fine. REST has to implement into response. All of the types have to implement from request parts, except for the last one that has implement from request. So this is really just the thing that generates the info blocks we looked at earlier. But then let's look at the actual call here. So what does it do? Well, this is the sort of equivalent of the service trait, right? Of we get a request in, it splits that request into parts and into parts is really it splits the body of the request from everything else about the request. So think like the method, the URI, any additional headers, all of that goes into parts and then the body is kept separately. And the reason it does this, and the whole reason why from request parts is separate from from request is because you can only read the body once. We don't want to keep the body like in reference counted memory or something. So that's why they sort of enforce this from request parts only gets access to the metadata of the request, the parts. And the last thing, the thing that gets to implement the from request trait, that one is also given access to the body. And so we'll actually see that this means that you can't swap these two. Uh, if I get rid of this and I import touch map. Ooh, yeah, up here. So it says, well, this is a relatively unhelpful error as you'll discover many of the errors when you use traits in this relatively advanced way are annoying, but it says the trait bound function, JSON and query implements handler is not satisfied. And then it tells you about some other random types that are satisfied. And. What it doesn't tell you, which is a little bit annoying, is that it is because in the arguments to create user, this implements from request, this implements from request parts, and that wouldn't work because when Axiom runs the handler, it's going to run all of these things, all the argument extractors first, and they don't have access to the body, and then it'll run the last one, which does have access to the body. But that means that JSON here is not given access to the body, so it would need to implement from request parts, but it doesn't because it needs access to the body, and therefore only implements from request. There is a thing that can make this a little bit nicer. So there's a, I think it's mentioned here. Yeah, here. So at the very bottom, there's this thing called debug handler. And debug handler is an attribute that you can add to your handler, and it will generally give you better errors about what went wrong. I don't know whether it'll work for this case, but let's see. Uh, use Axum debug handler. Uh, and I need the macros feature. See what it does. Uh, let's see if it can actually help us with this one. Jason consumes the request body and thus must be the last argument to the handler function. Great, so it gave us more information. It's basically, it's a procedural macro that looks for patterns in handlers that are commonly erroneous and tries to give you better errors than what you would get from the trait resolver, right? So by using that debug handler error, it actually gives us a better description of what went wrong. Um, and it correctly identifies that these need to be swapped. Uh, and now everything's happy again. Okay. So let's go back to the, nope, that's the wrong page. Um, let's go back to the code we were looking at over here. Um, Okay, so what does call do? Well, it splits the request into the metadata of the parts and the body. Let's ignore state for now. And then this here is a macro rule for repetition and it's repeating over $ty. So that's these, that's all the arguments except for the last argument. And for each one, it calls from request parts. passes in a mutable reference to the parts and the state, which we ignore, and assigns that to a variable called ty type. This is just a variable name because all types are valid variable names as well. It's a little bit sneaky. And if any of the from request parts return an error, then we turn that into a response and return that from the overall call. We have that be the HTTP response. So this is where you would get errors like if you tried to deserialize the query string into some struct and it doesn't have the right fields, then what you'll end up with is that that call to from request parts for query will fail with an error and it will be turned into a response right here. And as a result, no other parts will be exported from the response and the handler will not be called. So this just calls all of the from request parts for all the arguments except for the last. And then it reconstructs the request from whatever is left of the parts and the body that are originally extracted. And then it calls the from request trait method for that last type. So it actually gets access to the full request that we reconstructed here and not just to the parts. And afterwards, it then calls the handler function with all of the arguments that it extracted, all of these things that we called from request parts to, and that last arguments. So this is where the tower service call turns into a function call to the handler with all of the arguments in the right order with the right types. That turns into response. We just await it because we know it's the future. And that response we call into response to in order to turn it into the appropriate response at the end. So that's the entire flow. That's sort of the heart of how do you take requests and turn them into these handlers. And you'll see a similar thing if you look at into response. So if we go over to into response here. And we look at the implementation of response into response for whatever this one, for example, some random tuple that is in an axiom core response into response. Look at it on GitHub instead, maybe response into response. Where's my macro? Down here. 412, 490. So impl into response. This is again using the, all the tuples, no last special case. So that's the same as that all the tuples, except for responses, the last one doesn't get special cased. So, we implement up here, we implement inter-response for tuples of basically every length. And the way we do that is we take the response that we get out of the last one. So the last one is kind of special, right? That's the thing that gets to produce the body. So similarly to, from, for the arguments, the last thing gets to consume the body for the request. For the response, the last thing gets to produce the body for the response. So we call, we get the response from the last argument of the tuple. We call intoResponse on it, so that generates an actual response type instance. And then we construct the parts from there, and then we call intoResponseParts for all of the other arguments. and they all get to, like we pass in the parts and they return the parts. So that way they can do whatever modifications they might want to. And that's sort of, essentially you can think of it as we construct the parts with the body and we sort of pass it through all of the inter-response parts that are the other tuples in the type. And what we end up with is a response where all of the intermediate handlers, all the response modifiers have gotten to act. And that in turn turns into the actual response. Okay. So we can look at the others too. Like the others look basically the same. This is a special case for if status code is one of them. The special case for if one of them is HTTP parts. So parts just means you've already assembled a bunch of things that are going to go into the HTTP response anyway. Those just get stuffed in there, but those are not all that interesting. And you see here for the R here at the end, R has to implement inter-response, which is because again, that last argument is the one that gets to produce the body. And so we can see that here too, if we tried to say, let's do append headers. Let me bring that in here. I think I can do this. just want some type that it's happy with. Right, and then this can be append headers of Beck. So in this case, I'm not appending any headers, but here you would get a similar kind of error if I tried to do this, because append headers is, oh, and then this. See now this won't accept it again and I get this horrendous error. Let me do debug handler and let's see if debug handler can be helpful here too. See what it says. Debug handler wasn't helpful here. I'll just do debug handler and let's see what happens. We do get this error, which is somewhat helpful. The trait bound status code, JSON user append headers implements into response is not satisfied. The following other types implement into response and it gives you a bunch of those and 60 others. So that's not super helpful. But the actual reason for this error here is that, again, append headers can't produce a response. It can only produce parts of a response. It doesn't produce a body. The thing that produces the body is the thing that goes last. And I think, you know, you might think maybe it should go first, but I think it truly is to match the order of the arguments in the argument list where the body type has to go last. So would they do the same for the response? I don't know whether I agree with this, but that's neither here nor there. And the reason why it's questionable to my mind is because all of these get applied in order. If we look at the expanded macro over here, you see that all of the type parameters that you pass in, they all get to be applied to the parts in the order that they appear. So the pass through here happens left to right, except for the body thing, which is at the end, but actually is applied first. So the ordering gets a little weird, but I think it's for symmetry with the argument list. Okay. So now we have the stuff for like how handlers get called, how this all turns into a service, like how routing works, what the service trait is. And so we're getting pretty close to understanding the sort of whole. Belgian waffle. I don't know the whole the whole cake. But there's at least one bit left. But before I go into the last main bit I want to talk about, let's do questions because I've been talking for a long time. And I'm sure this has a bunch of you going. I did not follow half of that and I'm lost. The body is also generally the last thing actually written over the wire, so maybe mimicking that. It's true. Like, I think there are arguments for keeping it last. The reason it's weird to me is because... that type gets invoked first to produce the response type. And then the others see the response after it produced it. So there's like a, the order in terms of time in constructing the response is last, then second, then third, and then fourth and fifth, which is a weird order to me. Um, the idea, the idea is that response tuples is that you shouldn't be able to accidentally overwrite the response. Yeah. You only get to overwrite parts of the response, but not the entire body, for example. Um, should I use tuple as a return type for all the handlers or use something else like implement response if it works that way? Yeah. So when you write the response type here, you can either give the concrete type or you can just do into response. This is what I usually use. Oh, right, and then I need to undo the mistake I made, the intentional mistake I made, but nonetheless. Ooh, it didn't like that. Let's ignore that for now. It's not important. Yeah, so you can just use impl into response here. And that way, if you decide to add another piece in here, it just kind of works. The main thing to be aware of here is even if you write impl into response, which is what I almost always do, you're still under the restriction that you can only have one return type. So you couldn't do something like, if rand is four, then return status code not found. Like this won't work and let rand is six. And if rand is four, because even though you used impl into response, Rust still requires that there's a single return type for the entire function. And so it's complaining here and you'll see those better in the actual output. It says mismatch type. It expected the return type to be status code, but here you're trying to return a tuple, which is not the same as status code. So you don't get to do this, even though you tried to say, I only returned something that's inter-response. So if you have a type that has to diverge like this, what you do instead is you do this, and then you do .interResponse here, and you do .interResponse here. Um, uh, I don't, I take that back. I want this to be response response. So here I'm saying, I'm just gonna return the entire response. And the way I'm gonna do that is by calling interresponse myself. Because that way there's only now a single type for the entire return type, but I get to still diverge internally. And response obviously implements interresponse. Oh, I see the reason why status code, we're very lucky we have David, who's basically the author of axum in the chat. So I'm getting a lot of useful insights there. So the reason why status code has to be first in all the tuples is precisely to avoid you accidentally setting it twice. So that if you set it here, you sort of know that that's gonna be the one that's used everywhere else, that's used for the actual response. What about having variadic types in Rust to not use these macros? Variadic argument lists are really hard to get right and to design into the language. So I wouldn't hold my breath for getting it for probably years. And I don't think we should wait until we get them before building things like this because they're useful. So that would be nice, but it's a much longer forward-looking solution. Okay, so the last thing I want to talk about, I think, is state. So I want to start by going down here for sharing state with handlers. So there's an extractor. And again, remember, extractors are just things that implement from request parts. That's the only requirement for something to be an extractor, is that it can extract things from either a full request, including the body, or from the parts of a request. There's an extractor called state which works a little bit differently. So when you want to have shared state between your handlers, think something like a pool of database connections or even just a data structure behind a mutex or whatever it might be. Or a configuration, anything you might want to share, here represented by app state. You construct the shared state. You start the router, you call route, and you call .withState. So this basically hands that state into the router so that it is going to pass it to all the handlers. And we'll look at how that works because it's actually really interesting. Once you call .withState like this, you're able to now in your handlers declare in your argument list, I want state and that is one of these. um and so let's go look at what that actually looks like so let's say for create user here we also wanted um i can't use that can i um arc um they just you yeah fine So I can do this and then I can appear define, you know, struct app state and it would normally have fields, but it's not gonna have them right now. And then up here for my route, I'm gonna call dot with state arc new app state, whatever fields that might have. So that's all it really takes. Now, every handler here, so I could do the same thing up here for root now. So I could say root here is also going to get access to state. And in practice, what that means is they're going to get separate clones of the state. And so that's why the arc is here. Because if you have multiple handlers, like those handlers might be called in parallel. You might have multiple handlers that all want access to the state. And so it needs to be cloned in order to be shared. You can imagine using like a static reference or something. But this arc is an easy way to get something that's trivially clonable in order to share them. So the requirement is that for anything you pass in with state like this, it has to implement clone. So it's pretty common to see arc immediately following state. And so now, you know, inside of state, this can access state and so can this one. And they are referencing the same state because if you clone an arc, you get a reference to the same thing. But how does this work? Like so far when we say from request parts or from request, it's all been about things that are in the request. How are they getting access to this state thing? Well. I'm going to ignore things like closure captures. You can read that on your own time if you want to. State is the most common way to go about this. The state extractor also implements from request parts. If we go look carefully at from request parts, this is an async trait, that's why it has a little bit of a weird signature here. So let's look at the source instead. From request parts is generic over S, S for state, we'll see why in a second. And the from request parts function does get parts, which is the parts of the HTTP request, but it also gets a reference to the S, to the state. So this is how the state extractor can be constructed from request parts because from request parts also gets access to that state. That seems easy enough, but this generic is scary. Isn't this going to end up everywhere? And you see there's some weird distinction here between outer state and inner state. That one is actually just because if... Imagine that there's like... Here I want like root state and here I want create user state. So I sort of want them to get specialized states, but root state and create user state are both like part of the app state. Um, so this has, I don't know, root just root state and, um, create user state. Imagine that this was the structure I had, um, this, this indirection here of having two generics. And then you require that the interstate, which is what you actually need, uh, implements from ref to the outer state. So. from request parts is past an outer state, here app state. And as long as you can construct an inner state from a reference to the outer state, you're fine, right? Then you can call the handler with the state that it wants. Like you can call root with root state by extracting the root state from the reference to the app state. But this distinction, we won't have to think too much about. But the implementation here is implement from request parts of state for. State of S, great. So there's just a requirement here that the moment down here where we say, let's go back to having this be appState for simplicity. When I say I want a state of this, what that actually means is that this handler now only implements, remember that these functions only implement handler if the arguments implement from request parts, but state only implements from request parts for this generic state. So that only this doesn't implement, doesn't implement from request parts for any S, it only, implements from request parts S or app state or arc app state really. So this now means that the implementation of handler for root is conditional on S being equal to arc app state. If there's any other S here, then root would not implement handler for that S. So if we go now back to handler, the handler trait is also generic over S. T is the argument list and S is the state. So what this really means is impl S handler, you know, T S for root. This doesn't exist. There's no generic implementation of handler. Root only implements this. Right, this is the only version of handler that root implements. So what does that mean? Well, remember how forget get required that the handler implements handler for any s. Okay, great. So this is generic over the s. So you can pass in to get, you can tell get what state the handler is going to require, but inference works in both ways. So what actually happens here, and this is, we're getting into a little bit of like nuances of the trait resolver, but basically, When I call get root here, what really happens is the compiler starts to infer what these types are, what these generic types to what? H, T, S, and B. So H is the handler. So that's going to be equal to root. That's because that's just what's passed in. So that's easy. This is the type of the argument. Like it's dictated by the argument we gave it. T is gonna be the argument list, which for root is the empty tuple. It takes no arguments. So that's also easy. B is the body, which is a box body for everything in axiom. So it's a boxed body, which might be a streaming one or an already allocated like full string or byte slice. But the S isn't tied to anything. So what actually happens here is this sort of an inverse inference where the compiler knows that root only implements handler if s is arc app state. And therefore, it infers that s must be, so it's not equal to by virtue of being called, it must be arc app state. Because that's the only way that this will actually work. And route, so now we go back to router. Router also has an s. Okay, so router is generic over some s. And when you call route, that's a method router over s, and that s is tied to the s in get, so the inference goes all the way back to infer that this actually means that this router, which is S and B. Well, B is box body. That's fine. But the S here is inferred that it must be app state. So the inference goes back, forget. And then for post, it then goes sort of forward that all of these S's have to be the same. So for post here, They must also be the same. And here is not inference. Here is just, it must be equal to that. Well, so now let's see what happens if we just, let me get rid of, oh, do I want to get rid of these? Yeah, that's fine. I'll get rid of these. No, I don't want to get rid of them. I take it back. So what does that mean? Well, that means that at the end of this call chain of calling route, what we have is a router where S is Arc App State and B is Box Body. And we know that in order to pass this application, oops, in order to pass this application into serve, it needs to implement tower service. Technically a service service, but ignore that for a second. Well, If you look here very carefully, we implement Tower Service for a request over B. for router of unit and b. So the service trait is only implemented for router if the state is empty. Well, now we have a problem because the inference here told us that at this point, s is equal to arc app state because that was inferred from the fact that we pass in root here. That is the only s that works. But in order to be able to call serve down here, the sort of S inside of app must be equal to unit. But our gap state and unit are not the same thing. So how do we bridge those? Well, that's where with state comes into play. So with state, takes in a state and returns you a router with a different state parameter. And in our case, we make use of that by saying with state, and then we pass in the app state. And we can just say what state is required next. We can just say with state, this is now unit. And in fact, that's inferred for us because S must be unit, otherwise this call wouldn't work. So the way to think about the S on router is not that it's the type that you've passed into the router, it is the type that the router is going to require before it can be used as a service. Or in other words, a router with some S that's not unit needs to be provided with that state S before you can use it for anything useful. Right? So, which makes sense. Like, root, we can't use root as a handler until we have provided the state that it's going to eventually see. And so, hence this sort of slightly weird but really neat trait trick that makes it so that it's just impossible to construct a router and try to use it to serve requests if you haven't passed in the necessary state. But this should raise another question mark for you, which is, okay, we've done all this business. Let me get rid of all of my extra annotations here. Okay. Um, so down here is equal to unit. So how does with state work? How does it accomplish what it does? Because root requires the state. Like it, it needs to be pat. That state needs to make it in the, into the request somewhere. Right. When we looked at, um, this from request here. You see, state is equal to a reference to state, and state is just passed into the call in handler. But if we now go back all the way to tower service, which remember is the trait that actually drives all of this. In call for tower service, the only thing you have is a request. But in call for handler, in fact, let me put a handler here, might be easier. So call for service, the only thing you get is the request. Call for handler, what you get is the request and the state. Where does that state come from? How does the state make it from the router and into the handlers when they actually get called? Well, we actually found this earlier. So see how on handler there's call, there's layer for stacking them, and then there's this innocent looking function withState. And what withState does is it takes a handler, anything that implements the handler trait, and it gives you a handler service. So you provide it with the state and it gives you a service. And that kind of makes sense. And let me show you how that actually works. So a handler service holds the handler and the state. And then it implements, where's the implementation here? Implements, this is tower service now. So it implements tower service request for handler service, where H is handler. And the call here is handler call. Pass in the handler. So this is basically the same as handler.call, right? Pass in the request and clone the state out of ourselves and pass that in. So that's how the state actually makes it in, is that in order to turn a handler into a service, you have to pass in the state that it needed. At that point, it no longer needs the state. You don't need to pass the state in with every call because that's handled automatically in this impl of the type. And so that's what happens for you in router. If we go back here to router. So when you have a router and you call with state at the end there to make it so that it no longer needs its state, well, router internally has like path routers and fallback routers and whatnot. But if we find the path router with state, let me date that up real quick. Path router. Where's my path router at? Over here. That maps into method router. And method router with state. Ah, that's a With state. Uh, sorry, let me dig this one up real quick. Right. So method endpoint here, we ran really deep into the stack, but basically this is the thing that get produces. When you call get, when you call the little get router function, and you pass in root, you pass in a handler, what it does is it boxes. In fact, let's look at it. Why not? We have nothing to hide. We're here to learn. We will do so relentlessly. So what does get do? Well, all of these do basically the same thing. Where is my top level handler fn? Top level handler fn, where you at? Here. so many layers of indirection. I understand why they're there. It just makes my job harder. Okay, I'm not going to show you then. I take it back. But what happens is when you call get, it takes the handler that you get in, it boxes it so that it can store them all together. And then it produces one of these method filters. So this is the thing that we looked at earlier where it implements service. And depending on the method of the incoming request, it chooses which handler to actually call. In the case of the get constructor, it's going to do the same thing. It will call that handler if and only if the incoming method for the request was get. And the individual thing that it stores is one of these method endpoints. And the thing that it stores in there is either none, if there is no handler for like post, for example, then the handler is none and so it does nothing. Let's ignore route for now. The default state of this type is this boxed handler. So that is, You know, when you call get root, it takes root, it boxes it, and it constructs a method endpoint boxed handler, and it sticks that in there. When you call withState, if it discovers that what it currently has is a boxed handler, then it knows that that handler still hasn't been passed its state because it implements the handler trait, not the service trait. And so when you call withState, like on router, it ends up calling withState all the way down into all of the handlers, including down to this little method endpoint, which then calls intoRoute on handler. which then gets us to into route, which we're almost at the end of the journey, I promise. So into route over here. Oh. uh where is into route again it's like over here somewhere no not that one it's no it is in boxed it is over here uh right here into route i was trying to find the actual call to it Oh, she I'm gonna I'm gonna skip one step. I apologize for skipping it. But When it finds that what it currently has is a boxed handler and it is provided with the state, it calls this intoRoute thing, which indirectly ends up calling this fromHandler function for boxed intoRoute. And what that does is it takes the handler, passes in the state. So this is how we go from a handler to a handler service. And it creates a route, which is really just a boxed service out of that. and turns that into an erased handler. And an erased handler all the way up here is a handler that you've already turned into a service. So it already has all of its state. And when you do that, it gets turned away from being a method endpoint boxed handler into being method endpoint route. So route is a thing that doesn't need its state anymore because already been provided it. I know this part is convoluted and hairy and hard to follow But it's a little bit important, which is that the moment you provide the state to a handler It turns into a service That's the key part to take away from this. So over here, the moment we've called withState now, now router is a service. And so we can actually, after withState, we can now call route again with something that requires some other kind of state. So let's in fact do this because it might actually be useful to demonstrate. So let's say I have root one and I have root two. And root 2 takes a completely different state. It actually takes a mutex over a bool. Actually, it just takes an arc bool, why not? So these take different states. So how do we make this work? Well, the way that you make that work is, you construct all the routes that require one of the states, and then you call withState with that state that it needed. And so now all of those handlers have been turned into services that internally store the state that they needed. And so now if we call, if we create a new route that requires a different state, then now we can pass in that state over here, and now, These things had already been turned into services. So they don't see this new state. They're unaffected by it. This route, this handler, which doesn't have its state yet, is provided with its state. And so now it's turned into a service and doesn't need its state anymore. And you can keep doing this. So you could keep adding more and more states if you really wanted to and have different routes that require different states. And this also means you get into really interesting things. Like you can imagine that you have handlers that require different states and you actually say, well, this with state only provides like half of the state. So it provides, actually, I'm not going to talk about it. Unnecessarily complicated. But this piece is important to understand, which is that once handlers are provided with their state, they stop being handlers and start being self- standing services that do not need their state anymore because they already have it internally cloned. Okay. I know that's a rabbit hole, but I think the conclusion from it is useful. And hopefully if you rewind and watch it again, then it'll eventually make sense. It took me a few goes to piece this part together as well. One of the things that hopefully you find useful from this is to see the value in sort of jumping through the call chains and seeing where do the types change from one to another? Where do things implement? different traits, look at the source of those implementations and figure out how that worked. So hopefully, this was useful, even if the specific conclusions from it are somewhat inscrutable. Okay, let's do more questions now, because hopefully and presumably you have some from this. Oh, Yeah, so one thing that's worth pointing out here is that even for the people who built this, this was not an obvious design from the start, right? Like this is convoluted, but it's not as though some people are just so smart, they come up with this like immediately, and it's just perfect. Quite to the contrary, what happens is you start with something that is, you know, not as strict as this, not as complicated as this. And then you discover, for example, that, hmm, there's a foot gun here, right? Like it's possible to put in a route and forget to provide the state for that route, but still try to start serving requests from it. So now you get runtime exceptions because you didn't pass in the state. That's bad. How can we solve this? And you iterate on the design until you get to something where you can guarantee it at compile time. It's not as though this like fell out complete in the beginning. Rather, it took a lot of iteration to get to the point where you have something that is enforced at compile time to this extent. The other thing I want to point out here is that I didn't dwell on this too much, but there's a lot of boxing happening here. There's a lot of trait objects. There's a lot of trait erasure and generic erasure into boxes, which does impact performance. But in practice, I just don't think it meaningfully slows things down here. Because once you have a web server, your bottleneck isn't dynamic dispatch. Right, your bottleneck is stuff like IO or talking to your database. So if you can make the ergonomics of using the tool nicer by doing a box somewhere, it's probably worthwhile. Yeah, box.rs is, I don't know if it's the most complex file, but it's probably the most cursed file. Let me go just pull up boxed here, cause it is, it is fun boxed. There's a box is the thing that does all of this boxing things and providing the state and turning things from handlers into tower services and does like erasure of generic types. There's a lot of just curse voodoo in here that only you only really see how cursed it is when you start seeing how it ends up being used throughout the code base and how all these state transitions happen for the types. But it is really interesting. Um, uh, can you talk about multi-part send in axiom? Um, I think multi-part send is really just a, a body type that you can construct in parts. Um, and yeah, so it uses, there's another library called, called molt, molter. Yeah, which lets you produce multi-part types by essentially providing the constituent parts separately. And then when it tries to produce an HTTP response, it streams it out with the appropriate encoding. Similarly for, like, we didn't really talk about this at all, but here you can do things like. colon me and now this is actually a string argument in the path that can be extracted and can be any value and the router has to deal with that but that's actually not implemented in axiom itself axiom has to deal a little bit with it but it's actually implemented through a library called matchit that i mentioned at the beginning which does efficient routing of from like strings with these parameterized parts to it into like, what is it it constructs? It constructs like a prefix tree, like a Radix, or it's a Radix try that lets you efficiently do like string matching of real URLs against the patterns and then find the appropriate value in a map that's actually a try. Really neat data structure too, that Axiom is using below the hood or under the hood in order to do its mapping an incoming request into the appropriate path handler. And you can see those parts in routing path router. So the whole thing in path rounder is the sort of key part of it here is this node thing. which, let me see if I can find it. Here, the trick is always you just look for the implementation of service. Maybe that was the implementation service, actually, now that I think about it. No, I can't find it. Route, validate path. No, route is for creating one. Where is... Is it... Is it... Ah, call with state. Um, so it extracts the URI from the request, it extracts the path and they call self dot node dot at a node. Here's one of those match it things. And what that gives it back is the. Handler associated with the path pattern that matched that path. So all of the actual like string searching happens inside the match it library. And then this really, you can access almost like a hash map, except that the lookups are. are fuzzy, if you will, or parameterized, or are searches, rather than direct match lookups. Are there any somewhat near future Rust features that will obviously serve Axiom well? I think type alias impulstrate will probably help a little bit. There are a couple of places currently where Axiom has to box, but only because we don't have type alias impulstrate. David can hopefully confirm that, but that should be the case. I don't know if there are any others that are likely to come anytime soon. Oh, of course, asynchronous traits, which are closely related to type alias imple trait. But currently, there are a bunch of places in here that use async trait. So if we looked at what was the one we pulled up handler? No, not handler. Home requests. Yeah, so if you look at from request, for example, you see that it actually looks really weird. So it has all these lifetime traits and it has like pin box and where tick life zero implements async trait and stuff. That's all because under the hood, it's using the async trait annotation on that trait, which means anyone who implements it also needs to use async trait. And so being able to just have asynchronous functions and traits directly as part of the language would let them get rid of that as a sort of pack, which also then would reduce boxing, right? So one of the things that async trait does in order to do its job is that it does everything through boxing and dynamic dispatch, which you would no longer need if you had true async event and traits. Does the app state gets copied everywhere? Yeah, so when you use state, I mentioned this briefly, but it might've been lost in the noise. The state that you take into handlers, that the inner type here must implement clone. It does not have to implement copy, but it does have to implement clone. And it ends up being cloned once for each handler. I don't think it gets cloned for every request or for every connection. It just gets cloned for each handler. And then it ends up in handler service. All right, so remember we looked at this. So a handler service holds the handler and the state. And then when anything gets, when a request gets passed in, it just takes, oh, I guess it does clone the state actually. It clones the state for every request. That makes me sad. I wish this could be a reference, but it probably can't be because then you couldn't have multiple requests in parallel. although technically you could, but you could never request that outlives the handler service. So the reason why it has to clone the state forever request here is because if you look at the tower service trait, the future that you return from call does not have a lifetime bound. So it has to be able to live independently of the borrow of self. So the future you return cannot borrow self. And therefore you can't return a reference to the state because that's contained in self. So that's a clone for every request. That does mean it's only cloning the arc. It's not cloning the actual contents of that arc. But it does mean that you're like potentially contending on a cash line for the reference count for every request. Is that going to be your main bottleneck? Unlikely possible at immense scale. At that point, you would probably use like a sharded arc or something instead. But it does make me sad. I wish there was a nicer way to do this. One of the ways you could do this, because you know that the state is going to live for the entirety of your service, and this is one of those where everyone's going to get mad at me, including probably David, which is instead of using arc here, you do static. And here too, you do static. And then up here, instead of arc new, you do box leak box new. Because. When you create a box, when you leak a box, you get back a static reference. So I think I can even do static here. Or I can do this, I suppose. And so that way, what box leak does is it does, it heap allocates the thing you pass it, and then it leaks it. So it basically never frees that memory, which means you can get a static reference to it. And references are clone and it's static because it's a static reference. So it will satisfy the tower service bound for the return future. So that's the other way you could do this. And then the clone is free. There's no reference counting. The risk, of course, with using leaked memory like this, instead of using reference counting, is that the leaked memory is going to stay leaked for the entirety of the program. It will never be dropped. The destructor will not be run. and that memory will never be reclaimed. Now, that's not a problem if the entirety of your program is running the server, right? Then it's fine for that state to just be leaked because it's just a single instance of the state. So it's not like it'll keep ballooning over time and cloning that static reference won't make a difference. That's just copying a pointer. It doesn't allocate any more memory. So it's just really that one allocation that gets leaked and lives for the remainder of your program. But your server is running for the remainder of the program anyway, so it would stay around regardless. So you're not really losing anything by leaking. Now, the performance benefit that you get is this only really matters if you're running in highly concurrent environments. But cloning an arc, it requires incrementing a reference count, which is on a shared cache line across all of the users or all of the reference counted pointers to that one memory location. all of them are going to share a cash line for the counter, which means that if you have lots and lots of clones all happening at the same time, they're going to contend on that cash line, which means they're going to slow all of them down. And so you might start to see a performance collapse if you have a lot of handlers that you state. Because all of those handlers, every time they're invoked for every request, are going to try to increment the counter on that cash line and decrement it again. And so you end up with a lot of contention and contention is... pretty sad when you get to highly concurrent states. So you have like lots of cores, lots of concurrent requests, and you're using the state all over the place. This might actually be a sort of lifesaver for you. If that's not you, then just use an arc and it's fine. Okay. Yeah, so this is very much one of those like measure first, like don't start to leak memory on purpose until you actually measure and discover that you truly need it. Because it does make your application like more painful to work with, it makes testing more annoying because now every time you start a test, you might leak memory. And suddenly the lifetime of your program is actually quite long because it's every test spins up one of these and it remains until all of your tests is completed. So think carefully about this. Don't do this lightly, but it is a trick that you can pull if it turns out to be actually important. Couldn't you put it back into a box after you free your server if you cared enough? I would hope all references are destroyed after the server is dropped. This one's tricky. So the proposal here is basically, well, I could do... Uh, this, uh, and then down here I do state and then down here, I do sort of, I reconstruct the box by saying I would use, have to use from raw of state, uh, technically in order for this to work. Um, I think you would want to. I actually think you have to be a little sneaky here. You need some unsafe because of provenance, which we're not gonna dig too much into here, but you could do this, and then unsafe this, and then from raw of this. And so that way, after your server exits, you are actually gonna drop, and then we can just change this into this. The reason you need to do this is that if you instead did this, and you say this is going to be a static of this, and then get rid of the unsafe, and then you try to do this as star and mute this, that's an invalid cast. So you would have to do this and cast it. And this would be app state. So the compiler doesn't yell at you for this, like this compiles, but from a provenance point of view, a point of provenance point of view, this is actually incorrect. Because what you're doing is you're taking a shared reference and casting it to a mutable reference, and that is disallowed in all of Rust. You are not allowed to do it. So this version of the code, even though it compiles, the unsafe here, the unsafe assertion is incorrect. This code is not actually safe. You have to do it this way, I believe. And if you do it this way, I think it's OK. Where the state that you pass back never leaves as a mutable pointer. But you can do this. You can recover the leak. It's just, do you want to do this? Unclear. Do-do-do. Will this be better if the Tower API included lifetime bounds? So there is an argument that the service trait should move to use GATs, like generic associated types. If they do, then call could be generic over the lifetime to self, and the future could have a lifetime argument that gets associated with the future. We might end up with that. The downside by doing it that way is you can no longer handle concurrent requests. Because when you do a call, the future you get back is now tied to the lifetime of the mutable borrow of self, which means that you can't call call again until that future is resolved. In other words, you can only handle one request at a time. And I don't know if that's a... requirement we want to add, or maybe the trait needs to expand to also allow for things like concurrent calls. We'll have to see. Maybe call turns into taking a shared reference to self. That's one way to get around it. Do you recommend using Axiom for serving ML models? The issue I'm thinking about here is having a message queue in the backend to reserve requests if model inference takes a lot of time. That should be fine. You might end up using something like Tokyo blocking in order to say that the generation part actually is going to be kind of slow so that you're not blocking handling of other requests, but that's not inherently a problem. Oh yeah, the other question for the unsafety here is whether, do you actually know that once the server has exited, that nothing is keeping that reference to state? Because it's really easy for them once they have a static reference to the state to just store it somewhere, somewhere else static. And now this unsafe invariant is no longer true. So this is pretty risky. You need to, it's like a global invariant over your code that you need to make sure you don't violate. Why not also make service clones so that every request has its own service? That feels unnecessary. Like you don't really want to add clone bounds unless you need to, right? Because sometimes it is actually useful to have the call mutate the state for things like keeping track of how many requests have come through. You don't really want to have to clone it for every time. You would lose the ability to do something like that. Um, uh, why is actics web faster than axiom? Is it? I very rarely believe these, these benchmarks because they're usually micro benchmarks and micro benchmarks. Don't really represent a real workload. Um, At the same time, it's not as though I'm opposed to using Actix webs or something either. I like the way that Axiom does things because it integrates nice with other ecosystems. And I like this way to write code. I think there is more Rust complexity to it, but in some sense, that's what the language is for. So it eliminates more foot guns at compile time, which does add some complexity and cognitive overhead. But I also think it's better. All right. I think that's all I really wanted to cover for Axiom. Do we have other questions? Anything else that people want to know before we say goodbye? Let's go stick this back here so I get rid of my unsafe. Um, you seem to use the, use the term Providence a lot. Can anyone in chat or you briefly explain what Providence is? It's very hard to briefly explain, explain Providence. I would look up, uh, Ralph Young's introduction to Providence. Uh, it it's really good at giving examples for this. Um, Can you talk about the from ref trait? Yeah, so from ref is not really all that magical. Oh, from ref. So from ref is mainly used in things like states. So remember we looked at this. So state implements from request parts, but it implemented via this from ref trait. And the idea here is that if you have an imple of from ref, which direction is from ref go outer state. So app state for specific state. If you have this imple, then now you can write an async, a root handler that takes a state S, state root specific state. You can write a handler like this and you can still call router with state with an app state. So you might wonder, well, why is that useful? This can be useful in things like libraries where your library needs to have access to like a database connection, for example. So there's like some, like, I don't know, delete user or something. Like your library provides a route and that route requires that you have like a DB connection. or whatever it is, some connection type. But the application that you have written, the larger application that includes that library and includes the delete user route from the library, you don't want to have the requirement that the state that you pass around is just connection because you have all sorts of other state you want to include there too. And so the way that you do this is you implement from ref app state for connection. And so that way, you basically write how you can construct a state connection from a state app state. So it's a way to have a level of indirection between the state that any given route needs and the overall state that you provide. The fallback router is just... If you have a fallback router, what that means is if the main router fails to find a way to route the request, the fallback router is invoked instead. Is action production ready? Yes. I think so. I don't have a reason to believe that it's not. Um, custom HTTP method handling. Um, I think for method filter, um, no method filter is not what I want. I want, uh, on. Oh, I guess this actually does specifically only handle these. That's fine. So what you would end up doing is instead of using... Come on. instead of using the method routing router from Axiom, which only supports the sort of standard HTTP verbs, right? And so it has a pretty hard-coded list of mappings from the known verbs to other things. What you would do instead is you would just have your own type that you implement that has a service. Wow. Where's the Impul? I just want the Impul. I just want the Impul. Here. And you would write an Impul like this. Impul, service request, request body type for, and your type goes here. And then in call, You know, the call with state here for the method handler is really just match on the method, like just figure out which method was in the request. So you would do the same in your type in the implementation of call. You would look at what is the method for the request and rather than try to turn it into one of the well-known HTTP verbs, you just check whether the method is your own, the HTTP verb you're looking for. And if so, call the handler. So you wouldn't do it through the normal method router. You just write a tower service. Let's see. Using think asynchronous traits will stabilize this year. I hope so, but my belief is fairly low. Why doesn't Rust have a way to async drop? Because async drop is really hard. Okay. In that case, I, oh, there's one last thing I want to mention, which is Axiom extra. So there's a crate called Axiom extra that has a bunch of extra utilities for Axiom, uh, things like handling cookies or, uh, Jason lines. So the format where you have multiple Jason object on separate new, uh, like new line, separated Jason objects. Um, and protobof and stuff. So these aren't in the standard Axiom crate because they have to take a bunch of extra dependencies. And so you want to keep them separate so that the dependencies of Axiom itself is smaller. And some of them I think are more experimental or the versioning of the underlying crates would mean that we would have to bump the version more often. I say we, but I mean David. So this is a good crate to know about if there's some extractor, for example, that you really want to have and you don't see it in Axiom, it might be in Axiom extra instead. They have like other routers as well, I think, for query parameter based routing. I think there's also other response types. Yeah, like CSS specifically so that it sets the content type, that kind of stuff. All right. I think... Oh, there's also axiom server for HTTP support. Axiom server. Interesting. I don't know how this differs. This is also by a different author. I don't think you need this. I think you can just do it through, like it's already through Hyper in, it's already through Hyper in Axiom itself. Alright, let's call it there. Thank you all for coming. I hope that was interesting. Annoying that it got split in the middle, but what I'll do is I'll, I guess, stitch this together. So if you were watching the video on demand, you don't realize what I'm talking about, and that's fine. It'll be a treat for the people who are here live. Alright, I will see you all next time. I don't know what we're going to do next, but we will do another stream at some point in the future, and it will be fun and interesting. Alright, bye all.

Channel: Jon Gjengset

Views: 64,197

Keywords: rust, live-coding, axum, web framework, crate, explainer

Id: Wnb_n5YktO8

Length: 132min 26sec (7946 seconds)

Published: Fri Jul 28 2023