Tiny, fast ASP.NET Core APIs with native AOT | .NET Conf 2023

[Music] thanks very much well here we are NETCOM second day uh thanks to everyone who's been watching so far I'm pretty excited to talk to you about uh Native aot with as. net core it's one of the things that we worked on kind of for the first half of the release during this year um and then we may have been working on something that we announced yesterday for the second half of the year but um I'm really looking forward to talking to you about Native aot today so uh tiny FAS hore apis with Native aot what is that all about exactly so why native aot what is native aot all about well it's basically these three things you want to get smaller apps you want to get faster startup and you want to use less memory why would you want to do those things well for as. netcore at least for this release the primary focus was around enabling more Cloud native type of scenarios we heard from customers who were deploying aset core apis maybe grpc apis or minimal apis htdp apis and they were deploying them into containers and they wanted to understand uh some of the behaviors that they were seeing like why does it take this amount of time to start up why is it using the amount of memory that it's using um and why is it the size that it might be um and so especially when they started comparing uh to other Stacks which we can't avoid you know we exist in in an ecosystem and people are always going to uh compare as.net core to to to other potential Solutions and we obviously welcome that um we we started going okay this is obviously something that we probably need to look at here what is it that we can do to try and improve each of these three things how can we make as.net apis uh smaller uh start up more quickly and uh use less memory so let's go to code because you know I do have slides we'll go back to the slides a few times but I like to explain a lot of these things with code I'm just going to create an empty net core application here uh we'll call it uh where are we at netc aren't we so let's call it netc uh 23 and as I said we're just going to start out with I'm going to untick htps I don't need that uh for this scenario so I'm just going to start out with the most simple aset core application I possibly can Okay so let's open up our program.cs here and what do we see well we see our familiar uh web application you know cre Builder we don't have any Services CU this is just doing hello world and then we go ahead and run the application all right well I'm going to go over to the command line now and I'm going to have a look at what happens when we interact with this uh application that I just created um from the command line with regards to publishing so I called at netcon if I have a look inside here I've got my netcon project so here we are and I'm going to go ahead and publish that so uh remember publishing an application using the publish command is supposed to set up the application ready for deployment to some type of uh production location you can see here I've got a message that says it produced a couple of a dll over here in the U during the build phase and then it published the output of the application uh to the publish location so if I go ahead and look at that right now I'm getting nice completion over here oh except I I wrote it wrong that's actually not where I want to go for this one so I want to go over to I've beenin uh doing release by default for publishing now uh net 8 and we'll go to the publisher output okay so I can see here what files I got I've got my uh dll which is my application which is actually really really small it's only 5 kilobytes and say well this is a good reminder that by default when you publish an application we do what's what is called a framework dependent deployment or FDD you might see that acronym show up in documents and so this application requires a runtime to land on you can't just take this application and run it on any machine it is framework dependent um now it does have an executable here so that is what we call uh effectively an app host so you can run that executable and your application will run but all this is really doing is locating the dll next to it by of the same name and then it still needs a a framework to to land on and so that's one of the reasons why this is so small there's no asp.net core in this dll there's no net core in this or net in this in this dll it's really just your application pieces which is so small um what we find though is in a lot of scenarios people are wanting to deploy uh the whole application not just their code and their dependencies but they also want to deploy um something that's Standalone or self-contained and we have a different deployment mode for that so if I go back to my project I'm going to go into my project file and I'm going to add a property called publish self-contained and I'm going to say true and so now when I go back and I run publish again we're going to see a slightly different Behavior this time um so that was very quick I'm going to pass the self-contained flag and type it correctly there we go and so oh what we got here unable to resolve for a package sources enable oh I must have a feed that I don't want to be using on this machine so let's have a look at our feeds here uh that's fine that's fine I'm just going to check my Global package sources here I might have a leftover uh package Source from setting up a few things all looks fine over here that looks correct that looks correct let me go and quickly have a look in the location that I'm executing in here let's have a look no I'm not seeing anything that's a strange issue for me to be seeing okay let's make sure package was enabled were not considered why were they not considered unable to resolve for. net package sources let's just try that again I'm going to try restore and I'm going to force it just here okay it all looks fine I'm going to try a build here that all looks fine let's do a publish and we'll do a self-contained it's still giving me that error that is unusual okay let me go back and we'll do a clear here just to make sure that I don't have anything left around that I uh want to so we uh what do I want to do I want to we're going to go ahead and uh you know what I'll just do this manually I'm going to go in here I'm just going to clear my new get cache I think I may have something latent left over unfortunately I did have to change machines a little bit late before while prep preparing for this demo so let's just see we can't unblock myself here okay well that's going on on just want to make sure I'm really not seeing a new get config here let's have a look and if we can't get this fixed in the next second I will just hop back to the slides and we'll talk through what we're going to look at next all right what are we seeing over here now all right that's complete let's try that one more time then we'll go back into our uh app that we created which was netc we'll have a look inside here all right let's do a clean here just to make sure we're in a good State okay and then we'll do a publish let's make sure our regular publish works okay that seems fine let's do our selfcontained publish okay no it's really not going to help me there okay let's come back to the slides all right so why does native aot work or how does it work I should say when we publish native aot which I'll get back and try in a little bit moment and hopefully we won't run into the same issue um C that you write in your application is compiled into I that happens thanks to the C compiler and it happens during the build phase and then when you go to publish your application the IL that was produced is compiled to platform code and as I said that only happens during publish so platform code meaning it targets the uh platform that you're going to run your application on x64 uh for Windows or Linux uh arm if that's what it is that you're targeting the published app in this scenario though has no jit okay there's no jit in the runtime that's embedded in your application when you is native aot it still contains a runtime so I it's important to acknowledge that these are still managed applications there still memory management going on there's still a GC going on but it is a standalone native executable there will be no jit that happens when the application starts up which is one of the reasons why the startup uh can be so much uh quicker it'll also be single file and so that's what I was trying to demonstrate there um and it will be trimmed to reduce the app size and trimming is something that's been available in net for a few releases now I had uh David talk about it yesterday and we mentioned it briefly in the keynote um and there are some uh compatibility uh considerations to take in uh when uh when using trimming because effectively what we're doing is trying to find all unused code in the application and then remove it in order to reduce the size of the application it's particularly important in Native aot because things that we can express in ilil and that then get turned into platform dependent code by the jit typically um cannot be expressed when the jeta is removed and you have to fully Express them in the native code up front and that's what can lead to Native aot compiled applications actually being larger um than their Nona nonnative aot counterparts um and so that's why we trim them so aggressively as part of doing so and then as I mentioned they're also operating system and architecture specific so when you create a native aot application you end up publishing for a specific Target by default when we did that initial publish which was a framework dependent publish that was a portable application I could take that dll that we saw which was my app dll was about 5 kiloby and I can run that on anywhere that net is installed assuming I have the right runtime the right version of the runtime available or run times as it might be in the case of an aset core application you need the aset core runtime and then you need the net runtime underneath it and you would run that by you know typing net space your dll name um in the case of architecture specific that's not like you end up with an executable uh if it's a Linux XE it'll just be the name of your application if it's a Windows X it'll be the name of your app.exe um and then that is what you ship around uh as part of your Deployable asset or you might bundle that into a container for example and run it in a container so what is the impact of having no jit so I said there's no jit in a native aot application well you lose some capabilities that traditionally you may associate uh with net applications but you may not consciously associate them with legit aspect of net so you don't get any runtime code generation because all that code generation is happening upfront as part of publishing your application so there's no platform optimizations going on at runtime after you've deployed um so you can opt into those as part of your native aot uh uh compile during publish and that is you know choosing uh processor instructions to Target so that certain intrinsics run certain ways but by default what you'll get is a you know a fairly compatible application you'll also get no dynamic pggo we've talked a lot about Dynamic pgo uh this conference huge addition to net 8 which is on by default for applications but because it effectively relies on this concept of tiered compilation in the jit um by watching what methods are being called and then identifying opportunities uh to to recompile them while the app is running to make them faster um that's not a thing you can do if there's no compilation happening in your app at runtime and so you don't get that benefit now there is static pgo in the native aot World there is a profile in there um so you and you do get the Ben benefit of the compiler being able to see all of the application uh during the compile phase so it's it's just a trade-off it's just a different view of the world going on there um there's no assembly. load file so you can't scan for assemblies and then load them into your application which is common in like plug-in models why because well those assemblies would be ilil and then they would need to be jitted into machine code and there's no jit so you can't do that um there's no expression compilation so if you're building up dynamic EXs or using a library that does that and then it relies on being able to compile that expression for performance that compilation just will know up it won't actually perform what will happen in a native aot application when an exec when when an expression is executed this actually gets interpreted uh there is an expression interpreter in the native aot runtime so it will work but it certainly won't be as fast as it may have been if it was compiled like it would be uh uh with with the jit and then lastly uh reflection not omit or refit which is a very popular way to do uh high performance code generation at runtime again no jet so we can't emit IL and have it compiled because there's nothing to do so all right what about the impact of trimming so I talked about uh there being no jit and what that means for your app but what about the fact that there's no trimming going on well unreferenced code the one your code that has no callers is just removed and it's unreferenced code from the point of view of what the the compiled tool chain can observe um it needs statically analyzed it doesn't run your app it simply analyzes your app statically at publish time uh to determine what it thinks is not being called by your application and then it removes code that is not being called so that can lead to problems for certain types of pattern um there is no assembly or type scanning as a result because you might be Tri trimming things away that aren't used uh and then you go to scan for them or whether they're assemblies or types you're using reflection apis and then they're not there because they got trimmed away um the opposite thing can also happen which is if you're trying to make the app as small as possible there are certain code pattern that can result in code not being trimmed away and that's because the static analysis can't determine which branch is going to be used and so it has to keep both it can see that there are call sites to both it can see that those call sites are rooted from the middle or from the start of your application um but it doesn't know which one you're going to use at runtime and so it keeps both and here's a very rudimentary example you may perhaps you have some config uh feature switch uh which you pull out of configuration at runtime and then if it's one value you do one branch and if it's another value you do another Branch well that API design in Native aot will typically result in both branches being kept even if in a certain environment only one of them will ever get used and this type of thing was actually something that we had to deal with quite a lot when making asp.net core itself native aot compatible we did have to do some API redesign um in order to address some of these things let me show you some examples of that if I go back to the project here and I'm going to change this to publish aot and we'll hope that publish aot works and I don't hit the same problem I just hit before with regarding my new get feeds ah this keyboard is different to my other computer that I just switched from Okay so we've got this create Builder API up here which is the root of starting your your as.net application the create Builder offers lots and lots of of defaults you get default configuration providers you get default logging you get Cal as your default you get certain middleware added developer exception page and development routing is added for you by default so and so forth um but some of those things can result in the app being a little larger than we would have liked it given the target cohort that I mentioned earlier we're really looking at Cloud native sort of um API focused apps for this release uh with regards to Native a so in order to help there we created this new slim Builder API and the slim Builder API as you can see is available on the same type web application you can call it in the same way you just get less of those defaults so for example there's no automatic light up on I here so if you're deploying uh to Windows native aot is not supported with the is as. net core module you can't compile native aot and then run in um is the same way you can today with as. aore and so that's removed from here similarly the routing system is still there but we removed things like the redx uh constraint-based route because redx is actually quite a large codebase and the way that the routing constraint stuff was built ends up pulling in a lot of code there that you would then have to keep in the application even if you never used use a redj based route constraint and there was a number of examples like that throughout the Builder where we chose you know we we made some decisions that trade-off between what do we want to work by default without you having to call more methods or perhaps support you sending a configuration value that turns on something in certain environments versus how much size you want to pay for in your application uh when using this API if you're publishing to Native aot so I create some Builder this application I know for a fact will absolutely work if I publish it as native aot because the slim Builder um still supports routing it adds it by default the map get method down here is relying on our inpoint routing and we've done the work in ASP asp.net core to make these map calls uh support native aot by way of a new source generator that is enabled by default for you whenever in your project you uh set publish aot to True which is the request delegate generator and that's going to generate a bunch of code uh in your application scrolled away like Source generators do so that these methods here rather than going through their normal code path which is what they do in as. netcore obviously and do a bunch of runtime code generation to generate a very efficient uh route Handler method for you we instead do that at compile time um and it's it's effectively the same code if you were to go and look at the code um you would see that it's it's it basically looks very similar to what would have been generated at runtime um with just a few little cat and those cve artarts are documented um but what it means is that this can now be native aot compiled um and the trimmer can understand what it needs to keep and what it can safely throw away which is fantastic now if you want to go even smaller so I know like the numbers here if I was to use the Builder a native aot this this app would be about 15 megabytes on Windows if I use the slim Builder and then I do it it's about 10 megabytes on Windows which is pretty good because a self-contained app uh that's not native aot compiled is more like 90 megabytes so that's quite a huge change you 90 to 10 but if I want to go even smaller there are some things you can do so we have a new method here as well which is create empty Builder and it doesn't even take ARS because args would imply that we add the um argument based configuration provider for you which we don't because it's an empty Builder so instead it takes a new instance of an options type and so the empty Builder is pretty much as empty as you can get it has no server so if I try and run this application I would get error because there's literally no web server that's added by default but I could go back and you know add the server I say Builder do you know webhost do use castol and now I've got a web server added back in here and if I wanted to craft Castrol even more I could say use Castrol core and then I wouldn't even get the Kestrel defaults it would be my responsibility to bring back in exactly what it is in Kestrel I want supported do I want hb2 and hb3 support um do I want htbs support to be on by default you would get to control all of that by these lower level methods and all of these things do add up they do do add up to quite a difference right now there's actually no routing added in this application even though there's a map get call down here there's actually no routing services so if I was to try and run this and hit this endpoint I would get an exception because I didn't add the routing Services because this is the empty Builder so I could go ahead and add those by services. add routing and then I would end up with the routing Services as well and you may have noticed in intellisense there's a routing core you know similar uh pattern here where you can get go drop down to the lower level method sometimes there's a slim method and most of the time you'll find this's core method and that's just less defaults it adds the absolute core required for a feature to work but you don't get any of those nice defaults usually and if I do that if I was to do all of this you know not even use routing perhaps I could even comment this out and just use a classic middleware for my Hello World instead like an app.use and pass a middleware delegate you can get this down to like 52 megabytes on Windows but you don't get any logging you don't get any configuration providers you can't read from environment variables by default you can't read from your app settings. Json by default we think those are reasonable defaults to have even when native aot compiling and running in a container and the type of workloads we're talking about so we left all of those things on when calling create uh slim Builder but now you kind of understand the difference between what create Builder does which all the defaults that you've enjoyed in aset core for pretty much all versions the new slim Builder which just turns off off some of those defaults because we don't really think they make sense most of the time when doing this type of application development and then there's the empty Builders like I you know to use hanselman's uh example that I want to drive uh stick I'm going to do it all myself just give me the controls and I'm going to manage everything without you getting in the way and depending on what you call and how you do that you will end up with uh different size applications so I I'll I will tempt the demo gods and I'll see if I can publish this application now that I've marked it as aot no it's going to give me a wall of air so I'm not going to do that I obviously have a new get problem on here at the moment but that's okay all right what other considerations are there when targeting native aot well there's no C++ CLI or com uh as I said before it will be a single file application you do have extra build time prerequisites so you can't just uh build using net CLI that you use today you will need that plus something extra you need a native compilation tool chain on Windows that's the visual studio C+ plus a desktop development workload um on Linux it's tools like clang and then on X on Mac OS you need uh the xcode command line tools installed um and then we have this all obviously covered in the docs to give you the absolute install uh lines and things that you need in order to do so you cannot publish crossplatform so if I'm on Windows and I want to publish native aot for Linux I can't do that as simply as I was trying to show you here uh you have to be on the same platform that you're publishing for in this world um and so tools like Docker can be very useful for that if you want to do Docker build I recommend you go and watch the uh container uh container talk coming up that's I think is later today um and uh dock Docker build is something that you could use for that on Windows you could also use WSL uh have Nate installed in WSL and do your your building inside there what other publish options are available we talked about framework dependent native a but there's a whole bunch in the middle as well if native a seems a little bit too restrictive for you right now bear in mind you can still do self-contained deployments to get one a standalone application that doesn't need a framework you can still enable trimming uh to get it trimmed down to only the parts that it needs and you'll have to deal with the trimming compatibility and you can enable ready to run which will pre-compile your application so that it doesn't need to jit at startup but then you still get the benefits of jit because ready to run doesn't remove the jit from the application so it can still do tier compilation and things like Dynamic pgo and then obviously single file is something that you can turn on without native aot as well but native aot is definitely sort of the peak of the optimizations and as a result there's some compatibility things that you need to deal with there's lots of other publish options that you can use here as well to tweak a few of these uh uh different modes with aot you can optimize for size versus speed the default is to optimize for size which makes it slower but you can also optimize for speed for Speed which makes a little bit bigger but does run uh faster then there's things like invariant globalization support for Event Source server garbage collection and the new garbage collection adaptation mode or datus that I know uh David fer talked about yesterday in his performance talk uh which is optimize and and attempts to use much less memory while using server GC um and and adapting that memory based on what the application actually requires so asp.net core native a what does it support kestral middleware minimal apis as we talked about grpc jot authentication authorization in8 there isn't support for MVC or web API razor or Blazer or signaler but those are things that we're considering for future releases uh for data access the postgress uh SQL adet driver and SQL light are both supported for Native aot Dappa aot was released just yesterday so if you're using Dappa there's an aot version of that there's also a small library that I've written that you can try looking at an enity framework core is something I know the team is interested to look at in net9 I'll just skip those ones I want to get back right back to these related sessions so to learn more about this there's a great talk coming up I'm sure from chat and Richard about net containers which is later today and then tomorrow morning very early there's a talk about C interceptors which includes some discussion of aot in there as well so resources here pointing you to docs and whatnot but uh otherwise thanks for this time I'm sorry that my Nate gave me a little bit of problems here but I hope that you learned something about Native a I encourage you to go and read these docs and uh if you have any questions reach out to me on Twitter

Channel: dotnet

Views: 10,348

Keywords: .NET

Id: FpQXyFoZ9aY

Length: 25min 25sec (1525 seconds)

Published: Wed Nov 15 2023