All About C# Source Generators | .NET Conf 2023

hi I'm Sean wildermuth I'm a Microsoft MVP an instructor and speaker I'm really grateful to net comp that they've allowed me this opportunity to give a talk here at net comp for the net 8 release it's very exciting let's get talking about Source generators so what are Source generators though they've existed before down at 8 they're being used more more as we approach later versions of net including net 8 these are a Roslin feature that allows you to become part of the compilation cycle the way this works is compilation is going to be running now this is not just when you do a build but also while you're using tools like Visual Studio Visual Studio code and other idees it's going be looking at your code as you create it and going to be compiling it behind the scenes in order to do not only creating code with Source generators but also doing things like syntax checking and showing you errors in the IDE while you're developing inside of this compilation pipeline there's a point where the source generators can be part of that compilation and so what happens is when it sees a source generator it's going to instantiate it and then tell it what code it is compiling because remember Roslin isn't necessarily compiling whole files they think of things in terms of symbols and so that can be down to the spaces and semicolons and those sorts of things though usually it's going to be at a somewhat higher level let's say at the class or the method level and it allows you to write code that is part of that analysis so you can go oh that is something I care about and that's something I don't care about and when you figure out what you care about you may may be generating code and then passing it into the compilation Pipeline and letting it compile just like every other piece of code so the compilation resumes and it becomes part of that process so the important part of this is this is generating code and not trying to change the code you've written so Source joint iterators exist for a few reasons one you're really moving an experience you have from reflection to something else and the reason why reflection is an issue is multifaceted one we're pushing those sorts of code generation that you think of with reflection into the compile time experience not the runtime experience this also unblocks ahead of time compilation trimming so reflection is notoriously difficult for AO and in fact can break some of your codee if you're using aot publishing in order to take out code that isn't being used reflection isn't being part of that because you're often not only just interrogating that code but also trying to generate code from it and so we know that reflection Works badly with aot ultimately what you're doing here is just creating code and that becomes part of this larger compilation so that you're pushing these decision to compile time instead of runtime in a way that's fairly invisible to the end developer so you might think of this as something more used by organizations or core teams inside of Enterprises or obviously tool vendors and the like as well as Microsoft so let's look at some code and I'm going to try to talk you through sort of the idea behind them and let's actually build one I'll make the source code available afterwards so don't worry about having to follow along in every minute we only have a half an hour so I'm going to try to pack a decent amount of Concepts in here for you instead of worrying about the manua so I've got a pretty simple console app and Let's ignore the generator for a minute and all it does is use it reflection to get some types right I'm going to go through and I want to get everything that the class and is public I want to get those types and I'm just going to Output them and I have a couple other classes in here and so if I were to run this you're going to see it outputs those three classes there might be some other classes in here but we're talking about within our application right this is all the code that we care about inside of our CSR and this is fine lots of people use it there's nothing wrong with it but it would be useful to be able to do this at compile time instead of having this runtime experience and so what does this look like one of the problems we Face we look at the CSR file is if we want to publish with aot we might have an issue so let's actually close that and let's open up a console and all we're going to do is say net publish I'm got some memory here and I'm just pointing at just my CSR so it doesn't try to build at all and I'm telling it to output a normal publish this is the way we've been doing it aot is not involved builds it and it's really quick right because it's just doing compilation and what this ends up looking like if we look at it in file explorer is our actual executable the thing that has all our code in it is 138 and it's only going to include any assemblies that are needed and this could be quite a large list depending on your application we're just having a console so it's nice and small we don't really have to worry about it but what if we were to turn on aot publishing now just to be clear what this is doing is it's compiling your assembly and it's taking that assembly and compiling it into machine code for a specific kind of machine most commonly for use inside of containers where you know the environment M you're going to be in you might also create different versions for you know win 64 and mac and Linux distributions um but effectively we want to do is make this so we don't need anything else on the machine to exist and so by turning that on when we publish this I'm going to put it in a different directory so we can compare them later it's going to compile it and then you're going to see generating native code and it's giving you a warning because we're using reflection right and because of part of what aot does is compile it and then remove types that it doesn't know that it needs it's saying that the types in reflection may be removed and in fact we can verify that by going into our aot example and we can see our aot example here and let's just say glassless console doesn't show anything doesn't show anything because all that reflection piece was ripped out because reflection and aot don't work together cuz reflection and aot don't work well together and so just to show you one more piece about this before we get into the code and we can see this size of our executable is maybe larger than you might imagine it's 1.2 Megs right and if we go back and look at our normal version we can see this is only 138k right well this 138k is only your code it doesn't it requires you to have a framework installed whereas this aot version this is the whole set of code base right it has taken all of the framework that I'm referring to and included it in this assembly in this executable so that we can run it without having any other dependencies so while it's bigger it's truly not bigger because you have this other big set of dependencies that it's going to try to load so how do we fix this so let's look at our other project this is a net 2 standard project and all Source generators need to be uh net standard 2 doesn't mean that aot exists everywhere just that way it's loadable in every single way so let's close these for a moment and let's talk about this generator class cuz all I've done is create a class I haven't actually done any of the work to actually creating this and so the first thing we're going to need is we're going to need some dependencies so let's go over to nugate and I'm just going to search for code analysis because that happens to be the two pieces we're looking at these are part of the rosling code I need analyzers always blindly accept all licenses because we trust everybody and so we need both analyzers and C these two pieces in order to write a source generator and so what we can do now is do two things I'm going to flatten this a little first I need an attribute called generator so that rosn knows that this being part of a project is going to be a generator so I'm just going to add the using statement there and I'm also going to implement the I incremental generator now this is a new way of building generators they used to be a bit more heavy-handed you need to do a little bit more work with this incremental generator but they're much much thinner they're much less likely to bog down the IDE or the compiler and so it's complaining because we need to go ahead and Implement that interface and it just creates an initialization with a context class and this is where we're going to start to tell Roslin about what we need and what we want from the compiler now one of the things you run into very quickly is it because it's a generator it actually wants you to specify in the assembly in the project file and force extended analyzer rules true and this is just a special property that is for code generators so you want it to enforce these analyzer rules all of them not just the bare minimum once we do that we'll see that the warning goes away and we can actually do the work of creating our generator so the first thing we're going to want is something called a provider and we're going to get this from the context and there's a few different ways to get data from the compiler but we're actually going to look at the syntax cuz we care about classes we're going to create a class that just has a list of actual classes to do the same work we were looking it before and in the syntax provider we're going to create a syntax provider and what we're going to pass in here are really two pieces we're first going to give it a predicate and that predicate is essentially the filter what am I going to care about and so I'm going to give it a static Lambda expression and this is going to give me the node of the compiler and a second parameter that I'm not going to need and I'm just going to say if node is class syntax class declaration syntax so it is a class then I'm going to want a transform and transform says okay you've satisfied the first piece of this make sure that using is there now what do you want to do with only the things that qualify for the predicate so essentially filter and the code that actually does something you're also going to give it a static Lambda and the use of these Statics is so that they are reused that there's not much being generated because these need to be as fast as humanly possible and here I'm going to pass in a text object and as well a second parameter that I don't really care about and how I'm going to transform this is I'm going to say take the node that was passed to us and I'm just going to cast it to class declaration syntax so this provider is essentially going to say do I care about something and how do I get the thing I'm looking for in this case it's going to transform the node that we're given that passes this into something and there can be more complex code here but the idea is for this to be incredibly fast we also want to make sure that the provider we get is going to test for is not null so in case we might get a node here that's null we don't want the trans we don't want this to actually Return To Us in fact this would throw an error and because of that this might return a null for us we just want to make sure we're getting the provided classes that we care about and then we need the compilation and we get this by saying context and this is where the actual work is being done so we want a compilation provider and we're going to say combine and combine is going to take the information that we get from the provider so we're going to say provider. collect so as it's going through you're going to get batches of these things that qualify again because it's not always compiling on a whole project basis it's always also compiling on units of code this collection may only include one class that the user is implementing as you talk about it I always do that right collect so we now have this compilation unit that we can use and we're finally going to use that context to register Source output now what does that mean that means we're going to take this compilation that's going to contain all those classes that we care about and then we are going to tell it to execute some code of ours we're going to want to take those classes and do something with them so I'm just going to call it execute this is just a name you could certainly use a Lambda here but I think it becomes easier to see what's going on if I go ahead and generate it so the brief walkthr here is this is going to filter it we're going to look at them all collect them so that we can then say I'm going to create some Source output for you here and I'm going to do it in this execute method and we're going to not even use it yet I just want to show you that we can actually just generate code so I'm going to say the code equals and I'm going to use our fancy triple quotes this is mostly why I asked for it to use the latest language features because these three quotes make it a little easier to format your code so I'm going to put it in a namespace I'll just call class list generator I'm going to create a public static class so I'm generating a new class as this happens and I'll call it class names and it's going to have one property public static list of strings so names of classes again this isn't very practical but I think you'll see how it can help you so we're going to create that new list of strings and then we're going to populate them and all I'm going to do here is say some class right we're not actually generating this from our source code yet we're just saying oh this is a way we can inject code into to our project and so in that case I'm just going to take the context and say add source and I'm going to give it a name and the convention let me move this up a little is going to be something unique I'll call this your class list G for generated and then then CS for the last file and I'm going to pass it in our code so this while this is kind of dumb right now it's just generating this static code I've created we'll come back to actually doing the rest we want to go back to our console app and let's get rid of this instead of class names here we're going to say class list generator do class CL names. names so this will be a class that's inserted into our project and then we're going to just say go through this list of course it's not finding class generator because we need to add a reference here so let's add a project reference in this case though this could be new get references as well it does not have to be Project based but for testing this works pretty well and if we build build this now it won't work and the reason is and there's ways to tell nougat to do this we actually have to come in here and say that the output item type is analyzer now when you build something like a Nate package for this you can tell it to put this in the reference when you create it but because we're doing it all by hand and testing it you're going to need to add that manually and so what happens now we're getting an error because we're asking for full name but you can see class generator names now exists so in here I'll just say right line C and if we go ahead and compile it we're going to get this error and that's because in my generator I messed up this field needs to have a semicolon after it so let's save that as a change we built it and now it's finding it correctly and we should be able to just run it and we can see that we're just getting that one class in here right we're getting that code that I just hardcoded so let's make that actually work so the first thing we're going to do is you can see the second parameter is actually a tupal right and I'm just going to deep reference the tupal so I can deal with the pieces individually so I'm going to say compilation and list of syntaxes equals the Tuple of course we need a VAR back there this is just destructuring them so I can use them a little easier and so here I'm going to create a name list by saying a new list of string now this is what I'm going to store all the names in for when I want to actually do the generation so we're going to say for each syntax in our list now what is that list that list is an immutable array of class declaration syntax so these are the things we asked for that it matched and that we cast it into here so this is just a way to pipe it into something we can actually work with and we're going to want to get the symbol for this class declaration SYM sytax we do that by using the compilation that we created we're going to get get semantic model of this syntax object and we're going to get the syntax tree and from that and that's basically a model of what the code looks like and we're going to say get declared symbol of the syntax as a named type symbol so all this is doing is converting one of those class declaration syntaxes into a symbol that we can get the name out of and then we're just going to say Nam list. add and we need a quote before it and a quote after it so we could have maybe used at there to make it a little easier but we're just going to use that magic syntax of saying symbol I I put too many quotes in here there we go symbol to display name to display string I should say now we have the name list right we're just going to collect a list of names to put in here so I'm going to say names equals string. jooin I'm just doing this so I can have them on separate lines and I'm going to say comma sln for return and then I'm adding four spaces don't have to use the spaces it doesn't care what it's formatted like but just so we can get a prettier version and I'm going to say name list and I'm mostly using join so I only get commas between them because again what we want to do here is doing uh using double dollar signs here so that we can use double brackets to actually inject this and I'm just going to say names right and so just to be clear we're converting our different classes into these symbols so I can just get the name I'm joining them to create a list of the names so that this list of strings will actually just be a Constructor with all of them and let's go ahead and build it see if I messed anything up and if we run it now we're getting all the correct class names cuz as this is being compiled so as these classes exist here as a new one is created so let's create a new zclass and because I said it was public I'm only going to include public ones though you can decide what you want as far as the the individual items let's run it again we can now see Z is there as well so behind the scenes it's generating that class that we can use there now this isn't a practical example right who needs just the names here the important part for me as we publish it with our source generator I'm putting in a different directory this time and we look at the aot source same sort of thing though the generator is giving us a pdb that we don't really need that one because the source generator isn't actually used at runtime is if we open this in open this in file explorer we'll see that it is appreciably not any larger even though we're using that right cuz it's trimmed everything we don't need and if we run this then we're getting that same experience even though it's compiled for aot now get rid of the part about being practical so in practice what would you use this for and I actually have an example of a project over in GitHub called minimal API Discovery I originally did this facility in order to allow minimal apis to be registered a little easier than the way we are doing it today and someone said you know this isn't going to work well with aot you might want to change it to a source generator and so that is what I did essentially this project though I won't get too in the weeds here is creating an interface and it's trying to find all classes have that interface and if we look at this generator that I've created for it again not a lot of code here I'm simply using it to do the exact same thing I just showed you which is I'm creating a call to register and then I'm appending to it all of the different objects that I actually find right and so what this is doing is for each one it finds for each one it finds it's generating a call to registering each one of these and that's really going to allow you to create things that are mutable but only mutable by things that you know about at compile time without having to use reflection to generate them at runtime so here's a few links in case you need them net8 of course everyone should get that now that it's out here's a link to uh Source generators on Ms learn which I think is really helpful I have a YouTube channel I have two videos on Source generators in case this wasn't enough and uh there's my Min minimal API Discovery project so if you have any questions I'll be over in the chat and uh I'm happy to answer anything so thanks for coming to net comp and uh watching my video and it's Sean Werth and you can find me at wth.com thanks again

Channel: dotnet

Views: 18,917

Keywords: .NET

Id: Yf8t7GqA6zA

Length: 27min 58sec (1678 seconds)

Published: Fri Nov 17 2023