Code generation with the Dart build system

hey folks we get started here in about one minute but I wanted to see quick if the screen looked good people could actually read the code and see the terminal I'll wait for comments in line it looks like there's about a 20 second lag which is a tiny bit annoying so hopefully I can still hear from folks so put some comments in the chat if you could if you couldn't read all the text in the screen and if you can hear me okay and we'll get started here pretty soon okay I got some comments the text is a little small I'll scale up a little bit here hopefully not to drive to make you crazy including myself writing code is running code with a big screen or big fonts is tough but we'll get going let's see here everyone says fine it made fonts little bit bigger let's get started I feel a little bit like a talk-show host right now talking to an audience that can't see me oops already this is my first live stream so you guys are all guinea pigs see how I do this I'm streaming from my basement right now and with a nice blanket behind me so this is all all smoke and mirrors so today I'm here to talk to you about the build system in Dart my name is Kevin Moore I'm a product manager on the dart team I'm working on dart for about six years and I think I was one of the first people to build some kind of crazy darts built system and it's cool to see how it's evolved now that real engineers are working on it so before I dig in here um I wanted to kind of play some context these are kind of crazy and I guess in some cases scary times if you're able to watch this live stream on a Wednesday morning we're probably the lucky ones unless you guys are off a shift from doing a real job or you three out in the world so it's good to keep that in mind um everyone's taken care of themselves and hope everyone is helping out where they can um and this is really just my way of trying to stay connected to folks and encouraging other people to stay connected I know there's been a lot of folks doing live streams and kind of virtual meetups and spending more time in Stack Overflow and on communities and that's great we're really lucky our our ability to stay connected and look at screens usually is made fun of and these times it ends up being an asset so help everyone's taken care of themselves and I'm excited to see if this is something useful and if people like it so let's get started so let's talk about the build system in Dart so yes first I already see someone making fun of me for not using vias coat I just like IntelliJ I never figured out the keyboard shortcut so I got them configured right envious code so I'm happy to IntelliJ so the build system in dart really what I think about it is uh letting the computer do what the computer does best I got into programming I like to joke that I'm a lazy programmer and I love people who like programming I think are lazy and then realize they can get the computer to do it right if once it's worth having the computer do it I was the kid in high school I'd write the function on my ti-85 calculator to do my math homework for me and it might take me 20 minutes but I can finish my homework in three minutes and I was still ahead of people and so I still do that to this day and I've realized there's lots of places in software where we have to manually manage things you know write boilerplate um you know what was the there's a German term that people are using toil was the work like any time you're toiling as an engineer we're just kind of having to do repetitive kind of painful manual stuff that's a place where you really want automate and I think doing cogeneration is one of those great places where you can have the computer do the hard work for you and so that's where when I kind of talk about today so I want to start out with kind of just an introduction to package build and I think I have a tab open here look at so packaged build is kind of the root of everything and what's really exciting about packaged build is it's absolutely tiny I think it defines three classes I'm gonna actually go look in the API reference here quick okay one of them I guess more than three classes but really one base class and that's this builder class and if you look at the implementation of builder it's one class and two functions and so the first one is basically something you implement that says given some input string or some some extension that you want to start with what are your inputs given that input I will generate code with a certain set of outputs or generate I'm not necessarily code it could be anything text files HTML yeah Mel Jason JavaScript whatever you want and so that's the thing that helps you personally mapped from what inputs you're taking the outputs and that's fundamentally build us and then the build step is you called with a build step object which contains some context about your inputs and lets you generate output synchronously or asynchronously and that's why it's a feature or and so there's a great example of kind of a very simple building here it says okay I'll take a text file and I'll generate a copy that text file and write it out so there's a couple things that the analyzer gives us and actually that kind of goes back to the slide here so Hermetic is one of them and this is actually really important I watched a video I think last night someone pointed on my tweet that you can uh you can write builders and gave a cool demo of doing the Builder and the the developer used a builder that included the current time in the output and I immediately cringed just like no you should never do anything where your builder includes information that's not hermetic and obviously the current time is that necessary hermetic thing you want your builds to be consistent across environments and I'll get into some of that later so yeah don't include the timestamps in your build outputs if you're using packaged build it assumes that it's it kind of owns all the inputs something else you should never do in a builder issue should never read files using the dart IO classes you should always read them using the build step and likewise you should never write files that aren't using the build step I won't get into that too much today I think I'll do that in a later video but you really want to use the build system here and that's one of the powerful things about the build system many that gets the second point which is it's incremental because the build system assumes it knows the world it knows all of its inputs and outputs it does let you do things like write over files you have to create new files some people get annoyed with this but this is actually a huge feature it's actually um inspired by and based off something from it's called blaze internally at Google it's an open source project called basil it actually the build system the doorbell system integrates with with blaze really well we use it internally is this idea that you want things to be hermetic and incremental and that means you can do fast things you can do caching really elegent li you can only rebuild things that need to be rebuilt you can be smart and so that if if you change a file and it doesn't affect the build output the build won't get rebuilt or rerun which is great so you can do caching and other things and so hermetic and incremental are really nice features and the final thing which kind of plays on top of both of those and it's something that wasn't discussed at least in some of the videos I saw is this notion that the analyzer is kind of built in and comes along for the ride and actually I can show that here and the readme of the build package which is you have access to the input library and you can get a resolver for that library and so you can find things like the name of the library or all the inputs and this is kind of the core where you get from just a general build model to something that actually lets you do source generation so built into the notion of the build package is access to the source if the inputs that are in your build system come from some source files from dart source files as you can access those settings and you can walk the library and walk the elements in that library and walk the elements and classes in that library and that's where a lot of the magic comes and so if you want to generate sharing code is awesome and then generating code from other code is even better because that means you can hoist out a lot of tedious stuff like generating Jason serialization and let the computer do the work for you so that's super cool I'm going to take a look at quick at the chat to see if there's any kind of questions before I go on and of course with a twenty second leg here I'm not going to wait too long to get into things but let's see if there's anything I've missed looking you look quick everyone's excited to see me I don't say all-out to my friends in Latin America my wife's from Argentina which is super exciting so I'm glad to see an Argentine their friends from Mexico I'm really glad people are here chime in chime in quickly on the chat again with the 20 second lag I don't have to sit here turtling my thumbs too long so if you have an issue or a discussion while I'm going along just chat in there and I'll try to address things this will be on YouTube afterwards and I'm guessing since or I'm guessing most people who watch this and watch it after the live stream I'm trying not to spend too much time going into nitty-gritty the shat but I wanted at least you know connected folks there so let's talk about an agenda and this is us here as much for myself is for you wonderful folks because I want to keep track of what I'm talking about I mean really basically the simplest build builder that exists just to give a quick introduction then we get into Jason serialization which is one of the things I've worked on the most and is it's basically the og builder it existed before there was even a build system it's an evolved several ways which is kind of crazy and then I'll get into some other fun stuff around what you can do to Jason serialization building your own CLI how you can test your builder a bunch of stuff so let's get going to code yes to IntelliJ so let's say we have intelligent here and again chimed in really quickly if you have if you have issues seeing the code let me just switch over to make sure I can see my this is where I'm the keyboard shortcuts get a little funky here just to make sure I can see the live stream and that everything's looking good looks like it's looking good okay cleaning up some windows this is my first time you guys are really lucky to have me my first time so I have a package here I basically built it out with the speech and so it's just an empty package with not a lot going on at all just a main I'm gonna go in here quick and uncomment the version oh and let's make this say we're not quite ready to ship so let's say 1.0 dev all that out and it's a common thing to want to access the the version of the package you're in via code you can't do that right now without you know basically hand wiring diversion the only thing that generally consumers of your package have access to are things in the Lib directory which is fine and so let's create a builder herb that gives us easy access to our version so we'll go into our pub spec we're going to add a package called build version I think this is v1 thanks we'll find out and something else when you use this both runner and I'm not sure actually not sure either of these versions so I'm gonna start with this and do an upgrade dependencies and then this is actually something pretty common I do is once I do that I'll go on and say Oh build runner is 181 let's just do that it's good oops 1 1 it's good in general at least in my experience to have as tight of version constraints as possible and especially your dev dependencies which aren't if anyone if I was going to publish this in a way that someone else could consume this package the constraints I have on my dev dependencies aren't can affect anyone consuming my library and so it's a good way to kind of keep me honest by having as tella constraints as possible and so unless I'm testing in older versions of SDKs they don't support things I might so keep these pretty tight so build version that's version two so let's do that run this again cool and that's all I've done so I haven't built a build that llamó file or any of the other configuration I just edit some dependencies and so some context for people know so build version is a package that has a builder in it and that's all you need to know and build Runner is the thing that does the work when you're doing the build system so it's the CLI that you run that makes things work and so what it gives me is this command good runner and a great new feature in the dart SDK is we don't pre-compiled packages when you do pub gate or pub upgrade instead we do it first execution which means pub get and public greater faster and we only pre compile the things you use which is great and so you see now we have a few commands here let's do pub runner pub round build runner build let's see if I can make this scale a little bit better you will see so you see some stuff already talking about well because I've changed some configuration in this package and other things it's going to throw away caches because there's good caching here and what you can see right away is I have a version file which is super exciting so that's all I had to do I have any dependency on build runner and I defend it on a builder and now I have this version thing going on with a nice reminder saying me telling people do not modify this this is generating code so what's really fun here is I can go in here and do let's do and actually I think this won't work as well as I think because I think we treat the pub spec in a special way because we do not want if you change your posts back you might change other things about your build system and so if I go in here and change this to one point one dev and hit save oh it's gonna rebuild and you see really quickly this file gets updated we split horizontally here and so you can see the first build can be a little too slow but once I get the whole thing running coach generated really quickly so that's kind of your simple first builder on build version and so generally the places where I use this and I would recommend it is I use it with I'm doing CLI apps if you want someone build ooh my command line dash dash version and print up the version this is a really easy way to have access to the version information and also the source code for the build version is pretty simple I think in today's talk I'm not going to get into those things and wait till later to get into that more because I'm also seeing home quickly time's going by so let's kill our builder right now let's get rid of build version this is not that interesting and let's go to G since utilizing jason serializable so let's add jason serializable here i think of it version 3 and adjacent annotation here I think it's 3 something here there we go source Jen's coming in again maybe in the future talk up like with that let's see is Jason annotation here it is great and let's just pin this a little higher cool so the point of decency realizable what does Jason serializable let you do Jason serializable is a package that lets you annotate movement video around a little bit unless you annotate a class with semantic annotations and then it's it lets you it generate some source code for you which is super nice so let's create a source directory I don't like having dot G files and the root of my Lib directory because then they get imported by folks so let's do um that's where that dart is probably let's stay really old-school so class customer let's do just to keep things simple I generally like immutable stuff but for this demo it'll be easier for us to do now let's do bool is important and maybe just for kicks let's do even make an even though thanks very little sense I'll just for fun let's do map string to customer let's make this recursive a little bit and once call this friends I guess conference easier so that's a customer class nothing special there and now let's say we want to do Jason serialization with customer well first we can do is let's talk about how Jason serialization works in darts so I can quickly go to API to dart not dev go to convert library you see there's a bunch of stuff in here that discusses Jason if it's not corrupt by a video I'm checking out down here so we can look at Jason encoder and decoder or you can just use the Jason encode to decode those are better functions so Jason code takes a value and a function we won't get into that and returns a string that's it and Jason decode takes a source string that's assumed to be a Jason document and returns back an object dynamic meaning it could be a bool a string a number a list of things and a map of string to something to anything else and that's basically it the one special thing about Jason encode that trips up some people when they use Jason serializable is there's a special function in Jason encode which is once it goes through and looks at the input object if it's not a map a list to string a bool and um it says oh does that object have a to Jason function on it basically just calls to Jason on the object it throws a to Jason call over the wire over the fence and if it returns something it uses that output and so what's great is you don't have to manually have to call to Jason on the object you see the hand jason encoder a class an object instance that has a function called - jason on it and it'll get called so let's go back to the source code quick so I can do map string dynamic dynamic - Jason and I don't know how to do there yet so let's just leave it open and then there's a convention at least I push and I think other people do which is if you want to create a new customer you can create a factory so let's do a factory customer from Jason and again we're not gonna do anything with that quite yet and then see if I can make my screen just cool let's have the programmer look let's have a the system do the work for us so let's do Jason serializable just annotate that guy have the code showing up here so there's customer and one real quick thing here so it's going to happen is let's do so we have a part file that doesn't exist yet but it will here really soon so let's go back to our CLI and do hub and build it and let's do watch so get into pre-compile things because I changed the dependency so I added some stuff hey how did Jesus annotation and Jason serializable and so it has to pre-compile build runner and we're building a snapshot waiting slowly and again we know that this first build is a little slow because it's doing it does a bunch of stuff it creates a snapshot on the build script so basically it turns look in any configuration you have and it turns it into a script that's run really quickly and already we're seeing an issue which is B no default constructor which is probably a bad thing so let's just let's just do that quick and then boom it gets run really quickly and already we have our output so already we have civilization for the customer which is super great so we have a function here likes given a map returns a customer and you see I used the dot cascade syntax to go do the right things it casts the types correctly and the case of friends this is pretty nuts but pretty cool it takes friends if the thing exists is actually doing a null check there but if it exists there it mapping that to a map of string to customer and does all the right things let's see here's the recursive call to from Jason and likewise turning things from a customer into Jason there's the same stuff and what's great is because you know even though friends is as we talked about over here friends is a map of string customer that's totally fine Jason serialization will handle that so I don't have to do any special conversion from friends to Jason because when Jason serializable I'm sorry when the dart convert library goes over and encodes that in a run into those objects and called you to jason automatically well you notice this is just too static and i guess to top-level functions here i'm not wired up we actually need to wire them up and i think Annalise will even be yelling at us if I reran analysis so you have two functions that are not used so let's wire those up here so let's do from Jason here cool and let's do customer to Jason from this and everything's happy so I could spend a lot of time digging into more stuff you can do adjacency realizable um but there's actually a few other things I want to get to and I've realized its Arney we're only 20 minutes into the livestream and there's other kind of crazy things to look at I would really recommend it people want to dig in here I'll get into some more esoteric stuff but I'd really recommend I did all this crazy work to actually generate a table talking about all the ways you can annotate stuff and actually was craziest this is done with a builder as well so I actually walked through statically the Jace's annotation package grab these classes find their fields and I actually generate this table using a builder which is pretty bananas um so you spend time looking at this I I think about maybe two-fifths 40% of the issues I get filed on jisun serializable class are on the repository if you are asking for features that already exist and you didn't see that there's features of this already there are a couple features that are kind of the same if you're annotating a field versus if you're annotating the class and also you can configure all these things and build that yamo and so I get into how you can configure Jason serializable Bowl here just some other interesting stuff that's a little nuts the class itself Jason serializable is Jason serializable and so that's actually how I go from the amyl in this configuration to jason serializable and that's the next thing i want to talk about um the other thing I would really recommend people spend time on and let me go into the repository before you file issues or give up or get confused is look at the example repo I spent at least I tried to spend a bit of time creating some good examples here my price should update and make the docs better truth be told but this is a great place to start and so I actually gave an example of doing some pretty straightforward stuff like you know worth a you know you know what if this is not knowable it changes the output you know using constructors using named arguments that all works great annotating classes in different ways and doing custom conversion so I have examples of custom converters here so for a while I think I instead of an encoding date time with a UCF very with the RFC standard for date/time I encoded as milliseconds or microseconds and so I use custom converters here function since you can figure out how that works and actually I even go crazier with custom converters here so I actually have a syntax for how you can create a converter to handle special classes so if you want to have generics in your class this is a model for how you can do custom things so I'd really recommend people look at the example so this is the jason serializable repository on github there's an example subdirectory look at the example and the great thing about doing example code here and not in Docs I'm your last people really like Docs keep being here Cuse us all honest because I know that this code keeps working keeps building and asked tests and so I know this this code right here is pretty fresh and works well I look the chat quick to see if there's any questions Oh Philip it's Philip asking he's now Philip is um is on our deverel team and it's done a video on this already in the boring show so um is he answer oh he's answered questions for me oh god bless him okay good I thought he's asking you know back asking a question of me but now he's helping up folks too so there's obviously other things that do the similar stuff so you can look at something like bills value from David Morgan built collection and he has it's a much stricter model for things and uses a very very e-enough elaborate I mean that in all complimentary ways it's a pretty powerful system for doing immutable stuff using a builder pattern so there's other ways to skin you know just give this cat that's just one of them so what I wanted to talk about next so we talked about Jason serializable and Jason annotation a quick thing just for folks again that are curious and this is good to know is that you notice I have Jason annotation in my normal dependencies and Jason serializable in my runtime dependencies and this is a little confusing the Jason serializable class that I annotate with this is actually defined it's my voice um from talking so much it's actually defined in the Jason annotation package and so since that is in code that's in my Lib directory I want that to be in my normal dependencies they code this run when they run a build which is only something I do you know as the package author that's in dev dependencies and this is a good distinction because you you know this is gonna affect how your uses of your package are you know what things they can use with Jason annotation there's actually some code and Jason annotation we'll talk about that in a second but Jason serializable that shouldn't be a transit dependency that shouldn't be something that affects anyone that imports your library so keep that in dev dependencies same thing that build runner so I haven't you know personally Jason here and I'll leave that as an exercise to the reader um but what's fun about this I think a lot here that realize is that a Jason is a valid subset of llamó and obviously we use lots of enamel in the dart ecosystem and so I was thinking about this because I realized we have lots of places where we use yellow and dart we use it to configure builders so there's a build that yamo file you would configure things and the logic of creating a custom reader for yeah Mille was kind of a pain and I realized why are we doing this work when we have this nice Jason serializable thing and so I created a kind of crazy but it works pretty well um package that and added some features adjacency realizable to make dealing with the ammo much easier so I have a little cheat sheet down here give me one second open it up okay so I'm gonna come back into this I'm having a tease so wrecking our demo app I'm gonna paste in some code here I think I'm gonna get yelled at I want to make sure check - yeah mo it's actually dependency so let's bring in check the animal oops actually should mean that's better there we go so I think check TMO is being brought in by build runner but it's important to actually depend on it so let me bring in check PMO here it's also important that I give a good dependency so always add dependencies to your and versions folks it keeps us all very honest ok let's close out a customer like looked at now so let's talk about scenario where I want to you know parse some arguments I'm sorry I'm gonna take some input from my CLI and part of that is an arena gamma file which is super exciting and what I want is if I read my llamo file I want it to be I want to make sure that the L file is valid and I'm gonna get a nice object graph for my animal and if it's invalid I want to see nice errors right I want to see parse errors and give the user some input if they have if they gave me bad yamo and when you see a good example of this when you see a nice error message in your console you like that's a nice package and we don't see one you get sacked and so that's what I've done here so I created a very simple class and it basically has configuration and it has a name and account of the values and so without getting into too many things here quick I want to run it and see what happens so let's do dart and let's do bin main and let's just pack something pups back oops dark pressed night pops back yeah this is gonna fail because I'm running a dead version of the dart SDK that is a bug that's really annoying so instead I knew this and get rid of all this stuff so ignore this you should most of you should she has to be able to run dart Ning that dart because I have an old version of things it's not working so I tried passing in my pub spec to this command line and you see I get an error here which is I am unrecognized keys description version whatever and the only keys are supported our name and count so it's actually create a file at this thing room early support sample dot yeah mo cool and name is Kevin and count is 42 of course and let's do this so there I have a nice decent output of my configuration but you saw before where I have some nice features here right so I can come into here and say oh like let's make this let's like the count 42 is a string and rerun this and you see o unsupported value for count and these great pointers to the problem to have there so how did I pull this off a couple things so if you look at the ya know package that exists in dart it's a pretty powerful package there's a couple interesting caveats and nuances that I forget the exact details for but it turns out that the map object you get when you partially animal from the piano package is not a map of string to dynamic or object it's a map of dynamic - dynamic and Natalie had very good reasons for this and I forget what they are but what that means is you actually need to say any map when you do your Jason serializable and what you'll notice here is there's actually special handling in here although you don't see it in this output it looks like to nuke the thing but there's special handling that does not assume that the keys are strings you have to do it you'll see that I passed and checked mean true and what that means is the gym so I'm gonna jump over the generated code the generated code wraps the creationist object where they checked new and that does some basically some special stuff that actually captures the creation of the object it captures the map the source object that contains the input now here it's called Jason it actually turns out it CMO but it's effectively Jason in terms of the format and then I have some other things here saying I don't allow unrecognized keys and all the fields are not nullable and then down here actually do some special work to say these things are required and you'll see that I call it basically have a set of functions so I call this checked keys function that makes sure that it passes in saying these are the allowed keys and these are required keys you see every time that I assign a field I don't aside to feel directly from the value from the from the input map to the input Jason actually wrap it the thing called check convert and all of these functions do is they do a bunch of work to throw the right kind of error that's it they throw helpful errors if you look at checked new you notice I do a bunch of work to throw this check from Jason exception and what's powerful about that is it actually includes the map that was used to create the thing and if you look at package GMO a little bit you'll notice that the yamo code the yamo package does a really great job of corresponding all of the values in the ammo back to the source input file and so you actually get offset information into the into the file that are the the content yeah Mel that you parched well that means you can get really really nice error messages and so you can format this some more and you'll notice in my little CLI here I basically do the work to call check to Yambol to code and I pass in the content and kind of the factory that's used and the search URI and so this checked yam will the code is the magic function that turns all of those little exceptions that are thrown and gives you a really nice output and in fact this will throw an exception this throws this parse DML exception and you can catch this exception yourself and do cool nice interesting powerful things with it and so I'd recommend people play with this if you ever in a place where you're building a CLI or anything that parses ya mold that's kind of user visible and you want to give nice error messages you can do this and in fact it doesn't you don't necessarily have to use the amyl either you could you could do input jason is your input and that it worked fine in fact I think I don't think it really cares that's actually I mean yeah Mille well yeah I could you know do the same thing here and do I can make this yeah Moe and parsing of this should work fine yep and if I come in here and make this you know the string you know I still get good errors so I um I do this all the time now I never part see Hamel without this and it makes things really easy so so we did Jason and then I'm extending things out - yamo well where else to be the case where we have some input we're parsing from the user that's well-formed but probably contains lots of strings and we want to parse it into nice objects um hint hint CLI so if anyone's dealt with our parser in dart again another great package that was done I think a lot of work by Natalie and I think Bob did some work there too this is a package that lets you define a schema effectively and map user input in the command line to basically map a map of values you know arguments options and values would it be cool if you could just write a class that did that so I'm gonna do something even crazier here I think and I'm definitely um fishing without a net I'm gonna go back into this class and well first let me go in here now I'm gonna add in a build CLI annotation class and I forget the version and let's get those things a patient's Australia there we go and so this is 1.2.0 and built CLI is one point three point seven and we'll go check quick to see there's like comments people that are excited about parsing llamo good cool so let's go back to main I'm not gonna change anything I'm gonna leave Jason serializable here why won't this estate Jason serializable but on top of this I'm gonna add a annotation which is build quit CLI options and then I go back and I can run my builder and that's you pre-compile because I added some dependencies and again I'm definitely doing this without a net so I hope this all ends up working building my snapshot I really push to have reticulating splines just sprinkled in here for all those folks who played SimCity that's an error that I need to talk to Jake about but a warning don't worry about that and we have an output so now let's go to our Jerry code and you'll see that we have output from Jason serializable which we you know it looks familiar oops Jesus utilizable so this is the code that we just generated and now you see all the stuff for an arc parser and so now we can now we can create an OGG parser from list so actually lean back up further there's a bunch of helpers here they're private you don't have to use if you don't want but the core thing is I never function that given a list of string args will give me a configuration object okay so I'm gonna go back into this code I'm going to get rid of all these things because I'm not going to use the building ml stuff quite yet and what I'm gonna do is I'm going to say find out config equals and then remember what I call this parsed configuration for arguments okay and that ran let's go into I forgot what's rectory I was in dart build live stream oops and of course you remember we have our crazy and already we're getting an error which is the source most nothing known but let's see here if I go in here we can see that I created there's a name and account so I can do - name equals Kevin and say - Jeff count is 42 it works in fact we make it a little more interesting and I can go print and what might as well just do config - Jason oops because I already had I still have the two jason code in there right so just a good deep breath I wrote no art parser and our parser logic I mean it's it's powerful that does what you kind of want right but it's tedious I have to you know reference counts and I have to do an in to parse the thing and I knew one to make sure that you know is there a bad number format you know and do I have you know do the right do the right thing there and I do that all for you in fact I actually did the works of that I call tryparse which will return null if it you know if it fails and then an odd no I actually do the work to throw a good format exception so that you know that oh my input is not the right type and it just works so I actually went and created your lesson sorry scrolling around a lot here let me make sure I find the right thing so the parser for configurations are here populate the parser and actually this code it makes a lot of sense I could get into it later it turns out that you generally you often want these special things so again I won't get into this too much unless you know art parser well sometimes you want to this function where you can create your own art parser maybe you want to customize things so you have these functions to call if you want to but in general I just call add option add option and this is where you know my my paranoia kind of spikes which is anytime I see the same string more than once or twice that's the smell to me like okay I'm referencing count once or twice that's bad I'm gonna fix it and so that's kind of my inspiration here but really quickly and say you know there's other things we want to do so let's go into this name thing it gets a little crazy but you know it's very readable I now know the configuration I'm going here and go CLI option which is a key for this guy and I can say oh like let's do abbreviation and let's just do N and let's do an abbreviation for this guy and let's do see sure and let's do Oh doo doo doo doo doo doo he knew your own converter default help this is the count don't let me write your help um this is the name and you'll see already oops it had to rejigger things I'm now you know defining the help for the command and the abbreviation so now she'll come in here actually let me go back in here again let me add in a final rule help cool um let's save that you'll notice that already I have a flag so these are done as options rules are automatically added as Flags which is fine any kind of what I want and what this means is I can go back to my main thing here I can go if config that help equals true I do that because it could be null I can do print and this is where you want the other things that I've generated so actually come in here and get the parser and there it should be a usage notice return so now I can come in here and that still runs I can do - check help as you expect source nut must not be no oh oh oh so I think let me do this and go nullable true is that it was like oh let's forgive myself in trouble oh no the source must not be known that's right just one second does everyone in the chat screaming at me no Kevin this is what it does oh yeah name must not be no let's just get rid of that for now all right no ha ha if help on equals true let's do that oh you know everyone seriously sitting in there and their things screaming at me Kevin what are you doing wrong in two parts are you kidding me then parse anyway oh this is where I get in trouble trying to show cool stuff you can't do help this way let's just get rid of that let's go do that there you go okay that's a little better um yeah cuz the income in parsing gets crazy um I could have done the default actually we can go back but this is actually the crazy thing right so I just remove the field it was done it worked cool we come back in here do that and we do defaults to zero there you go and so like the speed with which you can go and like oh I have a CLI that parses the objects it just works which is really crazy um I actually did this in a package I call peanut let's see it's a class decorator and forth to useful help okay I'm looking in the chat real quick nothing too crazy there I actually do this in a package package I maintain that's kind of nuts called peanuts and does a bunch of crazy stuff but wanted to show you here is seeing the read mean let's see do I have ducks in the repeat I do not a lot of Docs but basically so there's CLI options here for doing a whole bunch of stuff in peanut and this so background peanut is a package that does the work to build your web app and your flutter web app and actually we'll build it to another branch for you um in a very clean way so I can be checked out master with all my code with a web directory you know whether it's flutter web or normal dart web that I can run and then should SFP web directory in this example it's an example directory and then you can run peanut and I'll do the work to build the code I build it off to attempt to write directory for you and they update your gh-pages branch for you so you can push your examples without having to like check out your project twice and rebuild it or copy the output it just works really nice for you when I realized is I had a whole bunch of places where I wanted to have so I built this it has CLI options so I can say oh by default use this directory or you you know up into this branch then I realized well I actually want to be able to have configuration for this so that you know if the if you know it's not the web directory it's example I don't have to type in - the example all the time and so I created a yamo configuration and this is all done with um this trick and so I think watch it's not that oh you know it's here so I have options and the options is both Jason serializable and CLI options so I can create my arm parser and give you a nice options file that you can just use your CLI like you expect and then on top of that if you want also have configuration for I want configuration for my CLI I use Jason serializable with the check - llamo crazy and I get both and so with the exact same code so if i want to add an option to do something else with peanut i can add it in this class maybe tweak annotations and for free i get both a yellow configuration object and a CLI from one object one class um and it works great and then in my testing I can validate the parsing works correctly by just validating the - Jason from it so it ends up being super nice super user so there's all that crazy stuff and where are we in time I do have a really important meeting in about 10:45 so I want to be cautious of time here one last thing I swear in the show um let me bring up my thing they'll verify so you've done all this work to create a builder in your package and maybe you're building to code because you're generating stuff something that I ran into a lot was that um I'd forget to run my builder before I checked in um and I should back up a little bit and they might do there was another great person I should find his link here um I shouldn't find it I'll post it there was there was somebody that uh did a great series on the build package already and I could go into more of a kind of build itself and Saur Saur Shen's another kind of layered thing on top of that so there's cases where you're not building to source and that's generally like if you're compiling to JavaScript so angular does this a lot jas compilers kind of it's called building to cache so it doesn't dump the files in your package and there's lots of good reasons to do that my philosophy here I'm checking in built code is that if this is code you check in anyway if you're just using the build system and these tools is kind of a helper like your little handyman or woman it's good to check in that code because then people can just use your package especially if you're letting people use your package you don't want them force them to have to run your builder so I just consider the build system as like my little handyman running code that I would write anyway and I check the code in and that way someone's using this package they don't have to run a build of themselves that's kind of my logic for it what I found was you know if you changed the version of something you know you change the version in your pumps like that yeah Moe did you remember to run your build again and that's where build verified comes from and instead of wiring it all up I might just show you here how I use it and so this is the peanut package again and I depend on build verify you should see build verify here in my dev dependencies and what I do this is my convention I use and you can do it as well is that I tag it as presubmit only and I call this expect build clean and so this expect build clean again is a very simple function in this class and what it does is there's some stuff around if you want to change how things are executed yeah I know if you're in Windows I would actually check there by default I just run pub run build run or build but what this does is that make sure that for the current package um running a build shouldn't change anything again it should be completely hermetic and so and I take it this pre submit only and then if you look there's actually a way to configure the packet and test package and I say you know anything tagged with crease limit only should only be run during pre submit and then on Travis I'm jumping around here um you'll see that I have a task here that says run skipped because that's a skip thing and run the presubmit only tag things so just again because I don't want to rerun the build every time I run tests locally so I create a test and this is it I say my test isn't your build and I just call this function it just calls it as a lambda expect build clean is from built verify I take it pre submit only I configure the package the test package here with to skip it this is only run but faults not run and then on Travis or whatever your favorite CI is I say okay run that skip test and verify the build and all this does is make sure that the code you have checked in if you want to build no changes happen and if there has been a change it'll fail and this catches me all the time where I've changed the field or changed an option and I forgot to rerun my builder usually what happens is I change the version from a - dev to a release build and before I publish this catches me so I have a very strict rule which is I try not to publish a package unless Travis is green on master and so bills verify is your helper there so if your find yourself using if you find yourself using builders and your own packages that's a great way to make sure to kind of keep you honest um and that's the less the rests are the the end of my agenda I got nervous the beginning because I was going fast I was like I'm going slow and it's like oh no I need to catch up but I think I've gone through things so let's see here let me look at my questions quick is there a way to have fun to watch automatically and called build runner for us that is a great question I have not done that much stuff with the bill package in funder apps because usually when I'm using the build package and build things I'm doing it for kind of CLI stuff and tools so I need to find that out i I've heard that feedback from a few people that using packaged builds with flutter is tough I don't know if people have tried doing a flutter pub our flutter packages pub run or flutter pub run build runner if that works um try that out and put comments below I'd love to find out I'm happy to dig in and find out for people because I know a lot of people want to use this stuff with flutter um although a lot of these tricks are for uh for other packages let me look real quick to see if there's other oh we got someone else like leadings lines I'm scrolling through real quick to see if there's any questions here everyone to know about flutter I guess that makes a lot of sense I wanted to see quick I'm gonna go into hub dev and let's do so these are all my personal packages I maintain I see if there's anything else in here that's interesting well version we talked about build verify we talked about completion if you're curious I did some work so that for bash and CSH if you wire things up correctly with your arc parser we could actually do command completion in the CLI I need to spend more time in this package I haven't done much but if you ever wanted to do command completion in a dart app look at the completion package one thing is little nuts right now is that it has to run the whole app to get the arc parser so that the completion works and that can be kind of slow unless the app is already pre compiled and so I've had some discussions with folks about is there a way we could basically prepackaged your args in a way that can parse very fast and so if they wants to dig in there you can look at that source shin test is a package I use if you build your own source Jen generator sorsha and test is a great way to test that and has a bunch of really nice utilities for testing source Jen based builders I'm gonna look really quick here to see if there's more comments I know I said about an hour and a half I think I'm gonna stick in an hour because my voice is very tired and I have a meeting at 45 in Berlin discuss really quick I think before I say goodbye though is I would love to know if there are other topics that people would find interesting for another livestream I'm happy to do this more it's fun so I'm gonna wait 20 seconds to see if the chat goes nuts there it's nice to see that I have 45 people still paying attention and seeing what's going on I hope the quality's been okay and I swear the coughing is just for me talking for 45 minutes straight I usually don't um I would love to I think um something else when I get into is talking about source gin which I think may be just fun to talk to more people about so the build package does a very simple thing which is just a builder and so it takes kind of the core input is a single file or potentially set of files generally single file and generates a single file what source Jen lets you do file deleting builder yes you cannot delete files cannot delete files or change files in a builder and that's by design because you want it to be hermetic so anyway um package build is meant to go from file to file effectively what source Jen lets you do is generate bits of output for a part or a library from elements in a library and so it really is meant to go from dart source to dart source instead of just file a file and so when you see these tricks where I do and annotate something with build CLI or Jason serializable that's using source Jen package source Jen with package source - unless you do say four classes or any member really annotated with this annotation so four classes into this thing let me create we call a generator so it's an ax generator is kind of part of a builder and so you can compose many generators together into one builder and what that lets you do is do things like you know annotate a class and say oh I want to generate some helper for this which is super nice and there's a bunch of really nice utilities inside package source Jen as well for basically looking at a library and like reading a library reading and getting LR reading reading things members out of a class finding out the types of things which is obviously really important for things like Jason serializable and build CLI because you need to do different things if things are enums or other stuff in fact actually let me do that as one kind of final hurrah example here let me go back to my example this is these these are things that take forever to write but they're so cool and they work enum let's do um my options cuz I've got any things let's do Alpha Beta Gamma Delta sure I got my Greek down and really quick I'm actually gonna go into here and bring in my source control because I want to show you some what changes when I write this so here's my alpha beta gamma let me add a field to configuration and let's just call this this is called silly and let's make sure that the generator is still running it is unless you have changed so now what you'll notice is I did some magic here to handle the enum I can decode the enum and and do the right thing so go from a string to the output value nice what you expect and then I did actually the work in CLI in the CLI to actually parse it and I did the work so there's like oh like what are the allowed values for that class and so I think if I go back to my CLI now and I run help I get an error of course oh yes and I need to this is a bug I need to fix because jason serializable and CLI options both generate the same helper map it failed spectacularly so I'm gonna delete that really quick actually even better you know just not essentially met sorry I'm gonna go into this and I say Jason he ignored it there we go so now this will not parse with Jason and there we go and so now you notice that our yeah so I can you know passing an option that isn't in the enum and I get a nice error that it's not allowed in that option and so this is these are the things that just take you forever right it's like oh no just Brendon enum and then the person will just do the right thing for you so yeah donate to your favorite charity helping out bring this crazy times um if you appreciate this stuff cuz it was pretty nice to implement I won't take any payments personally but I'll take your karma and your likes and and donate my name to a charity of your choice because doing this kind of parsing and deaiing with enums is pretty nuts but when it works it works out really nice so it's 10:02 i've been going for about an hour now i think you two might kick me off at some point but i wouldn't to say thank you for everyone a webcam blocking oh yeah i said ten o'clock and this might be one of my children coming in to say hi so hope ever takes care themselves I hope this was interesting if you liked it um I guess add a like I feel like I'm one of those guys you know Squarespace is not sponsoring this channel um follow me on Twitter and YouTube I guess to subscribe if you want to I probably do most of my talking on Twitter if I do these again and let me know if doing another talk on source Jen in particular would be interesting or something else I guess some people mentioned server stuff which I do a little bit I'd be happy to dig in there also let me know if I talked too fast I wasn't thinking too much about how quickly I speak and generally I speak too fast so I need to be careful about that as I said before you know we live in scary times take care of yourselves and people around you and help where you can and we'll see you on the Internet

Channel: Kevin Moore

Views: 1,501

Keywords: dart, dartlang, dart-lang, build system, code generation

Id: iVoz7kJoLFQ

Length: 63min 57sec (3837 seconds)

Published: Wed Apr 22 2020