Building a Plugin for Nrwl Nx Workspace v11 - BEEMAN.DEV BLS023

[Music] foreign [Music] hello there everybody this is bls023 beam dev live stream and today we're gonna be building a plugin for nar1x version 11. so let's dive right into it what i'm going to do today is i'll share my screen in a second here let's get rid of this one and open the other then share screen and here we go here we go there we are let me get a second right here and put myself in the corner so the goal of today's stream is moving the code from my nxpm stack repo into a new nx11 project and for those of you that not aware of nxpm stack this is a set of schematics that i created a while back and it allows you to scaffold out a full complete application so it's a full stack application you can install the cli you can run init with the name and this will scroll the complete app for you so it consists of an api and it consists of a web front end it has prism as a database and it connects to graphql so it's a full stack application generator and this is built using nx10 so it's the it's the version that came out in spring of 2020 and i'm gonna rework this into um nx11 so this is the goal of today now if we take a brief look at how it's structured then we see that i got three packages ci the cli in the stack now the ci is not very very special it's not very big the only thing that it does today is generate a github workflow so this is one of the parts now the reason i called this ci and i didn't call it github workflow is because my idea is that you can pick whichever ci you want to use so in some point in the future you should be able to select travis ci or circleci to do your ci or gitlab whatever you're using so this is also the room to to contribute if you're interested in contributing in this stack you're more than welcome just hit me up on discord i will chat about it and i'll walk you through what can be done and what we have here um then the cli well that's the cli itself we're not going to go over that and then the stack this is where the interesting part the meat of the application is so here we have a bunch of schematics and we start out with this one the initialize and this initialize schematic calls in to all the other schematics so initialize here when we look at what it does it change a bunch of existing schematics and the main ones here are that it creates the app the api and it creates the application for the website and then it does a bunch of other stuff in setup now i'm not under the impression that i'm going to finish reworking this all today but what i want to go over today is create a new workspace so this is going to be the new project that i'm going to be working on for an xbm stack create a new plugin new nx plugin and i'm gonna start out with the api so i'm gonna have my new plugin generate a new api application for us and set a few things up and we'll see how far we go so that's it for the introduction and uh let me quickly close uh this chord here so i'm not switching over it all the time i'm gonna go to my this is open source going to go to my open source folder and i'm not sure if i already have an xpm folder i do uh so i'm actually going to go to another folder and i don't think i got an xbm folder right here so that's that's great so yarn create nx plugin and i'm going to give it the name nxpm and the plugin that i want to create so this nxpm is the namespace it's the organization on github or on npm and the plugin name that i'm going to give it is api so right now in nxvm stack i got a plugin called stack which is which has everything but i'm going to divide this so it's going to be web api there's going to be some initialization stuff but stack is too generic and everything's stuffed into one library and i want to separate this out a little bit so this is what i'm going to do right now i'm gonna start this yarn create nx plugin and this will take a little second to get here but in the meanwhile everybody is already in the chat and i'm gonna say hi to everybody hey orlando buenas tardes hello there bill nice to see you thanks for your comments on on youtube we have john here john nice to see you sergan hey there also good to see you and lars everybody welcome i am super happy that you are all here uh bill asks a question actually any chance you can extend the schematics to take input interactive to select between graphql and not so great point yes definitely so the schematics come with a way to handle input and you can add a schema that allows you to define what you want to generate currently it's super opinionated the whole stack is opinionated for me this was the easiest way to get it done um in fact i don't want to depend on most of the utilities i'm using so the one i'm gonna depend on for sure in this stack is nx workspace because that's basically the common denominator i have in all my projects all my serious projects in the opinionated stack i'm going to go for nest js as an api and angular as a front end but i'm also working on a very similar stack for a client of mine that has react and react native and i definitely want to be able to add that additionally i also want to say i want you to allow to use graphql or rest api rest api so these are things that i'll make configurable at some point and if this is something you're interested in working on again just hit me up on discord and we can talk about adding you to the repo and having you contribute this is super welcome so it looks like the project got generated so what we can do is start it up here nxpm and i'm gonna open this in ide and while that's open we can start actually looking at what got generated lars that's a good question normal dev kit yes i'll probably say schematics and all the whole lot of time but i will be working with generators where possible so that's definitely something that i i need to be aware of i'm going to be doing nx11 we're going to be doing generators for a lot of these those i'm going to be wrapping um angular because a lot of stuff from narwhal is still not available as a generator so yes i'm going to be doing it all in nx11 with the dev kit so pardon me already if i'm going to miss miss it i will definitely make this mistake more often i mentioned no workspace yeah it's um so the idea is starting out here a new plugin this allows you to add stuff to an nx workspace i'm going to start out with the api this already has an executor which formerly was known as a builder we don't need that for this api because my project at this moment only consists of generators and here we can see the generator api and this is the one that i want to do and one of things i want to experiment with during this live session is checking out in to which extent i can do this test driven so 20 20 21 for me is the year to improve my testing efforts and um nx helps a lot with this my ide helps a lot with this so there's really no reason to not do it so this is one of the things and actually nxpm stack the original package right now has quite a quite a complete integration test an ntn test so this is something that i'm going to be basing this work off but i don't think that's the most interesting to share in a live stream so i'd rather just start out with from scratch show you folks how you're able to do something like this and then just to repeat it the first thing we did was run yarn create nx plugin give it the name of the scope and the first name of your plugin what this generates is a empty not an empty an nx workspace that's in i think they call it like using the oss preset so you don't have apps and libs but instead your libraries are called packages and your apps are here in e to e and this is something that you can set in workspace.json here in the nx.json so here you can see workspace layout is divided in such a way that the applications go into e2e and the lips here now one of the things i often start out with is set up my prettier rc so i'm kind of opinionated in what i have as my preferences so i'm going to copy over this from another project i need to be in the in the project directory and copy over my prettier configuration file so this will do my configuration and i'm also going to lint staged and husky so that this gets applied automatically when i'm done and the last thing to to note here is that i'm gonna be running these scripts on commit i think pre-commit i'm do not gonna do lin staged pre-push i'm gonna do format so i'm copying over this snippet of configuration opening here my package json and um when this installation is done i'll add this one so this is basically some some set up some craft that i always do all of my project at least they're formatted the same way and normally i would do this off camera but i want to register it from scratch so doing everything i'm just checking out ciao good to see you always nice to have you here i already got comments like why is chow not on the street well at times sometimes i do streams by myself but i like i appreciate people asking it okay interesting comment from lars about plugins working differently on windows and locally or or github workflows and locally and that they are slow yes i i really noticed that they are slow so in my experience the the most most of the time gets spent in actually installing the packages because if you do an end-to-end it will create a project in your temp folder and we'll install the dependencies and this at least on my connection that's the bottleneck so i wish we maybe narwhal or we as a community can come up with a fix for that that it will maybe cache your dependencies locally your project i rather cache cache and nuke the dependencies some sometimes when it breaks than having to reinstall it all the time so there we go i added this husky which is the the get hooks so pre-commit i'm going to run lint stage and linstage is going to do this formatting and it's going to run an xpm stack so this is a bit meta meta but i'm actually using nxpm inside nxpm and in order to do this i'm going to install the current version of nxpm stack just so i got access to the linter and this linter what it does is sorts your projects in annex jason so these ones shorts to my name and it sorts the the path here in in um in this ts config and inside your workspace.json it sorts it as well so that's that uh one of the things i'm interested is why it didn't add the add here i'm gonna do this manually but i assume that that would have happened before so we're installing nxpm stack here and in the meanwhile i can already format the code now again this is all a bit of setup and this should and this shouldn't take take too long anymore and we can start really writing the plugin so in the meanwhile i'm going to quickly run the formatter and yeah i got a new connection by the way so it should be more stable than it was before before i had a connection that used to lock up at times but it looks like it's it's better already let me know if anything seems off and happily fix it so with that out of the way not a valid object master okay interesting so my default branch is main as you can see here in the red part and i got a message here that says not a valid object master and the way to fix this is to go into annex json and look here at the affected default base and switch it to whatever you're using as the default base so i'm going to say main here and another thing i'm going to do is activate prettier so that it prettifies my files unsafe here in the preferences i should be able to turn on pretty it's already turned on okay i'm not sure why it didn't do this let me see if a manual reformat works reformat code because it's probably because it's in the root for now i'm fine with how it is um and i'm gonna run this format right one more time just to make sure that i'm all formatted and nice and clean to go and then after that we're going to go ahead and create our first schematic so this is this is that i'm going to commit it all and i'm going to do initial commit again so this is a second commit and of course i'll be sharing this code with you online now here's an error interesting live code you all know how it goes nxpm stack is not found and i know what the issue is i installed an xpm stacked package but what i actually should do is install nxpmcli another thing that i noticed is that package lock got created so nx is a bit unopinionated in which package manager you use and if you don't configure it on starting the app it will use package lock i'm using yarn in most of my projects so i'm going to get rid of this package lock and i'm going to use yarn as my main package manager here and then with that out of the way i should be able to do this linting so yeah there you go and we can say yarn and xpm stack that's lint and this should now run and by default these packages are created in the right order so we we won't see any changes but i can now do my initial commit and this should get us starting okay this all looks good and so the idea of this one is this generator it shoot let me get straight here this is where we actually execute the code now what you can see here is that this adds a project configuration and it adds this as a library and it sets some things now what i actually want to do i'm not interested in doing this manually what i actually want to do in this project is say hey api api generator schema it should take a technology and uh i think for now i'm going to call this pipe and pipe can have a value of nest and later on this will have go as well for instance so you can generate an api based on go but for now i'm not going to focus on that but i'm going to make it configurable already and this then also to answer the question that was asked earlier before by bill this will also be the place where i say hey you have to generate an api based on rest instead of graphql or both maybe so this is where you're gonna do that and um so this is the type is gonna be nest and here we're gonna add a property called type as well and we're gonna say that this is a string and it's a string of enum and i need to have a snippet there you go i got a snippet right here it's i think it should be this one or i can pick items and i can say that this has the item of nest and if this is not it and we can say this is type t and the description is [Music] the type of api you want to generate so this is my first antique type it's a string and it has a value of nest and i think i need to do this differently but i'm going to peek in my examples one of things i'm often doing when i'm working on new projects and migrating over old codes i find myself doing that quite a bit and i create the directory tmp so this is default in your get ignore so it won't add into your repository and when i go into tmp i can now sim link here and other projects so i know for instance that the current version is in oss stack so i'm making a sim link here if you don't know what a sim link is go work on your unix basics it's super handy to know these kind of things my editor already detects it i hope it likes it and i take a zip of coffee in the meanwhile but in this example i can take a look so this gives me the option to quickly peek into how i did stuff in in the other project and it looks like my editor is not liking this edition so i'm gonna quickly go ahead and make sure that this works as expected nope it doesn't i'm gonna just break it off for now applications not responding so force quit it and restart it again so start ide right here and that should be it for now and we have preston joining us preston good to see you man it's always a pleasure to have you around preston was on the show a few weeks back and we looked at how to optimize your github actions by using annex affected preston's also a big fan of nx and rightfully so because it's an awesome tool and for the for the occasion today i wear my narwhal pride shirt so this is uh if i work on narwhal on my stream normal stuff i often wear my narwhal pride it's really a life-changing changing tool for me and i really appreciate the hard work by the good folks of the narwhal team and in fact this version 11 so normalcy narwhal team the nx team basically decided to change a few things so we went over this briefly but whereas before it was schematics right now they're talking about generators which is a more descriptive name it's it's a better name and where the angular cli talks about builders at nx cli talks about executors so there's a few changes here in the meanwhile i'm going to go take a look at my package here and see if i can find an example of the configuration i want to do so we're here inside my stack and inside the schematics here i should be able to figure out what i need to put there but it looks like my editor doesn't like my project i'm going to give it a few seconds or a minute here to make sure that this loads in as expected i guess it might be the as for sure it's the fact that i'm live streaming because that's always the moment when things go wrong it might be mad of me that i chart i load in too many files at the same time one more time doing the same trickery here i'm gonna be stopping this one it's interesting why this doesn't work well i'm going to be stopping this one and starting it one more time and i really hope that that gives us the desired effects intelligent intel gib like this and get rid of the project for a second so this is starting need to pull it over from my second monitor and then i'll i'll move on with this and in the meanwhile we have dominic join us dominique nice to see you man it's always good to have you around as well dominic's also a big fan of nx and a plug-in author himself so for i think for a lot of people there's not a lot a lot of new stuff but it might be cool to see how we integrate the existing schematics with the new way of doing generators so your previous work is not lost you can still use your existing schematics in fact if you do an nx workspace right now and let's say you pick the nest or the angular preset nx in the background will use existing schematics for you so it's not that the nx team converted it all and we can take a brief look at how they did this because it took me a bit to figure that out i'm gonna be starting my new browser window here and picking up this one so if we go to the nx project and if we look at how these folks implement their packages and we can see that let's say nest right here so annex it so nx is self-hosted so it's it's nx is using nx to do its stuff and here inside schematics let's look at application so what this does right here is well this is still using a schematic so this is still the old way and what they do here is they wrap the angular schematic so they say rep angular def kit schematic they pass in which schematic it is and which application and this gives you a method and this method is something that you can then include and and and use in your own apps so this is something that we're going to use oh geez that it's it's really stuck i'm not sure what to do i think i'm gonna go move over to my other editor and leave this for what it is for a second because it's basically hanging on me and i'm not sure what the culprit is this is the worst moment this could possibly happen halfway in my stream but uh i guess that's how it is so i'm gonna force quit here and i'm gonna say this one just get rid of it and then instead i'm gonna go ahead and remove my ide settings ready i'll start it one more time just to give it a shot because this shoot there's no reason this shouldn't work as expected and actually so you temp series i think i know what the issue is i had a bunch of stuff in temp this one ct so yeah plugins are super nice and it's nice that nx embraces it and gives us a preset to do it so we don't have to figure it out ourselves but there's also a few things that are not ideal yet so one of things that lars already mentioned the e3 tests are super slow because it has to install node dependencies each time and yeah this kind of prevents you from quickly doing an end-to-end test so i'm super happy with the new setup where you can just include code and use it in your generator so you don't have to um have to register your schematics so it all work and this was different in the previous versions so okay we're back here let's see and let's stay where we are right here so yeah we're going to use this application generator and this should allow us to pass in the options here uh in which way we want to generate our nest application so that's what we're gonna do right now is it really gonna go ahead and do this wrong again let me pick up this one and see where we are right now so we're going to go to death nxpm i'm really sorry folks for the delay but it this is the life of a developer most of you are probably um aware of the demo effect that sometimes these things just take longer than they should i'm gonna go ahead and open webstorm i need to make sure i increase my font size so i use two editors i use webstorm for my day-to-day stuff and i use ide for my um for my screens for my streams just to make sure that i'm in like a larger font size for my screens streams so now loading webstorm and there might actually just be a lot of activity on my machine so let me quickly inspect hold on there for a second to see what's going on here and if this thing is actually super busy which it seems like it is yeah okay so i'll be reorganizing this in a second and then right about now we can really start working on the actual plugin so okay going to keep an eye open on this one while this opens at least the data connection stable so we got that for us which is nice and i think we're back yes we're back okay awesome great so in here in this generator so the goal of this generator is that it generates an api for me and um we set type to nest and i'm actually going to go ahead and export this type so we can get a hold of it i'm going to say export type api type is nest or well we only have nest at this moment it's going to be api type so this is kind of nice so that i can use this in my code and actually i'm going to i want to make this nina alias to enum i'm sure this works as expected or if it doesn't oh yeah my machine is super busy with random stuff so one of the comments of dominique here unfortunately found a lot of apis i just i use just work with the amulet dev kit 3 so not easy to migrate yeah i found a similar thing and another thing i found out is that it depends on how somebody builds their plugin if you take advantage so you should probably always use abstractions that come from narwhal workspace like get project or something that get project configuration so that you at least don't depend on some hard-coded version because we are at a time of transitioning from a version one workspace file to version two so to quickly go over that workspace json here it's a version two and you can recognize like okay you now have targets this used to be architects and i have builder executor this used to be builder other than that i think it's mainly the same i think these are the the main two differences but yeah sure enough if you open a workspace json in an unexpected format it will not load properly if you don't use the right abstraction so what i want to do here is um basically get my options and if you're not a um we're not in an sjs repo i think api type is nest uh we probably want to throw an error oh new error the error i'm going to say is unknown type and then the type and this will make sure that there's a notion of different technologies but right now at this moment we only support one now i think here if i run this test i'm not passing in any type so this will most likely fail and this will fail because it doesn't know the type so it will throw an error and let's double check if that works because that means that we're right on track now this is what i think is one of the biggest benefits you can just await your code so you can make a very linear flow okay it cannot find schema you cannot find module schema from source generators api generator.ts source generators api generator.ts and this cannot find schema interesting it wasn't an it was unable to find schema.dets you want to include this file in your import which is current currently ts js tsx okay interesting if there's somebody in the live stream that knows this i'm super happy to hear if you have an answer and it looks for schema.json and actually they should both be here so i'm kind of amazed why this doesn't work so what i'm going to try out is run the nx test command and give it the api which is the project i'm working on which is this generator to see if this helps we can actually get rid of the executor because we are not going to do an executor custom executor at this point so i'm going to delete this file here executors directory actually i'm going to get rid of that one and here in the executors.json going to get rid of this file as well and in the package.json we have a reference on the executors gonna get rid of this one as well and now you cannot find executor so i'm gonna run the same thing over again nxtest api and we're gonna see if that gives us the output that we expect failed because it cannot find this schema.json okay this is interesting because i would assume that this would be included so let's see ts config here is pointing out to these ones and then we have spec here and okay so it should include this d.t.s as far as i'm concerned you know what we're gonna go ahead and undo this i'm not sure why this would have any effect on how it works but um let's see if this fixes it and then inside the generator what we actually want to do and i'll see what the error is still on the same one now the interesting part is i didn't really change anything on this generator just yet so i'm curious why this fails because that means that it might be that the default configuration that they generated doesn't really work and they there were some issues before i that's how things work in our industry things fail but let's see what this does and one quick way to try this is to basically add another plug-in don't change anything and see if the test runs so this is always my now it passes okay interesting interesting interesting so it looks like if i add an export here and i run my test again looks like it breaks and this it might be something here in my compiler options but okay it doesn't break right here okay interesting enough let's add back the next one let's actually use the enum that i have there and run the test again now i'm curious how this works okay this seems to work i'm not sure what the issue was but then it also means that this should work right here so i can use this in my ide again it should run successfully now in this test what we actually want to figure out is that we create the application so this api generator it needs to it needs to create an app and we're going to say that type is nest so the type is api type nest issue then run and one of things i can do with this situation which is super helpful is set breakpoints so i should be able to set a breakpoint here and let's see a breakpoint here just to name a few and then i can run my tests here in debug mode so i can say debug my api generator and this is super powerful because this gives me an insight in what is running in my code and what is inside my my memory so i don't have to do console log driven development which i used to do well i probably still do it today but um yeah a lot of console logs here in my in my projects and this is a better way to do it interesting so if i run it right here i get the same error i had before very very interesting and i'm not sure if i understand why i don't think i understand why this is i don't think i understand why this runs differently than the ones i run in in the comment line actually than the ones that i run here because if i if i go ahead and run it it should run maybe it's something in the run configuration between the api debug and the api runner i really can tell and now it doesn't run anymore okay interesting so far the demo gods are not happy with me and i'm not happy with them frankly [Music] so what i want to do actually in this generator is take the generator that comes from narwhal nest and use the application generator from there so in order to do this i need to install the narwhal nest dependency so i'm going to go ahead and yarn at there's the at normal nest now one of the things that makes so one of things to note if we look at nx we can see that nx divides their packages into technology mainly so we have nest we have next we have node react web which are web components we have angular etc express these this is all technology driven right and then there's a few others like jest which is technology cyprus which is a technology so nx really focus on technology what i want to do with nxpm is abstract one level higher and focus on functionality so in nx you're talking about a in nxpm we're talking about an api which can be nest or can be expressed or can be go and you talk about a web but that can be angular and it can be react and later on we'll have native which can be our mobile which can be native script or react native or ionic for instance so this is kind of the idea i want to abstract it one level higher and not have you pick the technology but merely have you pick the functionality you want so cool with this one here i should be able to find this novel nest and from here get the application generator so i got the application generator and i got the library generator now instead of doing all this i can say application generator and pass the options in here so um i think i want to give a name so that's options.project name sorry we need to look at the normalized option so this is a normalized function it takes the options here and it normalizes a bunch of stuff that's the project directory it finds the project name which is based on the directory it finds the workspace layout i think we don't need most of these tags is something that we do do need but now i think we can just say normalized options dot project name there you go and this is whatever we should generate and we can get rid of the app project configuration because this is the manual way to do it super interesting but not something that we're going to do today and this is basically what our generator looks like so it's a generator it wraps around it and it calls into another generator and when you change this sufficiently you can build full full stack apps but this is the magic behind nxpm now add to that a lot of opinions and a lot of shortcuts to make sure it works so there's a lot of things that can be improved that i didn't worry about just yet and um i think that's that's kind of a nice thing um it allows you to really quickly iterate over ideas and and and get stuff out of the way even more so when the tests work which they currently don't and uh there's anybody in the room that doesn't know what i can look at um i'm more than happy to try it out but we can at least see that this build ran and i can also say nxtest api and see if this now runs again it might be related to being doing the build actually and let's see how this works and we can also look at the e2e because the e2e this is one of the things as i said before the current nxpm has a quite a big e3 coverage and my goal is to move over the e3 test and make sure that it still runs without changing a lot of things but then re-implement it in a new way now still got schema error here i need to check out if somebody knows how to deal with this because i don't know what we do what we can do is say nx e2e api dash e to e so the api e3 is this project right here it's an application or at least inside this this this system and what this will do is it runs it makes sure that you have a project gives it a unique name it will create a new application with this project and it will run the builder now this will fail because we don't have a builder anymore so i i hope this fails this directory this should still work so you should be able to install this inside a separate directory albeit we're going to go to apps so this something we look at and it should add tags so this is the api and this is actually a bit confusing so there's a bit of confusion between the names we're in the api package but this one should actually be called application so we're going to wait a second until the api test runs e2e test and then gonna rename this one api to application and make sure that it still works because the syntax i want to use here and this is the part where you can see this i want this to be app or application so this is basically the the convention you say okay generate an xvm api generate an api application and the reason i break up these things there are multiple reasons one is it should be better manageable if you have small chunks they should build faster so if you do effective builds they should build faster if you only have a change in the api it should only do rework the api and the related apps but if web didn't get touched it shouldn't do that let's see okay it should create the api i couldn't do that there's a jest timeout okay i know how to fix that one and there's generally a bunch of things that are not right so first thing i'm gonna do here in just config i think it's called test timeout yeah i'm out and i need some test environment to note so these are two things that i often set when i do testing now inside the end-to-end test oh i need to make sure that i do this in the right one so here just config in the api set it to here now inside this one what i actually know okay the actual zoom run so it should just create an api here inside the directory and here inside the tags i'm not sure if we're gonna be doing this so i'm probably going to exit these and exit these so they won't run and this also because i got the test coverage from the other project so this is something i will be reworking in this in this project now it's kind of messy and still setting up i really hope i figure out what the error is causing here with the schema because this is kind of blocking me right now i was hoping to be able to do this test driven and look at what gets generated in the unit test and make sure i can inspect and have that workflow because the unit tests are super fast or relatively fast compared to the end-to-end tests and as you see right here empty end-to-end test takes a while because what it does here in in the temp folder it goes into nx e to e and it starts new project here so it literally starts a new nx workspace which is this ensure nx project command and it will it will build it all out but it will also install the node modules so this is one of the one of the reasons why it's so slow and this is also the one we need to as we as a community probably need to come together with a few smart folks and figure out what is the best way to deal with this because i think we're losing a lot of time waiting on end-to-end tests direction dictionary for a second and here in the generator i can take a look okay so this failed and these two are skipped it should create the api but it doesn't okay and this is because um i told it to go look at application but application is currently not a known schematic or generator i should say so i'm gonna go over here into my packages now the way the way this works is your package.json has a list of generators this points out to another file's generator.json and it's actually this name that it's looking at so i'm going to change this to application but also the path where it looks like where it looks at and the description so i'm going to make it all application generator and i'm going to add an alias here alias and alias takes an array of string and this is going to be app so you can say nx generate at an xbm api app or application and this should work exactly the same cool application so in order to accommodate this i need to change this api word here to application as well so it can find this and inside this folder i need to do some search and replace as well so this application we actually load generator which is a nice common name i kind of like to have unique names where possible in the mono repos so i intended to call this application generator and actually api app generator but i need think about this i'm not i'm not going to make the move today but if you have a lot of generator.ts it's kind of hard to do finding your files by name so i really like the functionality in your editor that you can say okay give me generated.ts but if there's a bunch of them this gets super super messy so let's let's keep that in the back of our minds and right now what i want to do here is search and replace application or everything that's called api to application i'm going to write it full out so this is going to be application type application schema generator and this is going to translate through to my app the whole schematic and once i got this all done here in schema.json this is now the application generator so that means that we now can say give us the nxpm api semicolon application and let's run this one to make sure that this works as expected i'm going to run it from here see how that works if that works and then let's see if this gives us a an api and one of the things i like to do in these kind of situations is set up a basic ide that works so my goal is that i have an nx works an empty workspace and i generate an api an sjs api and for me this is probably going to be the first first proof of concept if you will and in order to make sure that this works what we're going to do is add a github workflow so we can run this stuff inside github actions if we push it up and of course i'm not going to write that all from scratch so what i'm going to do is create a folder here.github workflows dot github work flows and inside here i'm a copy over my workflow that i have from the current nxpm which should still live here in the tnp folder so tmp okay unable to find generator application in this packages so interesting i'm gonna go to first let me finish up the github workflow thingy so github workflow here you see i have a building test so what this does it uses nx cloud which you should it on ubuntu 14 i'm currently on ubuntu latest note version 14 i'm testing the the following i'm loading in a postgres service so that our api can has something to connect to we're checking out the repo we're caching node modules we're setting it all up we're doing a format check so if you don't have if you send any files that are not formatted it will build out and it runs test ci now test ci currently doesn't exist so let's start with that one let's create a test gi script here test ci and what i want to do on test ci is east yarn nx end-to-end api into end so this is at the minimal thing i want to happen on my end-to-end test later i will enhance it a little bit and use affected but for now this is the way to go and i can copy over this workflow now actually because this project will eventually take over i'm also gonna i'm also going to copy over my funding and issue template and these live both here and issue template is for github issues so to make sure that you have the right questions and the funding template gives you the sponsor button if you're not aware you can sponsor me on github i got some lovely sponsors already thank you very much it really really helps me motivate i really appreciate the support so cool so we have this api right here and let's take a quick look if we can find out why this doesn't work so what i'm going to do is build it nx build api this prod i don't think that's just brought really helps anything but i'd give it a shot cannot find configuration for production yeah it doesn't even have a production configuration and these plugins will just build and that's it now this build will end up here in packages api source generators and actually here needs to be [Music] probably got rid of them just when i wanted to open these okay build is done so inside packages and let's refresh here we got an api uh generator.json and here we should see application yes this is application this is the name of the generator it goes into the source generators application which is this one and i'm kind of surprised that this a end-to-end test doesn't work so let's run it one more time and we can now say yarn test ci so it runs exactly the same stuff and i really want to see this test green so we can move up and send this out to our github repo and the repo that i'm going to use here i'm probably going to be using the same repository just to make my life easier down the line and i'll do some rebasing magic in the meanwhile in xpm you get this one get the repo url and add it got that one right here so i can say git remote add origin and passing url in order to be able to do push i need to um create a new branch because these two don't share history so i'm gonna create a branch next and this is the one i'll be pushing up so i can already push it up right now these two don't share history okay cool so we got a a passing test and it's a minimal test we don't really test anything but at least it's green so that's good so that means that if we do a commit right now and we have this all set up and this should do a green run in our certain gi in our github action so what i'm gonna do is set up basic api plugin and push this up so it will start now most of you are probably familiar with github actions but if not let's take a quick peek at what it looks like so i'm going to do a git push using shortcuts here that come with my shell z shell and then when we open this in a browser we should see that this happens okay let me organize this a little bit and pull this window over here so we can take a look at what happens so it see next had a recent push this won't merge so it will probably complain like hey you can actually not merge this stuff they say there's nothing to compare i want to see if i can make a pull request anyway i think i cannot make a pull request anyway that's interesting this is because these things don't share history so i'm trying to make this into a new branch and you know what actually i'm going to create a new repository in the nxpm repo and xpm organization so that's gonna be here nxpm and this is gonna be stack next and it's gonna be public and it's going to be empty so this is a new empty repo and i'll deal with merging these things afterwards so what i'm actually only interested about is getting this origin address so right here i can say hit remote remove origin and add this one in right here and this also mean means i don't need to worry about let's just put it in next anyway so this one is pushed up right here gonna go back to the main branch after this do a push as well back to the main branch push this one up as well it pushed up then i should have both of these running and one of this should trigger a github action to run so inside here inside my stack i can now go here and say that i want to change my let's mute my cell phone for a second inside here i can change the default branch so default branch will now be set to next because this is the first thing i pushed to but i'm going to go ahead and change this main should be here and this should be the one that i have as well switch to another wrench yes instead of next i want to be able to use main there we are update yes i understand the implications and inside here stack next i will see that i push to this there you go had a recent push and right now when i do compare and pull request it will create my pr set up the basic plugin this is all the changes that i added so i'm going to create this pull request right here and this should now trigger the github action to run so while this works and we should see that this action runs here perfect i'm going to go ahead and switch back branches here to next and i'm going to run a tool called yukum hukum i'm not sure how to pronounce it huk um and what this does is shows you the status of your of your github actions right in the terminal so you can continue to work if it's green it will succeed if it's red it will fail i'm going to quickly summarize what we did here so what we did here was we run yarn create nx plugin and we get the name of our organization in the name of the plugin scaffold out a nx workspace in oss mode in open source mode in package mode however you want to call it which gives us a packages directory for our lib libraries and an e3 directory for our projects for the applications that we're running and this is something that's defined inside the workspace layout property in nx json also because our main branch our default branch is main we call it main right here and i added my prettier configuration right here installed husky lint staged and configured that one here as well and additionally i installed the add nxpm cli which is actually the package that that's not that one this one nxpmcli which is actually the package i'm going to rework inside here and i'm running the stack linter on this one after that what we did was go into the package here and change this generator from application from being api to application and that's about it we made sure that test works at least locally because here they failed but we made sure that the test ran right there and that's basically it and this is what i try to do as soon as possible i try to set up initial test coverage and having these tests run automatically so i get notified of fields of failures early and often now let's click into this one and see where it is actually filled it's the yarn build okay yarn doesn't know what to build that makes sense i didn't configure this so here inside package.json i'm going to say yarn build and this actually nx build it doesn't do anything right now i'm just going to say build api and this is kind of a shortcut for for now but this will make it hopefully passed through this one and make sure um so use correct build command and what i like about this workflow and this ucomb tool you can whatever it's called is that you can do it all from the comment line so right now i'm doing my commit and next thing i'm gonna do is push it up so that's going to be right now push it up and after this i know that it's in that it has a pr already and with this pr it will run automatically so what i do is i make commit i push it up and as soon as it pushed up i wait a few seconds so right now i'm waiting a few seconds so it can kick off this github action and then run this hookum tool and this will directly say okay this is the the next test run i'm gonna do it right here so for having a fast quick feedback loop this is pretty ideal to not have to leave your environment in order to run your tests on github and see the response obviously when some something goes wrong you need to check anyway but yeah this really helps in speeding up that that stuff uh so yeah that's basically what we did right here to set up this uh this package and this structure and one of the few things um one of the things i'm gonna be adding right here and this right here so this generator i will have a library generator as well so api will have a library and then if we look at nxpm the current one here this was pretty much an experimental phase and i learned a ton of things i also learned a ton of things that don't work so if you see right here naming is hard as we all know this is not really clear what it does so i need to change this to application i need to change this to library this is the end-to-end test so i'll provide my stacks with a working end-to-end test for the apis which is my preferred way of testing apis now these ones are actually things that i'm starting to call features so i won't have two libraries one authentication data access one authentication feature but i will have like one feature auth and it consists of multiple packages so i this is hopefully will bring down the list of schematics that there are because this is basically an implementation detail the fact that i built my libraries in two packages corn feature feature and data access that's an implementation detail so i'm trying to minimize this and for web it even gets more out of hand because for web we have a few libraries that are not super common assets and style these are not very common libraries what they actually do is basically a mechanism to copy some files in your dist they don't add any typescript any logic so i i really hope to clean this up a little bit now what we see here is that this build completed so this is awesome when we go back here into github actions we should see that we now have a green build set setup basic api plugin so cool this ran the whole thing this ran the ci as well only took 15 seconds so that's that's pretty neat right here it does go faster which totally makes sense so my connection takes longer than on servers that are the connection that github is connected to so next thing up for me i can merge this in it i doesn't doesn't really have to merge in right now but why it is not and i'm going to merge this one in here and this is basically the workflow that i have so set up the basic plugin make sure that at least one i don't need to restart and delete it make sure that at least one of your end-to-end tests and plugins works add your configuration script so it adds itself to github actions and it runs the stuff there and then this is a great starting point to move on to other stuff so and right now i can check out main pull in my latest changes and sure enough this is my main plugin now um to to cover the base here uh what i'm gonna do to make sure that this uh works as expected is and that i got the structure that i was actually expecting and i add another plugin so i think we already have we have the narwhal nx plugin right here so one of the things i can do is next generate at normal slash nx plugin and what we want to do here is add a whole new plugin so we're going to say plugin with the name web and we're going to try run it just to make sure folder i mean i'm not in the right folder yeah now i am i'm going to dry run it just to make sure that we are getting the right um the right thing outputted and it says import path is missing so import path is the thing [Music] sorry folks import path is actually the path that is here so for our web plugin we want to have the import path set to at nxpm slash web i run this this should generate something in packages slash web there you go it should generate in packages web and add an end-to-end test for web as well so let's add that one get rid of the dry run i will also add packages for the cli i'll add a package for ci as i mentioned before i'll most likely add either a common or a core that just has some common shared utilities that i use between all of these but for now we're interested in adding the web so this one got added here now our web plugin will not have an executor so i'm going to get rid of executor right here i'm gonna get rid of executors.json and gonna get rid of the executors here package.json then inside here i'm gonna go ahead and rename this web so this again this will be application and in the generators.yes here web is also going to be renamed to application source generators application generator and then another thing i want to do add here an alias so this is alias which is app that's all great that's all good let's see if this runs so run the web generator here just run the test and in the meanwhile we can go into the end-to-end test or the web and make sure that this one's updated so again i'm not interested for for now i'm not interested in the it's here and in this as well so we're going to skip these two we're going to skip running of the builder because we're not working with the builder and we're going to change the web here into application then this should run our end-to-end test so we're going to run it locally here run web e3 and actually in package.json what i can now do get rid of this letter here what i can now do in my test ci script what i'm actually interested about is not running just api e3 but i actually want to run the affected so affected as the name suggests only executes the affected code so when i say nx affected e2e this should actually only run my web e2e okay comment field npm install i think i know why this is let's go back to our generator right here let's open our generator right here and here i'm going to do the same so i'm going to get rid of this one i'm going to get rid of this one no files we're going to leave the files out there our web we're going to start out our web with angular so i'm going to say yarn add this d at narwhal slash angular add that as a dependency and this will give us the same thing we can do just generate an angular application by using their their method so we want to say from a normal angular and this is i think the application generator so this should be here application generator and this is what we're gonna execute right here so we're gonna await application generator and as a name pass in normalized options.name indexing is done right so and inside this web spec let's see there's some red squiggly line here i'm not sure why that is this one to the side and figure out if we can see why this [Music] fails somebody sees it please scream semicolon expected well that's weird i don't i don't do semicolons interesting it's actually a tslind version so okay cool so this one got generated and if this one if this one succeeds as well let's restart the typescript service for a second restart typescript right here now inside my package json so i'm going to go ahead and change this so my test gi runs nx affected affected e to e so this one and an x affected test so yarn nx affected e to e and check the test and let's do test first because tests run faster so if they fail i'd rather have them fill fast on the tests then wait a bit first the whole e3 and then fill on the on that and fill in the test later so let's go ahead and see if we can get this one in so i'm gonna start a new branch and i'm gonna call this beam and web make this a little bit bigger okay um implement web plugin and this web plugin i'm going to be pushing this up right away after it done committing and formatting and then once this is up there we can hopefully run this one inside ci and it will run it will not fail well that's always the wish i'm gonna push it up here and once that's done i'm gonna create a new pull request i'm going to do it all from the cli so let's make sure that this runs formatter should run awesome and then hub pull request against branch main and this will figure out what changed and it says okay implement web plugin so we can save this here now it did create a pool request and because we have github actions on pull request we can now say hookum and this will figure out and it will now run so you can see you can create a new branch create a pull request check out your implementation here and if this is green we can even go as far as merge it right away so you can do your whole github workflow from out of the terminal and this is definitely something i recommend you're looking into because these are the these are the time savers this is things we do all day every day so yeah definitely interesting there and so this is the basically the same thing of adding a custom adding a custom [Music] adding a new plugin to an existing workspace so the way we do that is by using this nx plugin package and take the plugin generator give it a name give it an import path now another thing that this nx plugin generator has and we can list this so we can say schematics list schematics that's in the name of the plugin this will give us all the schematics so this is not what i expected um another way to do it which is the more manual way you're using vs code by the way there's great plugin nx console that should list them out actually for intellij editors there's a community based plugin which is great as well i don't have it installed right here but the manual way of actually looking at what you have available is to go into narwhal nx plug-in in your node modules and look at collections.json here you see okay we have a plugin we have an e3 project you can add migrations in your projects and generators and executors so if i want to have an extra generator inside my web project i should be able to say i want to create a generator give it a name library and this will most likely fail but it doesn't know where it needs to generate it so it will ask me which project do you want to generate it in which is the name of the project for the generator in my case that's web and this will generate a new generator with the knife library so this is your workflow once you have the plugin created you can start adding more generators to this plugin and this will create them right here inside packages oh let's open it write up packages web source generator and sure enough it added the library here so this is the recommended way of adding stuff to your package i'm gonna go ahead and get rid of the changes by the way so i'm gonna stash drop my changes so there won't be won't be any changes and i'm gonna go back to ci to figure out why this doesn't work so we're gonna look at the failure here hopefully it's something simple i always hope it's something simple and inside this build let's see where it is failed so actually the e3 right so the thing it's complaining about not a valid object main now this is something that we've seen before not a valid object main and this has to do with the default branch so this used to be master and only master but now it's even says it usually master nx kind of expects that this is a master well it's not and what i think i need to do is in my affected change what branch i'm comparing against so what i'm going to do here in affected e3 is say lars already mentioned this if you're doing affected you might have to do some magic to make it work this is exactly what i'm trying to do so what i can say is base and i think let me see if i got it inside i'm looking at this side of the screen because i got a clipboard history where i tend to to to save this stuff you got already pasted this in if i need to tell it which is my base so here my base is remotes origin main and i think this might fix it but i also need to do another thing because origin main is not being checked out by default and i need to make sure that i check it out but i think this fetch depth makes sure that i'm checking out the right one so i'm going to make a commit here and this commit is fix set base property for for setbase property for affected command and question from lars here yes we can set uh remotes origins remotes or germain directly and this is something that as actually preston showed me when we were looking at how to improve the speed of your github actions but of course this is not only limited to github actions this would fail in any rci and it will solution will work in any other ci as well so right now i did a push and you can see recent push didn't trigger a workflow this is because like it has to you have to give it a second for these things to start at least that's how it should work so sometimes it takes a little bit for the github actions to kick in i actually also need to push that really helps and push you're never going to get the workflow there so push it and then we can run the hookum workflow and this should now work as expected so right there pretty sure that it will start right now there you go okay cool so this will most likely fix our issue and sure enough i need to actually if i want to do the same i need to go ahead and do the same for test same for lint i think actually all of my affected ones basically just going to go ahead and do it and uh yeah this effector is super powerful and as uh the larger your repo is the the more it it gains you in time uh in my previous nxpm attempt i didn't really bother with it because i just had like three packages and uh they were super small but in this one i the goal is to extend it quite a bit and this will really help us prove this so i'm actually going to amend this change and not edit it and run it again push it up again so pushed up here and as you see this is already running so there's one workflow running right now on my current branch but when i push this up it will trigger another one it will not allow this because i need to push force and force picture on feature branches now as you can see on the top top part of the terminal there is one running already it's stopping the container looks like it failed again yep filled again so let's see what happens there and circle of life for developers to do stuff with tests not a valid object name main okay i think this has to do with how i do checkouts yeah this has to do with how i do the checkout and it's this affected test that fails because i said i want to test first so let's see if this next action that i started actually addresses it and we should see that fast enough so what i really wish this hookum had and i don't think it has that it lock all the outputs i don't think it cannot do this that will really help in in debugging and once that's there i can go ahead and basically migrate functionality so i need to find an issue with the fix maybe lars while he's here knows what the issue is so when i run the generator here when i run this generator right from the application and let's start hooking here so i got an eye on what is running there when i run this spec it will most likely work and then when i run it in debug mode it will not work and i'm oh there you go it already doesn't work so somehow it cannot find dot slash schema which is sit right next to this generator and i can tell why because this generator is here schema set set right next to it it can't find it it can include it ah he never used lars never uses debug mode okay fair enough not a valid object name main okay so i need to go ahead and figure out why this works and larx uses wallaby.js i never use wallaby need to check it out i think this has to do with main and hat and and actually honestly in order to to get this right i need to watch back the stream i did with preston it's already been a great reference for me to figure out this stuff what i think we can do is say run many e to e and do it in all and this is not an ideal one in terms of performance but this might be the quickest way to get at least a green build at this point then optimize the build time later so i'm gonna go open package.json here and instead of doing this i'm going to go ahead and say run all e3 do the same thing with test so running test running e2e gonna do this a last fix attempt and see if that fixes it lars suggests i have a command where i didn't pass it in i think i passed it into all of them in any case i'm going to give this a go fix use run many to run test and e to e i really like to see this one green so i can wrap it up but that's where you need some special git commands like in the link i pasted seven minutes ago in the chat i think these are not visible in the chat links don't get passed in sadly because youtube doesn't youtube filters out links generally which is kind of sad probably because of spammers recent push didn't trigger a workflow well that's because i didn't do a push and then we can see if this one works so the idea with nxpm at this moment i need to figure out how to get tests working i got a few things to check out but it's not interesting enough to do that on stream i can't see your i can't seem to see your links lars it doesn't doesn't get printed feel free to share it to me at another place but uh youtube sadly is not the place um so the goal for me is figure out get my test working ah github.com okay i can i can do this i think i can copy it let me see if i can put this on the screen right here nope okay so it's the ng worker organization so let's go ahead and go to that one i know where to find that one ng worker and x dash worker sorry nx dash worker and inside this project so this okay nx worker nx worker and ng workers different projects okay cool i i didn't realize that one so inside here they have a ci dot general and this is the one that's linked and then it's line 19 to line 20 8. so that'll be from here to here so if github event name is push use the action this one if github action is pull request use the action and do this ref and i think this ref is the thing that is missing if event name is pull request which is the same condition as this one do a fetch nutex depth 5 origin main curious why you use 5 here as a digit i think this is the one that i'm missing i'm going to go ahead and check that one out i'm currently not organized enough to be able to quickly move to the code base that i shared with preston because we have this fixed there as well but i'm going to go ahead and add this one right here and then scroll back to my repo inside the actions and figure out what was wrong now always when i okay i love this i pasted five from somewhere i don't understand what it does lovely this is basically how i do most of my software developments i just like learned it somewhere know it works why bother so when i when i start a new project i always think like okay this one's gonna be have only like green workflows this can only green ci runs never works that way it always fails at some point okay narwhal angular has now exported member application generator so this is actually a failure inside package let's go ahead and see why that is so inside packages and then inside web source generators application generator.ts and this should already fill okay interesting that they don't ship this they don't ship an application generator for the nx plugin now this is something that's kind of unexpected so we're here in narwhal angular source schematics source schematics and then application is what i'm interested in and then application.ts and then i'm curious if they wrapped the schematic here oh this is the whole application that they generate application generator so it should be there from normal angular which package did i install angular let's start at version 11. interesting but yeah i'm pretty sure that it isn't it didn't get um doesn't get exported here so this is uh this is an interesting thing so what you can do is basically do what they do so you can wrap around generators yourself so what i can do is pick this one this snippet and this will be my application generator so instead of importing it here i'm just gonna copy it over this is my application generator and for this i need to import the rep angular dev kit schematic and this is your best bet into migrating old stuff over to the new stuff so from schematic generator you wrap them in then where you use them you pass in the options right about here now this is the last go i might have said that before but um push them both up and fix application generator for angular i'm not too worried about the git history because i'm going to move this code into the other code base anyway so we'll have one history and it will be a refactor but i always like to clean up and i love to start projects from a clean slate because it gives you a chance to rework some of the things that you've done and basically revise the divisions the decisions you made at some point so the goal for me as i said earlier it's going to have application working sorry api working generating sjs api get my websites working so i can generate my angular front end and get the stack back together so basically all the things that are here authentication core functionality which is mainly configuration stuff and data access the e3 test should be added then the web this is going to be a big chunk so i'm going to have like authentication as a feature so this these three will be one feature of this will be feature core feature dashboard and a few miscellaneous ones and then have this initialized script and this initialized script this is currently the big one this is the one that is running it all so this is for instance where i do my prettier this is where i add a docker file to the project so this should be abstracted out as well docker ignore docker compose and uh this is where i run the gist of my application so um this will be an interesting one to migrate but i think once i got the basics running the rest of it will all be the same so arch gives me a new import path which is probably the right one that i should be using so let's go ahead and close all this go to the web application generator and so it'll be normal angular source schematics application then application okay right patient.ts and this gives us the application generator yeah and you know no longer need that one nor the wrap application schematic i hope they made a mistake in exporting it because for nest for instance you can export it right away so i think this might be an issue on their site that they don't expose this thing i think this is a pretty neat syntax get the application generator from normal nest and then in the future when i publish my lips this way you can say in your project hey i only want to use the nasds api from nxvm you should be able to install it just this way okay one more time one more filled run here and yep one more filled run and let's see what this is causing it's kind of rematch funny finds i'm not gonna look into it any further right now i gotta jump off in a little bit i got another talk to do so i'm gonna go ahead and wrap things up here today so i'll be back next week i'll hopefully have a few issues here that i addressed so i can really show you how to build it's been a pleasure having you here thank you all very much for watching bls23 follow me on twitter if you don't or subscribe on my youtube if you haven't already really helps me out and i see you all in the next one thank you all very much bye bye

Channel: beeman‎.‎dev

Views: 497

Rating: 5 out of 5

Keywords:

Id: eg7juPShUAE

Length: 98min 20sec (5900 seconds)

Published: Sat Jan 16 2021