Matthew Croughan - What Nix Can Do (Docker Can't) - SCaLE 20x

hi and welcome uh I'm here with Matthew I'm going to introduce him to and this is the talk uh what can what NYX can do uh so what is next what is next OS you heard of it what does it actually do the answer is everything using next we can reproduce anything anywhere anytime bring an ancient ham radio packages back from the dead cross compilation to any foreign architecture including bootstrapping the compiler itself isolating packages from each other at runtime Building oci images building embedded Linux image system images deployments you name it Nick's can do it some have even taken to Colleen NYX the art of containers without containers in this talk Matthew will demonstrate all of what he's learned and show you what NYX is capable of in a live interactive demo you can take the blue pill the story ends you wake up at the Shell and believe whatever you wanted to leave continuing to run installers mutating your system in a blissful ignorance not questioning whether perhaps your package dependencies are incomplete accepting that one version is more than any person ever needs never worrying how to get here all over again you take the next pill you stay in Wonderland and Matthew Will Show You How Deep The Rabbit Hole every participility goes and without further can you hear me yeah okay fantastic okay so NYX is big and I can do everything so I'm not gonna be able to get through all of it today but I'm going to be looking at my notes and I'm going to try and get through as much as I know and there might be a bit of interactivity and feel free to raise your hands and like ask questions and things during the talk I'm not going to go on forever a lot of this is going to be interactive demo based so yeah it's just interactive little tutorial almost so what can next do oh well who am I first why do you care what I have to say so I run next Camp which is a training event that runs every year and my name is Matthew Grogan and I like using next because I started off with next by using things like Docker first and realizing that it wasn't very reproducible um see I started with an embedded Linux company and they were using Docker in their pipeline they would tell people not to use the yocto or build root based build systems because that was too difficult and they gave them Docker files instead so we made our own little platform and told them to just look at the docker file make a Docker file that does a thing make a Docker compose file it does a thing and we'll run that for you when the device boots and before long this just got ridiculously complex and it was crumbling and I decided to go and do next instead so that's my Twitter handle if anyone wants to go and look at it so yeah this this is the the pictures of nextcamp which we run every year it's a lot of fun you can check the blog post I was at next.house blog and then the URL that you've got there so here's a Show's history of next where does next originate from the first next commit was by Alco before the thesis came out in 2004 the first papers on next were released and the next thesis in 2006. nixos was instantiated on 2006 as well um let's Jump Ahead to when I got involved in 2021. so in 2021 we had 1 700 contributors and 280 000 commits next year we had double that a little bit more and in just like a month between that last bullet point 2022 there we now have 5 000 contributors and 460 000 commits so that's quite a race of change so who uses it and do you know any of these people I mean Target's pretty obvious um arm of course arm user for their hypervisor that they're making called um icicle I think it is yeah and um iohk Andrew is a military company and rapidly of course I've seen a few people here walking around with replica shirts on yeah it's pretty cool so these are the people who are brave enough to say that they do use it today but a lot of companies actually view it as a competitive advantage and won't tell you that they use it which doesn't work in their favor because then they can't find any talent to do next um so the former Docker VP James Turnbull funds a company called flocks and the GitHub CEO Thomas Duncan also funds that company as well so that's a Nick's based startup that recently raised a few million and the network here of course is running next it's only nine percent Nicks it's not a secret trust me so uh so the repos on GitHub anyone can take a look and we've only got nine percent Nicks now and the repo's got a lot of Legacy in it a lot of ansible a lot of old stuff but uh yeah um it's running next now so what does NYX actually achieve these are the buzzwords that get thrown around right a lot of things claim to do this but do they actually a yaml file that I have in a repo doesn't tell me enough information to reproduce anything NYX actually achieves real infrastructure as code and it achieves real behavioral reproducibility of components right it also achieves software supply chain security and all disability and it allows you to make an s-bomb for your dependency chain and a lot of people forget this but next is actually just an executable right it's it's a C plus plus program I can give you it I can statically link it and you can reproduce everything that I've done with my next code on your machine and it's only around 20 megabytes I can send you that file you can mark it as executable and you can do next build next is also a programming language so next is the program that evaluates the next code and the same way that python might evaluate or interpret the the python code in dot py files but NYX is also a package manager as well and it has features for I don't know just saying like next Shell and I'll get a program temporarily just for the moment and it also has this trend of compiling things from Source you don't have to but this is the large Trend in NYX is that you want to compile things from source so what's next packages next packages is a collection of 90 000 packages now right this is where all of the recipes come from we call them derivations this is the next code that reproduces Python and g-lib c and Firefox and build everything from Source if this didn't exist I couldn't X build anything okay so it's just a repo on GitHub and it contains these 90 000 plus reproducible recipes and it is automated as hell there's Bots everywhere running rampant and automatically doing everything some people on the IRC channels don't even know when they're speaking to a bot they'll say why did this person do this but then they find out later that it actually was the boss that was automatically commenting or providing a review and we'll see some of that later so uh this is kind of hard to see on this display here but at the top right is next package is unstable and at the bottom left it looks like we've got uh open Susa and slackware right so this is a graph of freshness and size so on the top right that means that you've got more packages and they are more up to date on the bottom left the opposite is true and ubuntu's in the middle somewhere so what is Nix OS then so we've defined Nix next packages NYX OS is something that is built from those two things so it's built from the next language and the next package collection of recipes on GitHub and I can Define my own NYX OS with my own stuff in it my own behavior I can say I want to enable Firefox and I want to enable like node red or grafana and then I can do a next build on that next code and it will result in something that if I run it will make all of that true so therefore it's completely declarative like ansible or Chef want to be right and the reason I use next is because anyone should be able to compile my code that's it there should never be a case where you can't compile something on your machine but it works on my machine it's the same the clients is the same as the server it's like git right if I get push or get pull it like the server is not doing anything special it's the same as the client and that next brings this approach to package management CI should be the same as my laptop and they shouldn't need to have knowledge of how my code was built they should just be able to build it without looking at a readme and if it's broken on my machine it should also be broken on your machine next wants to break everything but it also wants to make everything work as well and this is why we need next if things were simple we could just go the old way and just do things in the traditional Unix manner put something in slash bin and then when you've got a new version replace it but software isn't that simple anymore so we need next because software is this complex on the bottom right is actually a figure from the next thesis written by Alco dalstra and that's the dependency graph of Firefox and I'm not sure what it is on the bottom left but it looks really complicated is it yeah so it's Jupiter someone says so did you so what do I do with next I just Define all my systems in it and you can look at all of my systems and you don't need SSH to see what users I have because it's all defined in code every element my system the way it behaves the way that my laptop is the way that my phone is the way that my servers are I run a mail server this way it's all in GitHub public for everyone to see and I also I've just recently started this project called nixified AI which aims to make running AI software this new ecosystem of Open Source AI easy to run with a single command so there's no reason that it shouldn't be that simple it's just that the dependency chains are so complex you need something to manage it and that's what next is so this is a a tool called invoke AI which is like a alternative to Dolly and you can just next run this anywhere and it will work as long as you've got an Nvidia GPU on the host machine so if you run that command whether it's in the windows subsystem for Linux or if it's on a real Linux machine it will have access to the GPU it will compile everything from source that it needs to but it will also be cached at the same time because I've already built it pushed it to a cache and then the next client just says that guy's already built it and then it pulls it because I've already built it but if my infrastructure disappears they can still build everything from source and this is the first image that I made with it because Ken Thompson of course is the hero we need but not the one that we deserve so what are people saying about next dolman seems to think that we're going to disrupt and commoditize a lot of markets and products I'm definitely here to make that happen this guy says that the same place Amazon was conceived made the new new starts off called flocks Mitchell Hashimoto says the problem with NYX is that when you use it you lose touch with other people's realities and the National Archive like Nyx because it helped them reproduce an old version of my sequel that you could never get compiling from Source in any other way and look Perkins likes it because he only needs one tool now he doesn't need all of these tools anymore you can do it all with next instead so this is where the presentation kind of ends because unfortunately no one can be told what next is you have to see it for yourself and that's a picture of alcohol dullstra holding what we call a next pill which when you swallow it you will never see the world in the same light ever again so if anyone's got a laptop they can just SSH into this and they'll be given a virtual machine whether it's on your phone or whatever right you just SSH into that the password is I hate next and then you get a reproducible virtual machine and I'm going to show you what that would look like now hoping that it's up by the way if too many people connect it will crash so uh be careful this is all running locally at the conference yeah the password is I hate next I'm about to type it anyway so if I click enter if type I hate next I'm going to get a reproducible virtual machine right anyone can do this and you'll all get your own VM to play around with Nixon okay so the first thing I'm going to show you is how NYX works from a high level and I'm going to come down because next is kind of a high level tool and everything it does is quite high level so I don't want to go low level off because you'll never understand it but if you just understand what it does alternatively to things like apt Pac-Man traditional package managers you might get a greater understanding so like I said next is a package manager so and it's also a language so I can go into a raffle I can say load next packages but what is next packages what is that it's a path so let's see what's inside of it it's going to read me in it and it's got the contributing MD so that must be the same thing that we looked at earlier it's this repo on GitHub okay all of those recipes so let's do that again let's load next packages which is just a path to that place and let's say python3 Python 3 is a derivation and we're now evaluating that derivation and it tells us that that is the the path the hash of the derivation so if we say that we want Python 3 of a slightly different version it's still called python3 but the hash is different so we know that something must be different about the inputs we know it's different and this allows us to have six seven eight hundred doesn't matter however many versions of Python 3 with different packages available to us we can have all of them at the same time so just because it's cached and um it'll be faster I'm going to go on SSH into my machine at home in the UK that'll be a bit laggy but it's gonna be faster to show off the presentation so I'm going to load next packages here as well I'm going to type python3 dot with and I'm going to tap complete that with my tab button it exists but if we actually go all the way back to the periods it says that we've got lots of like nice functions like in any language and we can go with programs and that's oh with packages and that is a function and it needs some arguments the first argument we're going to pass is another function inside of p is all of the Python libraries that you could possibly want right so P list P Dot numpy this right here is an instance of python3 with access to numpy and if I put Jupiter in its place that's an instance of Python 3 where Jupiter available in its path and I can build them both I can say let's build that one with Jupiter and let's build this one with Dunphy let's exit I don't have python in my system I don't want it never works I don't want pip in my system either but this has just allowed me to eliminate the need to have them so this one this expression this next expression and inside this this derivation output we have all of the traditional FHS paths file system hierarchy standard directories like slash bin slash Etc slash include slash lib Etc right and this python can import numpy but it can't oh well it's unexpected maybe go the wrong way around let's do that again so oh sorry that was the one with Jupiter I mixed it up okay so this one can access Jupiter but it can't access numpy right and we do that Rebel again load next packages python3. with packages a function p and then a list P Dot numpy this will yield an instance of python that has access to that so let's get in this one can import and unpay but it can't import Jupiter right so we've got isolation but we didn't need special kernel features for this we don't need containers for this all right so yeah that's just a quick demo of isolation without the need for containers so I noted that NYX puts everything in an absolute path in the next store right that slash next slash door slash hash and then the program name right that allows us to have any amount of any program that we want no matter how complex um let's have a look at my next door right this is many gigabytes of every instance of my system that I've had over the the past few months right and I could do a garbage collection on this because my system no longer refers to these These Old Paths anymore so the python that I have now is not the same hash as those old pythons so I can just get rid of stuff over time whereas on a traditional system you might have python3 and you might install another python through you might install another python through and things get left around nothing works and the system gets crofty over time but in next I can just do a next collect garbage and it'll start deleting all the things that are no longer referenced in my system so the things I'm not asking for get removed I'll show you a bit more of what I mean about that so in this micro VM since we have it this uh this thing that we can all SSH into if we want it I'm just going to show you what it looks like in Nix OS to deploy service typically of a uh a configuration.x and a video confim configuration.x I'm not going to have them so I just need to get them real quick so we'll get them real quick and I'll bring in all of the dependencies of them that we need um any questions yeah so we're gonna we're gonna express some of that so the question was can I declare a kernel like a whole Linux kernel with different patches and different things just as easily as I am saying I want python 3. right with these libraries installed and the answer is yes I'm going to look at some of that so yeah so every every input of them the source code everything ends up in that hash so if the source code changes the hash is different if one of the dependencies changes its hashes difference if one of the dependency source code changes it's all this recursive function yeah yeah that's right so let's do a simple thing and enable a web server right services.nginx.enable equals true and then we'll do a nyxas rebuild switch and what I asked for is going to become true and if it needs to it will compile nginx from source but it doesn't need to because it's going to look up has anyone else built this yet and if they have then it will just pull it from The Trusted third party which in our case is the Nyx OS infrastructure cache.nixos.org okay uh this is running on the scale Wi-Fi so it may not be so fast oh not yet right so I guess in the meantime while this is doing its thing we're going to define a Raspberry Pi image okay because I'm bored got no one else to do so let's look at some of these examples here this Raspberry Pi Nexus example I've got a configuration.net I've put a lot of other boilerplates in here to make this work so simply right both this configuration dot next here right I can say what I want this Raspberry Pi that's in front of me to look like when I boost it off the SD card right I'm going to be able to build an embedded operating system just from this code that you see on screen so on my Raspberry Pi I want to like I want to enable like gnome I don't have a gnome desktop so I'm going to go to Google I'm going to say how do I do that next sauce no all right so this is how you enable an X server and tell it that you want to use gnome and gdn and again it will compile all of those things from Source if it needs to so let's add that to the config and then let's say that we want to put blender in the system packages and run blender on the Raspberry Pi not sure if the GPU will handle that but let's give it a go okay and let's also say that we want it to be connected to the scale Wi-Fi so networking.wireless dot interfaces yeah yeah um what is it scale let's get okay scale public fast and the password of penguins right my username yeah I'll just keep it like that so you can see that actually works and SSH is going to be on let's next build dot hashtag tab tab images dot pi and that let's let that chug along for a bit and it says well we can't do that because gnome users network manager so it prevented us from making a bad image that we would have to debug and test because it failed at evaluation time in the next code before it even hit the device okay so let's say networking Dot we don't want to use network manager dot enable equals false and now we're going to make an embedded image that doesn't have network manager but does have all the things we've asked for okay so that's going to chain away in the background again this is opening on my laptop it says blender's broken on arm 64. so that's just Grace isn't it I think free card Works let's give that a go how's our micro VM doing okay so nginx is available and we can see the nginx is available here but what about something more complex than a web server well has anyone ever heard of the password manager bit Warden let's go and deploy that um I'm just going to go copy it from my existing next config because this is also a trend the next you can just copy paste code from other people's configs everything's portable and shareable and reproducible so I'll just copy that and if it needs to again it will compile bot warden from source and everything it depends on rebuilds switch how's our image doing so right now it's actually pulling all of the dependencies of free Cut and gnome and things like that and it looks like it's gonna download 411 megabytes and that's going to extract to 2.4 gigabytes of which seven jobs are being worked on seven derivations seven recipes are being built um yeah oh no sorry that's three are being built six are being fetched from the internet yeah so you got like a nice progress bar and stuff like that in the meantime let's have a look at what all the demos we got yeah container images okay so if you need to you can actually build a docker if you need to you can actually build oci compliant images with NYX which is a better alternative to a Docker file because a Docker file might say at the top of it from Ubuntu latest which guarantees that the next time you're on Docker build you ain't going to get the same result okay so let's push it to a registry and put a hash on it so the master because it's like from Ubuntu and then a hash but you can't actually reproduce the docker file but built that you just got a golden image you've got a gigabyte lying around true reproducibility is when you only need to store one kilobyte to reproduce one gigabyte you've got the build instructions in a text file and you do build on it and then you get the one gigabyte that you were expecting right so that's what next kind of achieves right next OS configurations that's when you can declare things just as I have here right so Vault Warden is up so let's go to it looks like it's on eight two two um I'm going to use bore to tunnel in this is like an alternative to engrock and we'll get back a port number you guys can all hit this if you want and you see that bit warden's there but doesn't that make State and stuff like doesn't that make files on the disk and it's nasty stuff I can just get rid of the majority of what that just did by commenting the code out right I'll just comment it out and then I'll do a rebuild switch and everything that's related to that component that Warden is going to disappear all of the users all of the system D services everything's gone and I want to bring it back then I just put the code back next turns an operating system into nothing more than an executable program in the same way that you would get C code and you would compile it and then run the binary it's just here my whole operating system into that same thing modifying a file in ETC manually with Vim is like a go-to it's harmful you should have programs generate these files for you especially considering it's so complex these days and there is no standard there is no standard configuration format which is why we need something like this so Vault warden's gone all the services have gone and everything's cached in the in the slash next flash store all the objects we wanted are all there right all the paths so we bring it back by uncommenting it and doing a rebuild switch it's going to come back much faster this time doesn't need to do anything all those build outputs are are in the next door already it's all cached so how's our Raspberry Pi build doing well it's building the X4 file system image that I'll flash to the SD card with DD shortly and then I'll pop it into the pie and then I'll show you all or through obs and it should pop up with gnome and then when gnome pops up I should be able to run freecad and all the things I've defined should be true all right so what else have we got let's pick pie flakes okay yeah flakes so have you ever had an experience with your colleagues where they're running something completely different from you and they can't get the same result as you like they're running an old version of npm so when they run npm they get a different result right well let's fix that so I think I'm going to do everything as a simple flake a flake in next is something that has all of its inputs and outputs defined clearly so I'm saying that I want to go and get next packages that repository of recipes um 90 000 of them which includes.net Java anything you want postgres any any program you want to build from Source we can do that so let's define a Dev show for the x86 architecture and let's give it an argument MK shell is a function from next packages which is going to be given one argument build inputs there's going to be a list of packages that we wanted to give someone reproducibly so let's say in here some unlikely things right and by the way if you want to know what packages are available you go to search.nexos.org and you'll be able to browse them all right so let's say JRE the Java runtime environment so what about the jdk okay so it's called jdk packages dot um dot net this will never work and what else is unlikely um I don't know I think that's probably enough isn't it has anyone got any ideas for unlikely packages yeah that wouldn't work something that's really difficult to install what's that VBA opencv let's go opencv ECC doesn't look like anyone's bothered with that okay so what this is unlikely enough right so packages.opencv right it's gonna have to compile some of that from source and fetch it so we do an x-flex show on this it's going to download next packages and then it's going to extract it and in fact this is so slow I might just copy it to my main machine back on let's do that um yeah just wormhole send fleek.net to my remote machine cool let's do a Netflix show on this okay it's already got that version next package is a cached it let's next develop on this Dev show it needs to download 481 megabytes and it's going to extract it to 1150 and we're going to get the open jdk we're going to get the Java uh stuff we're going to get dot net and we're going to get opencv and anyone who runs this command regardless of where they run it what laptop they run it on are going to get the exactly the same version of all of those things right and that's really what Nix is about is about making sure that everyone gets the same version of everything um oh yeah yeah so um like I said I've got to look at my notes from time to time there's just too much you know you can do everything so let's take a look at the old way of doing things so if anyone liked Resident Evil there was an old game called outbreak on the PlayStation 2 but the PlayStation 2 is old Hardware it's got old open SSL ciphers in it so you've got to compile in order to run this reverse engineered server for this old game that someone's reverse engineered uh you have to compile Apache with an old open SSL and Link it and these are the instructions to do that let's get started so the first thing to do is to double you get the source code right and then we extract it we get in there we run dot slash configure we run make we run sudo making store cool that's probably going to work but how do we know what the hash of this file is they could have just changed it and we wouldn't even know that the contents were different so that's a mistake a Docker file will also allow you to make the same mistake but next won't next is a reproducible build system it's a domain specific language for defining reproducible builds so what that means is that it won't allow you to make this mistake it will warn you it will say you you can't just fetch something from the internet without telling me what a Char sum is right and so yeah so now we've got to compile Apache so we've got to get these things along with apache's dependencies and then we've got to do the same process for all of those things right and that's probably going to work right um oh wait no it's not going to work because there's a pseudo have to get update and lib pcre3 doesn't exist on my system because no one told me what version of raspbian he was using so none of this is reproducible these instructions are meaningless and always have been so how does this look in next how do you do the same thing that he's trying to express and next so I packaged it in a flake and the answer is Apache dot override open SSL the old openssl that's it that will do exactly the same thing as his okay that will build me an old Apache it will link in the old open SSL and we're happy so looks like my build oh wait no that's the next developed shell so now we've got the jdk uh how is it jdk I don't even know what the uh Java say cool so we got we got that so everyone's going to end up with the same version of java C compile from Source if it needs to be if all of the infrastructure of nexos disappears you'll still be able to reproduce all this from source group for Disaster Recovery right you don't want to be left on Mars without the ability to build from scratch right yeah the docker hopes down okay so what else did we get we got opencv underscore version there we go and we've got the.net there we go so we've got all those things but the moment I exit the shell it's all gone I don't have the I don't have a opencv underscore version anymore I don't have.net so it's clean as well because I don't have a lot of Croft lying around on my system and I have to factory reset it NYX and Nyx OS remove the need for factory reset you don't need to do that anymore no matter what state you're in right um okay how's our image doing still building that Z standard uh flashable image almost done though it's doing a lot of checks to make sure that the the data actually was correct oh uh yeah so actually I'm doing something called binary format registration in the Linux kernel so I'll show you what that looks like so in fact let's just get into cross compilation okay so if I want to cross compile a program in NYX all I have to do is say next build next packages hashtag packages cross dot tab and we'll be able to see all of the different architectures that we can compile for and if I want to compile for risk five I just say packages cross dot risk 564 dot the name of the program because we've got every next package or 90 000 of them instanced with their cross can power Flags adjusted for what we want so let's say hello world I built that last week so it's cached in my next door but if we wanted to rebuild it we just do this and it will actually compile it all from Source again right this is compiling cross compiling hello world and then I if we look at the results of that we can see that it's a dynamically linked risk 5x kissable but my kennel is x86 but I can run it thanks to this thing called binary format registration right it will spawn qmu as a user in user mode in order to run the program and provide the standard output to it to me and in Debian it's really complicated to do that so let's have a look at how you would do that in Debian um I've got I've got a project called Next Blitz which is like Bitcoin nodes and things that references this so in order to do that same thing on Debbie and you would have to do this you'd have the app install bin fmt support and qmu user static and then you would have to do this and then you would have to do this and then you'd have to do this and this and that's like I can't do that you know I don't want to do that because I might screw someone up but on NYX and I'll show you this now on the micro VM that we've got link I'll next build next packages hashtag packages cross dot risk 564. hello and whilst that's doing its thing um I'll just show you the line of code that I've got in my config that does this so my t480 which is my laptop I just say I want bin smt dot emulated systems to I want to emulate these architectures and it will go ahead and it will compile qmu from Source in order to perform the function but I want which is to emulate systems at runtime so I can emulate all of those architectures just because of this one line of code on my nexos machine and due to bandwidth I can't really show you it that fast on this machine but I'm just trying to multitask um but these are real demos right like there's there's no limit to what it can do you know let's just take a look around my config you know it's like I say um I want my Fireball to allow these ports that's how I do it like so this is what I want my host name to be on the network I want to use systemd Network too and I want this behavior on my wireless interfaces and if I want to see what's available to mess around with the Nexus if I go to search search.nexas.org and type like benefit what does it do extra binary format binary form has to register with the kernel and that will Implement all of what the Debian documentation specifies here except I don't need to read any of it anymore and I don't need to know how it works I just want to emulate that architecture someone has given me 20 years of Unix experience for free right um I guess we can go back in time and like reproduce a really old version of MySQL just like determinate systems did okay so git checkout mix OS 18.09 next Rappel load the current directories next code and we should have access to like MySQL and stuff and the version of that is 10.6 that's really old right what are we at right now um 10.6.12 I guess it's not that different that's 2018 holy crap okay um post press 14.5 okay well whatever these things don't seem to change that much do they but like we can reproduce anything that we want um let's go back even further let's go back to 2013. oh that's too old it's not even that well that was when nixos wasn't even managed and get it was managed in subversion but yeah I'll just do 2018 for now okay um and let's build MySQL oh it's not shown as the log for that so probably there we go yeah so sadly the cache is so reliable like it's like a blockchain moving forward right like all of next packages is moving forward and get and everything's on cash.nexas.org still so you don't actually have to reproduce anything if you don't want to it'll just pull it from the cache um so we'll have to attend the substitutes is off to prove that that really is doing what I say it's doing [Music] um yeah still building the uh the embedded image ah right but we got our risk five uh binary so let's look at the result I can't run that right but if I just add one line of code to my Nexus config on this little VM that I've given you 've been fmt Dot registrations what was it I could just man configuration.nix to get the same results so I'll search that nextbus.org emulated systems and it equals a list of architecture strings risk 5 64. Linux yeah and do an Excel travel switch I'll be able to run that binary after that's done okay what else we got oh it's released so I guess we can do that yeah okay so we got the old um we've got the old Maria DB okay so let's get checking right let's check out that really old version from 2013 and let's build something um interesting thing about this is that um this hello world that we're building won't be able to run on the current kernel because the Linux kernel broke compatibility at some point so whilst we can still build it and Nix will actually it actually puts measures in place in order to um to pretend to be an older colonel so that I can rebuild this stuff so if we have a look at that you won't be able to run it that's that's what happens when you run a really old uh hello world in the current kernel well next can still reproduce these things and that's important okay well we're almost there on the embedded image it's taking a long time on this laptop I probably shouldn't have done it uh oh yeah rough sex pay this is an old windows program but I've made a package for it I've got a repository on guess so called hamnex and it's not much of a collection yet but it will be if I want to I can just run this really old wine program with next like this and here we go right so yeah so this is like an old wine program well it's a windows program that runs in wine and I made a derivation for it so I can just run it and this will work on your machine the same as it does on mine and you don't have to configure wine at all and I even made it I even made it put the contents of its state in a traditional Unix directory like dot program name in the home directory so you can rewire anything for next that you want um I just been reminded someone asked about the kernel earlier so there's this thing called nexos shell that I'm going to try out now written by someone again in next Coast Nix code and all you really have to do is specify vm.next and you can run a VM with the kernel that you want just the same way that you would Define it in an xos configuration okay okay so what kernel do we want to use do we want to use an older kernel because we can do that uh you can see that in in next packages there is a library of of Kernel versions oh I'm in the options that's why no anyone know what it's called oh here they are so packages Linux packages and then you've got all these different kernels right you can choose from any of them and if I want to use an older kernel I can just say so so I want kernel 5.5.19 okay so let's get nexo S shell which is a program written by someone else and let's run next to our show oh well looks like Linux 519 wasn't it doesn't work anymore so we're not going to use that um six two I can always go back to an older version of next packages though and get that to work and I'll show that off in just a moment I've got a language server and Vim which tells me when I'm not doing things correctly in next so this is going to build a whole virtual machine image and then it's going to boot it with the parameters that I've specified inside so this has just boosted me into an xos VM transparently um yeah and it's also passed through the current directory into there as well so it's kind of like not only is it a nickel it's an X OS shell so you've got a whole kernel though so the kernel here is 6.2 because I asked for it um again it would build back handle from Source if it needed to I'm the kernel on my host is 6.1.12 and if we modify that again and say we want 6.1 and do an extra shell it will do that too oh the uh it's because I didn't specify the state version in the nixos config but I don't really care about that oh because I didn't specify the version of like I'll show you I can get rid of it so um system.state version so you would specify us and then that would remove it [Music] um so again like because this is the full NYX OS I can say like system.state version equals 23.05 and I can say Services dot node Red Dot enable equals true I could do the same with grafana right and let's get into that next offshore um the state version prevents moving postgres versions up so for example if you define if you said I want postgres on this NYX OS and then you upgrade next OS you don't want postgres to come up with it because that would cause State corruption because the new postgres may not be able to handle uh the upgrade it might screw your data off or something like this so we prevent postgres from moving up automatically with that state version so it's up to you to change that number if you want to get the new version of postgres okay so system CTL status node red so node red is running inside of this nexos shell that we just defined um on on post 1880 there you go so node red okay our image is done let's flash it okay that should be pretty fast in the meantime our little micro VM just compiled qmu from source that's kind of hard to see because of the spam but we're now going to be able to run that wrist five binary that we compiled earlier in this micro VM we run file on it we say it is indeed a risk five executable and we run it we can run it well we didn't have the reboot to do that right we're just running the program because the Nexus modules are just that intelligent okay any questions yeah so let's let's uh let's go to the next raffle the question is how does the cash dot nixos.org get populated with the old GCC Etc and the answer to that is a mechanism called binary substitution so if I say packages.python3.width packages oh I need to load next packages first if I say oh on my laptop python3 dot with packages I'm going to give it a list P Dot numpy that hash is going to be different than with Jupiter so if I bind these to variables I'm going to be able to inspect them and see what's different about them x equals that and y equals the one with numpy why has this thing called an output path on that path is what next asks for on cast on exhaust.org it says has anyone got this path and if they do then I don't need to build it but the this recipe is what determines that hash I've asked for numpy but if I ask for Jupiter that's not the same thing it's a different output path right yeah so like I mean it's going to get complicated at this point because how you can power Python 3 is complicated but if I go to search on xbox.org and I click Source on python39 it will tell me how it compiled python 309 it's quite complicated but usually everything is just a call to this function called MK derivation okay so we say that we're going to MK derivation make a recipe and the P name is going to be python3 and then we're going to do we're going to get the source code from here and it requires bashing the build system so we give it Bash and we're going to patch it and we're going to replace some of the strings in the files and stuff and this is the whole build process encapsulated in fact I I might as well just show you this thing um yeah that's right so we have to go back to the old function so whenever I load the old next packages this file will look very different but it doesn't matter because it still defines how to build it and it will be able to fully realize everything it will be able to go back and get the old dependency tree because the dependency trees in front of us is the whole of next packages is the dependency and it contains all of the dependencies and we can build all of the dependencies Through Time okay um yeah let's see if our image is done flashing and oh yeah I'll show you some determinism morning soon okay so our images finished eight is ten gigs so let's hope it was worth it yeah uh great so I'm going to put this SD card in the pi I have put it in the right way up I hope I'm going to go through this screen I'm going to plug it in and this is why it's useful for embedded right I can just Define a Raspberry Pi that I want to exist in theory and it will blow up before it even gets to the pie so it will tell me that I've configured something incorrectly so the amount of debugging that I need to do is reduced and then it's going to have the behavior that I specified I wanted some things enabled I wanted freecad to be available I wanted known desktop I wanted all those things and it's just going to go ahead and do it you energy response or does it ship without me if I want to build an embedded device and I don't want yeah so this this image is going to have next inside of it but you're not really limited you you can do whatever you want with next given enough time and energy you can say I want to to use next to build a small config file or I want to use NYX to build a embedded system that doesn't even have like a like an init system you can just say I want a process to be there instead but right now we've got full NYX OS so my username my password that was what was in the template okay I'm this is the full system and I hope it's connected to the Wi-Fi as well it takes a little bit to launch gnome on first booth and there we are on the pie all right we built all of that from source 100 the derivation of recipe yeah so so let me just see if this is connected to the internet and then I'll get back to you on that the question was how difficult is it to build a derivational recipe for something that isn't already there and I'm going to show you in the form of the uh the contribution process so is that connected to the internet one on yeah it is so so all of those Wi-Fi rules we defined are in there as well so and we have free card right yeah free cards there but will it work that's the question this is usually uh oh yeah there we go it might crash on boo yeah yeah let's launch it in the CLI instead I think it crashes because of the memory size on the GPU actually I I think I ran this before yeah but that's hardware limitations NYX can't help you with Hardware the only reason a next build should fail is usually Hardware so you've got memory limitations and you've got the internet and that's really the only reasons that things should fail so yeah let's check back on that in a minute see if it loaded so you said um how do you package something that's not there so someone recently and I was I was hoping that this wouldn't be closed already or that someone wouldn't have written a package for it so I could do it on stage but here's a package request on GitHub on next packages for a program called CSV quote okay and then someone comes in it literally hours later and packages it for them so let's see what my version of this looked like so this is a simple C program so I just say MK derivation give it an argument name give it a version argument tell it where to get the source code from and then I have to do these patches because otherwise it doesn't fail the test so I'll just get rid of these things just so we can build it up right this is all that's needed to like compile some basic C code so if I save this file and do a call package on it I produces derivation we can build that derivation and the result will go in the next door but look it just failed why don't fail [Music] oh yeah right that's right because the original program CSV quote which is on GitHub the original program has a make file and the when you do an install on the make file make install it will put it in the binary directory and by default the binary directory is slash user slash local slash Ben so we've got to patch that out in order to reproduce this program so the way I did that was by before we build I would export that variable so that the make file operates correctly next is just abstracting things and saying MK derivation is just going to make build and make install for you right so we give it the source code and it will automatically do all of that in a functional language so now that we've defined that we want to export this variable the program is going to work all right and produce an output so that's like a simple C program and how you would build it in next that's the recipe right there okay so traditional I mean we can look at package requests on GitHub it's probably not worth packaging one right now is it a what we could help someone out tnp Reaper what does that do cleans out temporary files where's the source code it's gpl2 what's it written in looks like a simple program oh where's the source code where is it okay looks like the source codes here and written in Ruby no chance not gonna happen [Laughter] all right so yeah I I think um I'd have to sit down with you and you can come and like ask me to like package some of your things and I'll happily package them if you want to see more in detail about how that works yeah oh yeah VM tests I haven't even got to that yet so for every package in next packages someone may have written a VM test in nixos tests and then it looks like we've got 643 of them what do these VM tests look like their function calls that spawn numerous virtual machines and then test the interaction between those virtual machines that you've asked for and you can run the test on your machine and I can run the test on my machine and we'll get either the same result with all we really care about and that is like exit codes like did the program crash or not so let's run one of the tests type completion let's try that again sometimes next needs to compute things and I've got a very basic one for node-red and it's a function call that looks like this I mean this is a bad this is a pretty bad way of saying it but you could actually just say um nexos test and then that would usually do the same thing but they've chosen to do it like this in next packages just because it's like Legacy stuff um so it's a function that we pass an argument name and the metadata like who maintains this test and these are two virtual machines that we want to spawn within exhaust configuration in so if I wanted the clients to be able to Ping a node-red machine a machine that has a node red system D service up and running I could also say services.mysql.enable equals true and then that would build me a virtual machine that has my SQL enabled that I could test against okay and then here's the test script it will start all the virtual machines it will wait for node-red to come up it will wait for the port to be bound inside of the virtual machine and then it will test that kill can actually reach it so let's run that test with big logs oh it's cached it in true next fashion so we're going to rebuild that because someone already verified the test cached at nixos.org already built that test so that's spawning two virtual machines and I can run that same test on my laptop and the only difference is the amount of memory we have available really um yeah so that that actually worked but if I introduce a failure for example like 10 node-red off I get rid of it entirely like there is no config here and I build that again I can't rebuild something I haven't built there we go that will do the same thing except this time node Red's not there for the client to see so it will fail and there is failing in this case it's um it's probably going to hang okay there you go so that's around two virtual machines except this time no Bread's not there and it failed and you can get more complicated tests like I've written tests for lib MTP which is that protocol media transfer protocol when you plug the Android devices that you have into a computer when you want to browse the files on them that Library always breaks so I wrote a test that proves that that Library breaks and now NYX OS knows ahead of time when that stuff is broken and they can fix it by way of a patch or something like that um yeah ah let's let's have a look at how what a next binary looks like when we build it like what what the what is linking to so on a traditional system and this is the machine back home and the internet's not great here but hope this works um if we do which like LS what's that um oh right sorry I'm sorry about that yeah everyone everyone's got those weird directories uh you shouldn't pay too much attention to them there be dragons so we do like which LS we'll get a traditional file system path right and if we run it from the path it's fine and if we do ldd on the path we'll find that it's linked to other traditional Unix file system hierarchy standard directories but on nexos we don't have much in slash bin we only have that and if we do use a slash bin we only have that and we do LS lib we don't even have it so those paths can't be linked to by the dynamic Linker so if we build something on next what does it look like let's have a look at our results here okay let's ldd that well it says that in that case it's not a dynamic executable let's let's build something that is like the regular hello world okay so it points to uh traditional non-traditional Parts it points to slash Nick slash door slash the absolute path of the g-lib C we want which is very different from a normal system but people actually forget again like I said earlier that NYX is just the 20 megabyte C plus program that I can send anywhere so this machine is running Arch Linux a really old Arch Linux as well so I'm just going to statically compile next and then send it over to the art Linux machine and then run next on that art Linux machine and do some of the same stuff that I'm doing here so this is a 23 megabyte file so it's not quite 20. it's not a dynamic executable and I'm going to just send it over to the Arch Linux machine okay that's pretty fast and Market is executable and now this dot slash next is available to us and I can do all the same stuff I was doing like um oh well flake science enabled let's let's fix that shall we so it's going to go and do all the same things that we'll do on my machine except it's doing it on this Arch Linux machine which is not xos it's not an X next is just a program you can run on any Linux distribution and you can run it in the windows subsystem for Linux and it will work there too there you go although it did find some weird bugs uh there with pearl and the language but not being set that shouldn't happen but yeah yeah I sat in the setting the variables so we're in next like it's a very pure environment those variables are not set because on your machine they're different and on my machine they're different so next is not going to set them to any particular value it's going to give you a clean sort of not sandbox but uh clean environment in which the program is running which means you can find some pretty interesting books and some programs a real quick thing like because of Nick's architecture we it's got a lot of massive data for all the packages so for example hello.meta well we know a lot about it firefox.meter know a lot about it it's got a license right we know we know what it is is it free Firefox is free um Zoom us dot meta dot free dot license dot free false Zoom is not a free program so that means that we're uniquely positioned to generate something like a software bill of materials so I made what's called a bundler in Nexia and you can pass it uh you can pass it a few arguments you can say I want to like use my function on a package from next packages and it will generate you an s-bomb for it so let's have a look at all of the runtime um license dependencies in Firefox it created the Firefox report for me and these are all of the licenses involved in Firefox I wonder if they're all compatible are there any interesting programs you wonder whether they're compatible or not or like what the license stuff is inside of them at runtime no okay let's try Chrome that's an interesting one see how that works so in order to do that as the fetch chromium it's going to do a little bit of analysis and this is all written in Nix code I actually didn't write the next code I just copied it from someone else um I made like a neat CLI for it okay oh yeah and of course uh something that might be confusing people is that the ordering in next code doesn't really matter the order in which you put things in the code is not important because it's a functional non-imparative fully declarative language right there's if I put something above something else doesn't affect the output and uh well then we'll end the talk with uh with this runtime report and then uh I guess if anyone wants to ask questions ask me after or we can get some people up that already know next to answer your questions that aren't me okay what's chromium so these are all the licenses involved in chromium wow okay let's have a look at the word compared to Firefox so there's 208 licenses involved well 208 programs involved in chromium I guess this is really not a not a good thing to test just the word count of it and 194 in Firefox but you can start to do this sort of s-bomb analysis stuff with NYX if you'd like to because of all our existing metadata in fact other people are not next or next packages use this use next packages as a a source of metadata um and there's a few projects coming up that just use next packages in that manner without next yeah I guess that's me [Applause] thank you oh free card uh well gnome went to sleep now it's just stuck there the GPU isn't cut out for this you know I would have tried oh yeah it says there on the left cannot allocate memory so this is a firmware if you're on the pie the socials yeah um sure oh yes the Stock's still going yeah so I just wanted to say as well like don't use nexos it's a bad idea paths we're not supposed to have hashes in them it's just a terrible idea we've been played for fools Okay so the socials yeah like if you want to contribute to next packages it's the easiest thing in the world to contribute to right you just make a file you put it in the repo and then you say I package this thing and we go from there and there's five thousand contributors that's like a big project I guess it was a massive project so yeah get involved and we've got a discourse forum discourse.nicsoft.org which is getting more and more traffic every single day foreign question oh right yeah that that's a really good one that we did uh just in the knock earlier let's see if I can find it uh something to do with monardo open XR Etc um okay I don't even know where it was but I've put it in my directories uh oh no um right I'll just show you a very basic one right so let's go to the next packages and let's get like um mosquito right which is an mqtt broker in next if we want to override something like any of these attributes in purple like the P name the version the source code where the source code comes from all we have to do is specify it in an expression so let's go to our flake here that we've been building up throughout the presentation so we got our Dev shell from earlier let's just get rid of that deviant say x equals packages dot the thing we want to override that override attributes put the old stuff in here and then we can redefine any of those attributes that we want for example we can say the source code from mosquito comes from somewhere else so we just copy this and replace the bits that we want and the language server is going to happily tell me that we've got knock off these things to find so we'll say packages dot fetch from GitHub the repo is mosquito as a string and the version is going to come from a different tag of the source code so let's say they're last release we want it to be version 2.0.13 right we want to reduce it by one little small version number we'll say that we want to be V 0 2.0.13 [Music] um uh yeah and we get rid of the sha256 because we don't know what it is yet and we'll do a next flake show here oh we didn't let that run earlier did we let me um Wormhole send this flake again to my remote machine which is much faster than my laptop yeah well let's take a look at that so it's it's expressed in the same way so so we're overriding the version number here let's build it and then we'll change the compiler flux afterwards this is going to build the old mosquito with the same process the same build process is going to be applied to the Old Source Code right now that we've got that chart 256 that Nick's gave to us we know what the shower is ahead of time and it's going to build the old mosquito it still calls it 1.4 but it's not we need to change the we need to modify the version attribute okay so that's going to build the the old mosquito with the same process and we can change the C Flags if we want to mosquito c-make Flags so we can turn threading off just by saying that we want to the cmake flags are going to be the old cmake plugs added to a new list which are our new cmake Flags my custom Flags so that'll now show up inside the build process in fact I'll probably tell us somewhere that it wasn't used um if we want to log at our bills we can just say next log on the output as well so that's so we can say see my custom Flags is in the build process though oh sorry there so it's part of the build process now just by adding it to that functional declared package foreign so that's like my customers key so that I can maintain and I can add flags that I don't agree with sorry I can remove flags that I don't agree with um I like I can even say this like I could either Define a new list that does not contain threading or I could just say lib dot remove this string from the existing list so it's not the old ones plus my new ones it's removing this string this element from the old one so we want to say we want the old cmake Flags but we want to remove that that one so you can compose and Abstract all this stuff in a functional language and make it easier to understand um because I can I can read that much better than I can a Gen 2 e-build file right it's a it's a it's a language it's not just a bunch of variables that get passed around between bash functions right oh uh I said earlier that I could inspect some systems without the need to for SSH right so I'm going to show that off now so if I load my flake for all of my systems so my name on GitHub my next config I can say what have I got here I've got some Nexus configurations I've got lots of them I've got my mail server I've got my Matrix server my laptop but what users are on my laptop dot config dot well let's check if opengl is enabled first opengl is hopefully enabled it is and I can tell that without having to go through a bunch of file system parts on my system in principle this next code is going to produce a system with the behavior that I want so I can just ask it what is the behavior going to be so there's less debugging involved I don't have to go traversing file system parts and looking at what the system at runtime is doing so more checking Theory craft the system in this domain specific language almost algebraically before I even put it on the machine right and I can say what users are in there so we've got users dot we've got systemd Network the next build users Matthew is in there Geo clue avohe as a result of all their configuration that I've got in my system these users exist but this is before we even get into running the system it's all static next OS is when you go into a system and you can say oh where's my micro VM gone you know I'll just spin up another one so next OS the question was what is nexo asked if I can just get the static next binary and put it somewhere else and just use next so next itself is a build system a programming language it's the main specific language um and a package manager but nexos is something that's built from that in the same way that Arch Linux is built with Pac-Man the package manager so in NYX OS I can say things like [Music] services.nginx enable equals true and I can read that will become true when I do an Xbox rebuild switch I can't do that on any other Linux distribution just by copying NYX over there is a module called home manager which allows you to achieve some of that because most Linux distributions are systemd based so I can give you next and then you can use home manager and then you can kind of have this experience on a non-nix OS distribution but it's much nicer to be able to have these interactions with the operating system I can control like the GPU drivers this way I can control the system D services that exist and there's a whole like library right of those so search at nexus.org grafana I want to enable grafana and these are all the options I have to play with grafana I can't do that with just next I need an X OS in order to achieve that and that's what the difference between Nix OS and Nyx itself Nicks can run anywhere it can produce the service files it can produce the config files but it's not going to put them in place that's up to you to do or you can use nixos and then nixos will take care of everything for you let's say I copy the binary and they can work system software that for example like you see the answers that the question was can I use the static next on next in General on a red hat system or an old Fedora system or something in order to play with new packages is that right or old packages like really old stuff that runs on the next three and stuff yeah you can do that yeah okay oh sure so let me let me show you you just reminded me to talk about Docker for a little bit longer okay so uh so I mentioned earlier that you can use next in order to build oci compliant Docker images and then load them into Docker but you know I don't have Docker enabled on this machine so I'm just going to enable it real quick I'm going to go into my config for my laptop and I'm going to put virtualization dot Docker dot enable equals true into my config and then I'm gonna Nexus 3 will switch and now my laptop is going to have Docker on it in just a few moments let's have a look at because it doesn't have Docker at the moment right like I'm just going to make that happen with the Nyx OS configuration which is capable of controlling everything go ahead what's up oh yeah so in in the virtual machine that we specified earlier if you just SSH into this machine you can do next minus shell minus P vim and you'll get them oh no I'll show you right now so we're in the VM we can just yeah you've got a Nick shell or a next M A nyxm there's kind of like the traditional imperative approach of doing app install thing apt install thing app install thing and then when you reboot your operating system or install it from fresh those packages will no longer be there because you fail to write them down but in next if you just do the totally declarative next code approach you don't have to worry about that you can just take your packages anywhere and keep everything going to get repo and stuff but you have access to this old command Mix end which is really supposed to be used for anything for by people it's supposed to be a program that is programmatic that you can use in a piece of software that you want to write that interacts with next uh yeah the correct way is either to use a Nick show or to put it in your next OS config yeah if you want to keep track of this stuff and guess right if you wanna imperatively install things and lose that data when you factory reset then you can use an XM but it just depends on what you want okay so we're just about to get Docker installed on my machine and then we're going to make a Docker container with Nyx okay so we got Docker it's been a very very long time since I've played with this so I forgot about like the permissions and stuff uh okay so let's build a Docker container with next so it's a simple function call to a function called Docker tools dot build image and then we tell it what program we want to run at the entry point and this bit in Brackets here is going to be fully built from source and realized we call it realization when we put paths into the next door it's going to be brought into existence by next so let's choose a more complicated program like mosquito [Music] so this will fully resolve like this to the following path it will look something like this when NYX is done evaluating and Computing this it will look like this in reality slash oh well vims just try to load the next door paths because of my plugins so that's embarrassing yeah so so it'll look something like that um so that that's the resolved path so that thing in purple in the brackets is just a substitution for that and by saying it in curly brackets it's actually going to build different source right so let's build this okay start going ahead and creating the turbo that we're going to load into docker and we're done so now we've got a path to the next door that has a tar ball in it and if we load this turbo into docker and it's going to run mosquito so it's like taking it from one run time to another runtime from next to Docker so we build the software with next and we run it in docker um because it keeps me in the same directory oh I've never used that before okay yeah what's it say punk rock oh yeah that's another thing I forgot to tell everyone it's like so NYX is a bit like punk rock because the Unix granddads are not gonna like it they won't understand it and they'll hate it but the kids are gonna love it [Applause] how do you run something in Docker again oh god well it tried to run the program but mosquito can't run as nobody so sorry what was that I am I could show an example of using Nexus rebuild switch to do that so let's do that um so I've got my mail server and I've got a lot of other things in here what am I going to mess with today oh I I forgot to mention that I build my phone my Android phone with Nyx in the exact same way and I'll talk about that a bit more in a minute but so so I've got a machine called H1 and it doesn't have Docker on it so I'm going to enable Docker on it by adding to this next code here virtualization.docker.enable equals true right so a H1 is a machine in my configuration it's going to take a little bit to evaluate this but yeah so H1 Nexus configurations.h1 it exists so let's deploy that to the machine in the real world but exists so if I SSH into H1 it's going to take a bit because of the internet boom we can see that the current system is some random hash you know P6 F5 but I'm going to deploy this new one and the diff is that we enable Docker online 18 here that's the difference between what we are on and what we're deploying so all we should have to do is next off rebuild switch minus minus flake the current directory the name of the machine and set the target host to Matthew at page one should do yeah indeed that that seems to be the case yeah that will be the case although I do have a deployment framework and I want to use it because I this is low this is uh lower level but I've got a deployment Library called nixonate which will just let me do the following instead and it just runs Nexus rebuild switch that's all it does because this is how I do it right I just do next run that and it will copy the next code to the remote machine and then it will use that code to do everything right so all it does and this is the full implementation of the script as it does next copy to the remote machine and then it does an excess rebuild on it and that's it that's all it does so shortly Doc is going to be in this machine we see Dockers not available yet on this machine that we're deploying to here we go so it's getting all the stuff it needs for docker a rootless kit Docker container d a Docker container D still the question was why would you want to use ansible when you have this and the answer is actually on the nexos redis there is a there is an answer to this so I'm going to show you what the answer to should I use ansible is let's talk all time ansible foreign [Music] okay so Docker is now enabled on that machine that didn't have Docker before right I just deployed it it's there now right and we see that the current system doesn't have the old hash so I do a deployments it copies the next code to the remote it does an exos rebuild switch yeah back the way that I would roll it back is by taking away the code that's managed and get so I take I get rid of that line all right the next run on that next command again um and it's going to be much faster this time so getting rid of Docker we've still got docker okay the deployment and we're going back in time Docker docker let's keep running it and Doc is going to disappear there we go job done yeah and I get rid of all the users associated with Docker the group Docker is gone all assistant D services associated with Docker are gone it's a clean system now so we can go forward and back and on my laptop for example if I do that same thing doing accessory will twitch over and over again it creates Generations in my bootloader where I can go back so it's all it's nexos will automatically version it kind of like get you to it every Nexus rebuild switch is kind of like a git commit so you can always go back to the old commit the old system um garbage collector old Generations no because I have a nice line of config which configuration limit so I don't have to worry about garbage collection because I can say you can you can only put so many uh configurations in to the bootloader yeah thing it's a function in next packages yeah yeah so that function so yeah so this is a function that takes two arguments the first argument is the name and the second argument is the config no this this uh this implementation of this does not use docker yeah probably oh so now we've got packages we've got docker tools dot build we've got lots of other uh fancy functions and features in here that have nothing to do with Building images right yeah yeah I can do Colony build next packages is the library so that collection of recipes that 90 000 packages also contains all of these Library functions right so the definition is is here yeah so let me show you some language examples so in the repo if I say x equals one y equals two I could say y plus X but I could also make a function that's just making values or attributes um so let's say double is a function which takes a single argument it already exists as a function now so um let's just say add together and it will take X and Y and it will add X to y and now I can give it two arguments one and two and I'll add them together right but I could also have a um a parameter set which is better than this because this is positional so sometimes I'll put things in the wrong order in this case it doesn't matter but it can matter so let's say add together set well ARG set equals an attribute set which is just curly brackets um a equals one this is an attribute set a is the attribute and one is the value of that attribute in the set so let's say that we're going to take um first thing and the second thing and that's a function which adds the first thing to the second thing now we can't just say one and two because it expects a set of key value pairs so for example first equals one second equals two and now that will work right yeah exactly they won't matter where the position of the argument no longer matters there's like a whole functional language for all of this stuff well uh yeah yeah I think I do uh yes yeah yeah uh let me think about laziness um things oh thank you it's hard to find a good example of laziness because in the rapple the laziness behaves differently so [Music] um let's see if I can do this so let's say less in x equals 1. and Y equals z in x you know the the this language server is complaining about why but I'm not uh will it crap out no it says undefined still um yeah I can't really give a good example of laziness I'm not really that familiar with it if I'm honest so it's probably not worth asking the build build image oh oh you want to see the implementation of it okay probably get this file open in a better editor one sec yeah well that that's why you should get off you should actually get up and just show people I'll say I'll do that in a moment so what did you want to know uh the implementary okay it's quite it's quite a big one build image args I caught the return of this is crazy isn't it so is it going to use augs it's just like a big fat build layout image oh okay it uses run command at the end so dockatiels build image it will extract the base image create the layer but how does build image get used no it's a function it's a function that when you call it and give it various arguments is uh going to behave differently what's up and now it can't be running the docker CLI inside of a derivation or without you yeah yes it's a really crazy one to debug though it's amazing yeah this function is incredible it's like a very very very very very complex function it's probably like I don't understand it either but I'll show you a simple function that can you can you can think about so we're going to the next Rebel and load next packages I do run command which is a function it expects a few arguments it expects a string name of recipe or derivation because they're the same thing um empty function call we don't want to pass any arguments just use the defaults and then the command that we want to run inside of a NIC build so we're going to Echo hello two dollar sign out this is a derivation and if we build it it will give us a next door path yeah but this actually ran and built the derivation I asked it to build it ran hello an echo sorry it ran Echo inside of a sandbox and then produced an output that ends up in the next door so if I cap that it now exists and it's got those contents in it but I could just as easily say give it some build inputs like um Firefox minus minus version into dollar sign out and that will actually build Firefox from Source if it needs to give me it and then allow me to run it inside of a Sandbox and produce an outpour from that so if I build that it's going to get Firefox and then it's going to put the results of calling it with that minus minus version in in the dollar sign out although I forgot I need to put because this is just the path of the next door and needs to be in bin Firefox it's inside of the bin folder [Music] no it's just it's currently right now is taking up bandwidth and actually building something yeah it's not very good at reporting the the status of the build inside the raffle really I wanted to do this properly it'd be like I could do it in the command line in a different way but let's stay with a smaller program or perhaps even do it on a machine on a better network connection oh maybe that's why the internet seems to stop working is it okay all my connections are dead I forgot to say let's get into a micro VM on the local network which actually works and then there we got the 80 character limit in qmu it's not going great this is a oh I still like a program like mosquito okay oh mosquito doesn't have the version option right but it ran that in the sandbox right so minus V and now we get a we're probably going to get a build output that is the result of building that what's happening now all right all right let's stop using the raffle and use some like a file or something okay that's going to produce the same result except we can now build it with the command line oh it's building something with the same drv what oh when you launch mosquito with minus V actually spawns it as a demon and it doesn't end that's that's what the problem is uh so we actually just ran the the program as a demon in the in the in the sandbox yeah yeah that's correct it just didn't report anything it doesn't give me the standard out when I'm in the rebel it's kind of like a black box when you're in the rebel so you can do anything you can uh mosquito what about came here no it's not like systemd it just allows you to bring any program from next packages and run it inside of the sandbox and uh offline and guarantee that it the thing that you're running doesn't have access to the internet or any other inputs that you're not in control of everything in this file defines uh where I'm getting stuff from so I'm getting qmu it's being built from source I have qmu now I'm going to run it offline in a sandbox and I wanted to give me an output and it can't give me output that I don't have control over because I've got control over all of the inputs I control where the source code comes from I control how it's built I'm going to get access to it offline so I can have the pure computation right now it I mean it comes up and down right so GCC right now is at version 11. it looks like yeah you can do that for packages um in fact I made a nice pull request here that allows you to change every library in the system to use any compiler flux so if I want to compile every single program and library in my system with different compiler Flags I can do it like this right I can say I want every program in my system to use O3 just like this by adding it to my config and it will have to build everything from source so let's do that um [Music] maybe my big machine configurations deploy that boom now every program is not going to work because all programs do not compile with these options they fail right so I'm just going to copy the next code to the remote and then it's going to build that and this is going to take like three days so yeah yeah next Camp we'll probably run this year I I haven't announced it yet so if you want to know more just go to next.camp and see the websites and stuff and check out the blog post right this guy's like coding a flake while eating a flake it's kind of like and this is the next sandbox it's very secure some good pictures here the traditional drink of next Camp is Nicks and kicks we can't get enough of that instead of Club mate we choose we chose this is what the food's like uh the place in Wales called the astral ship and we add some uh some nice times in the Quarry and stuff and like just enjoying nature and this is uh the results of a intense RFC debate and what does that spell Knicks yeah and then we decided to go up to Snowden in flip-flops like some nice pictures and we did it in our next lab coats as well so there's a lot of culture going on here a bad culture because NYX is scientific it's reproducible undeterministic which is what you want really in science you don't want computer science to stop being science because then you won't be able to get the outputs that other people do because it will all fail in the middle because someone made a mistake somewhere and didn't tell you what they used on the day that they made the result and here's Rob where's he gone oh he's gone yeah yeah we had a lot of fun it's not even the end yes so in the UK we have these things called flakes as well but yeah that's uh that's next Camp if you want to come along just thank me oh networking's gone don't know what's happened there yeah I think so yeah you go complete control over everything you have control over the way the candles build there's no it's not hard to change things it doesn't take forever it's cached with yocto and Builders if you make a mistake it destroys your compiler cache you have to remove all the cash and start again and it takes forever to evaluate whereas this is actually I I ran everything today on my laptop which is not even a modern new laptop it's it's pretty old you know get started

Channel: Matthew Croughan

Views: 8,660

Keywords: nix, nixos, ken thompson, linux, devops, ansible, kubernetes, podman, containers, gitops, git, nixpkgs, deployment, software, technology, reproducibility, developer environments, infrastructure as code, scale20x, scale 20x, southern california linux expo, SCaLE20x, SCaLE, What Nix Can Do, Docker Can't, Docker Can't Do This, Docker

Id: 6Le0IbPRzOE

Length: 124min 53sec (7493 seconds)

Published: Sun Mar 19 2023