Brave New .NET - Mark Rendle

hello welcome to the second talk in a row and I've got another one this afternoon and I'm doing a pub crease this evening and I'm doing pub Kampf on Friday if you're not get it was going to PubCon are you want to go to pub come it's just it's fantastic watch all the proper speakers make Complete Idiot's to themselves yes come drink mainly drink actually it's funnier if you drink so yes Barry's new dotnet you may have seen my my other talk about why I'm not leaving dotnet and this is where I'm going to dig in more into new Nets and don't make core and asp net core and what's good about it and the new stuff that you need to learn because dotnet is now a proper cross-platform standalone modern open source programming language environment framework platform thing and that means that there's a whole bunch of new stuff that you need to learn you can the good thing the good thing is that you can forget all the Ras stuff and all that XML web config stuff and all those PowerShell commands you can just forget all of that and learn nice easy bash commands and Jason configuration and maybe some llamo awesome any files and environment variables and that sort of thing because we have new net we don't have old mates anymore old net was big and slow and it only ran on Windows and if you look on the tech empower benchmarks with old dotnet it was doing all somewhere around 23,000 requests per second for the plaintext benchmark it was not quick it was easy it was very easy to build applications with old dotnet but they were big and slow and they were quite expensive to run because you had to have Windows servers and sequel boxes and all this sort of stuff and now we have donate core which is light and fast and agile and this is a Thompson's gazelle which is in the top 10 fastest animals on earth because I didn't want to do the cheetah because it seemed too obvious yes and don't make core cross-platform runs on Linux and so now we can spin up Linux boxes and using terraform or vagrant or whatever room we can deploy our net core code and we can just pick up the entire thing the CLR and the runtime and the SDK and we can just bundle it all up into a zip file or a tar file and then dump it on a Linux box and it'll just run and pretty soon like I don't know if it's in this year or maybe early next year but you'll be able to do native compilation you'll be able to use core RT and do that flash t : link native thing and just create a single native executable file that is your entire web application and just pick it up and put it on a machine and go run that and my web application is running and so there's all this new stuff and all these new things that we need to learn and of course we're developers and if there's one thing that you can say about developers it is this we fear change we're not good with it we don't like it when you move our cheese and so I'm I just want to kind of go over some of this stuff and and show you some of the really cool things that you can do with new nets that people who've been doing things like node and Ruby and Python has maybe been taking for granted for all this time this so we'll kind of cover the basics and then I'm going to talk a bit about docker and then I'm going to talk a bit about micro services vicars dotnet call an asp net core a great for building micro service architectures much better than traditional net when i was still doing traditional net and asp net MVC and web api five point whatever the hell it's on and having to deploy into iis app pools and so forth and people started talking about micro services and I was kinda like that sounds just really complicated and why would I want to do that once you get into lightweights standalone executables that you can just deploy and put in docker containers and you get a proper nice DevOps continuous integration continuous deployment pipeline this stuff starts to make a lot more sense but then once you start creating micro services you go ah hang on there's a whole bunch of other stuff that I need to start worrying about now because I've got this from running over here and this one running over there and need to worry about partitions and performance and all sorts of other stuff so I'm going to talk a bit about that kind of thing so but so to start off with new things things that are mainly different so the big difference for an asp net core application is that it runs on Kestrel and Kestrel is Microsoft's new net based webserver it's a console application so when you create an asp net core application you've got net new MVC it creates a console application and it's got a program CS file and the program dot CS file creates a web host and then runs it and then that carries on running until you press control C or kill the process and so this introduces there's no iis you can put it behind diet if you run it on Windows and you install the iis module iis is actually just acting as a reverse proxy to the kestrel web server so there's no none of that integrative pipeline you've got in asp net 4.5 or 4.6 so this is just a console application so the configuration story has changed and one of the things that you start using a lot more with asp net core is environment variables and an alternative configuration sources so we're going to look at some of those and so i've created an application this is an application that i use when I do my workshops and we build this up over a couple of days and it's it's incredibly silly but it's just very useful for sorting out the stuff that we need to demonstrate excuse me I've got a frog in my throat an expression in Norway I'm guessing probably not so this is my application rocket stop I'm getting ahead of the curve here I figure that when Elon Musk's start sending regular transports to Mars that's like a six month journey even when Mars has have its closest point so I'm going to hook some asteroids and put them like every month along the route to Mars and then the Rockets will be able to stop and people be able to get out and use the toilet and that sort of thing so I'm getting a hit I haven't worked out the asteroid bit yet but I figured I'd start writing the software to manage these rocket stops along the way okay so I've got my rocket stop application which I need to exit PowerPoint and then do the old tap thing so here is rocky stop this is running in Visual Studio code and I've got my rocket stop web application here with my docking controller and my home controller and that's that's my sort of MVC application and I've got an API application here which is a micro service and this is handling the the docking data okay now running these things as Microsoft so in if you do this in traditional net you end up with two web applications and in the kind of properties page they'll each have a different port and you run it on localhost something and localhost something else when I'm working with a dotnet core application and a s being that core application I have to run them both from the command line and they will default to running kestrel defaults running on localhost five thousands if you don't tell it differently so oh and this is this is asp.net core two as well where's my CS proj file so yes there we go so you see I'm targeting net core app 2.0 here and the really nice thing I don't know if you've seen this yet if you went to the 20th Damien or David's talks you might have seen this male there is a single method package that you have to reference called Microsoft or ASP net cord all and that just pulls in the entire set of asp net core so it's logging configuration dependency injection MVC razor the whole lot the nice thing is that now when you publish your application even if you publish it in standalone it runs an analysis over it and works out which dll's you actually need and only bundles those in so if you don't use razor it won't pull in razor and that sort of thing so talk a bit more about that later ignore the other things that are there at the moment but I've got so I got my neck core application there so if I want to run that I need to find I need to be able to tell it that I'm running in development mode because I've got some code in in my program CS here so this has got a lot shorter as well this is now just create default builder on web hosts so all those things that in ASP met core 1.0 and 1.1 you have to create your configuration and hook these things up this create default builder just does that for you now and so it looks for a file called app settings JSON and then if you set the environment that you're running in it'll look for a file called bat app settings dot whatever Jason so you have the concept of a development environment a staging environment in the production environment okay so two sets to tell it it's running in development I set an environment variable so I'm going to say this is Windows command line here so I'm going to say set ASB met core underscore environment equals development okay and then I can start this application up by doing dotnet now I can do don't let run and that will just launch it and then if I change something I just go back in and I kill it and then start it running again but there's something called dotnet watch that you can get which is very useful you say dotnet what run and it keeps an eye on all your c-sharp files and all your JSON files and any other files that affect the build of the application and if any of them change it kills the process and rebuilds it and starts it running again so it's very useful to include that in your code all you have to do is pull in this line here so it's a it's not a packaged reference it's a dotnet CLI tool reference and it's microsoft.net watcher tools and the version is two point zero point zero dash preview one dash final because dot Microsoft version numbers which I will be explaining at PubCon it's numerology so yes the one above I'll talk about a little while later on that's just there by default so that means that I can say dotnet watch run and that will build my application and start it eventually come on this is a state of the art quad-core 16 gigabytes of RAM laptop okay I do I need to get there is there is no such thing as a faster one than this laptop and never get one of those massive Acer predators right so the other but I've got my micro service and I need my micro service running at the same time so that my main web application can talk to my micro service and so here I'm going to set asp net core and the score environment equals development and I'm also going to set asp net core underscore URLs equals HTTP colon slash slash localhost colon 5001 okay that's fine it should still be working that's just something that's not there yet that I'm going to put in later on okay and so then I'm going to say dotnet watch run here as well so this is going to run on port 5001 and this is starting up and so this is using entity framework core again entity framework core 2.0 to talk to a sequel server database which is where it's actually keeping track of which space ships are docked in which bays so there we go connecting sequel connected running the migration come on come on come on come on come on so let's find the so I go to localhost 5000 find out whether that's at so there you go there's there's the web application running welcome to moon this is the rocket stop on the moon obviously that's the easiest place to build the first one and if I go to slash docking that should have right so yes there you go my migration ran my initial seed data was created and I've got my web application on port 5,000 talking to my micro service on port 5000 and one okay now when you build a micro service architecture the whole point is that you know if I had a list of Bay's coming from somewhere that would be another micro service if I had my catering that would probably be half a dozen micro services if I had stock control but I'd be like 12 more micro services so you end up with a lot of micro services and it's quite difficult to keep track of all the numbers and every time you want to bring up a complete set of micro services for testing or working with you've got to go and have that many console windows open and just run this and run this and run this and run this so that gets quite annoying quite quickly so this is sort of how you do this from the command line you set your ASB met call URLs and then you've got to try and keep track of what's running where and everything else which is quite annoying and that is why we use docker so is anyone using docker already for their deployments and everything else so docker is one of those things where people go oh docker dock is great you've got to use docker and you just go why I don't understand the point of docker and then you use docker for about how many times can i say docker without officially coming to a paragraph in my speech you use docker for a couple of weeks and then you just go everything must run in docker how can I how did I ever live without docker so what docker is is just very very small virtual machines okay it's actually not virtual machines it's processes running with their own pretend file system that they carry around with them so they have their dependencies bundles along with them and they're run in their own kind of isolated process space so a thing running in a docker container can't see the other things running in other docker containers and it can't see anything on the host machine unless you accidentally tell it it can or on purpose ability can sometimes but usually accidentally and so but they all share the same process space they're all running on the same kernel so you can run dozens of them all on the same machine I've had like twenty-something docker containers all running on this machine at the same time because they are just processes they're just processes that can't see each other and can't see the actual disk for example the sequel server that my application just attached to is not running sequel server on my Windows laptop because I like using my Windows laptop for playing Forza horizon 3 in my hotel room I carry an Xbox controller around just for that purpose and every time I want to play Forza horizon 3 I really don't want to have to bring up the services window and shutdown sequel server to stop it using 8 gigabytes of my memory so sequel server now runs on Linux and Microsoft maintained a docker container with the current age version of sequel server 2017 for Linux and I can run that just using this this little script here it's docker run in detached mode the minus D says running detached motor just start in the background and keep going until I kill you - II accept Euler equals Y which says I accept the end-user License Agreement set the system administrator password to secret squirrel with numbers I use that password for all my social media accounts please feel free to hack me and post nice things about go the - P 1433 1433 says link that port that network ports on the docker container to that network port on my machine so it now looks as though I have a sequel server running and listening on port 1433 and then that last bit there should say Microsoft / MS sequel server - Linux I forgot to put the Microsoft flash on there at the start ok so yes and I can prove that that's running in there by doing docker PS and you can see there that I've got Microsoft / MS sequel server Linux and it's been up for 22 minutes because I started it just before this talk so that was completely fresh pull and then my migration ran and that started up okay so and I can pull in I've got post screws I've got elasticsearch stacks and stacks pretty much anything you could possibly want to run these days you can get in a docker container so of course you want to take your code and put that in a docker container so you can get all that goodness as well and so we have asp net course so microsoft maintain what are called base images when you create a docker image you base it on an existing image so there's a node-based image and a python-based imaging there are dotnet based images and there's actually there's a dotnet core as a Microsoft flash net base image and the microsoft.net SDK base image and so you could download that and then shell into it and run the dotnet SDK inside the docker container without having to install dotnet actually on your machine the really nice thing these days is Microsoft have made Windows work inside docker containers as well and so you've got this it's kind of complicated so docker generally as Linux and it was using Linux II groups and namespaces and that sort of thing and then Microsoft when all containers those look good we should do something like that and rather than doing the typical Microsoft thing and reinventing all the wheels they worked with docker to make Windows work in containers and so you can get Windows Server Nano core and you can run dotnet core applications in that and you can also get Windows Server core containers and you can run full dotnet framework in that but you still manage it with docker so you still say docker pull Microsoft slash Windows server core and then run the load of stuff up things and then you can push that and you can run it in hyper-v and all this sort of stuff but everything I'm showing you today is running on Linux containers just because it's easier for me so there's a doc net core and a dotnet SDK and in there's an asp net core base image and an asp net core build base image okay those are two separate things until recently when I was doing my talks and my demos on my workshops we would just use the SDK image or the build image and the docker file for that would look something like this so here we're saying the first line of any docker file is from which is saying this is the base image I want to use to build my docker container so we're saying from Microsoft asp net core build work is essentially it's like CD but docker remembers it so every time the container starts up that will be the working directory and then the copy dot says copy everything from the current host directory where I've run docker build into the app directory as it is now on the docker image then run dotnet restore because you've got to pull down all the packages that you need to run your dotnet application your asp net application then run net bills and then set the entry point to dotnet run and so then when the container starts up it'll use that and it will just say oh in the app directory I'll do dotnet run and your application will start up okay the problem is that the asp net core build image itself is enormous it's like 2 gigabytes because it's got all the build essentials and the c++ sdk and all sorts of other stuff and the whole microsoft build pipeline and msbuild and everything else so it's enormous the other problem with that is that although you've built the application so when it starts up it is just running a pre-built application all your source code is still in the docker container and in the docker image so anybody who wants to can just pull that docker container down start it up shell into it and get all your source code out and frankly if I wanted to ship my source code I'd be doing node I I would rather build my source code and keep it all to myself so there was a hacky kind of way around this where you would have a two-stage what we called multistage docker build ok so you had one docker file that did the build and you would either mount a volume into that docker container or you would mount you would access the files inside that docker container afterwards and you would copy the published output from the build into another docker image and then that would be the thing that you actually ok but docker this would happened it feels like the standard way of doing it but in the next version of docker which you can get if you download the edge build from dhaka comm they have actually gone you know what everybody's using this pattern all over the place let's just support it out of the box and so now we have a multi-stage build dockerfile so now we can do this we can say from Microsoft asp net core build and give you the name so we give that image a name in our build process create a new worker and I'm going to call this SRC to differentiate it so this is where I'm putting my source code copy everything into it run net restore and then run net publish and set the output to slash app which will just get created and set the configuration to release so we don't get a debug build out and then in my sin the same docker file I put another frog and this is coming from Microsoft slash asp net core - okay so this is just the runtime so the build image is about 2 gigabytes okay it is big that's big for a docker file the runtime image is about 200 megabytes which is actually quite small and it's going to get smaller once they get this working on the Alpine thing that base image is probably going to go down to about 50 megabytes which is tiny and then on my second line in the second block I'm saying copy - - from build so go to that image you built in the first stage of this build and just copy the files out of there directly into here and then set work there's a flash app on this new image and then do the entry point dotnet and just run rockets DLL ok so and I'll just show you what the output from a publish looks like so just do a dotnet publish - alt dot dot slash dot dot slash count nope I do not know that hang on it's all right because I'm mucking it up at the moment yes it is right there it is there's the command that I got right first time while you weren't looking okay so this is an MVC application this is got CF HTML files this has got rays of using it okay which has always been fun for kind of building stuff so if I go to my out directory here which I've just created you can see that what we've got we've got a dub dub dub root folder so that was included in my publish that's on my JavaScript files on my CSS files and everything else we've got rockets DLL which is my application which is 24k and we've got rocket stop recompile views DLL and that is all my CF HTML files that have been pre compiled in advance and bundled up into a DLL so even the source code for your razor code isn't with your application we don't need to ship the views directory anymore which is pretty neat that is what this line here is doing the Microsoft Visual Studio Web code generation tools it pre compiles your CS HTML files into a DLL and deploys those okay so that's what's being created and then that's what we're copying onto our docker image from in our multistage build so if you want to get that it's going to be in the next version that is officially released but which was supposed to be in May still hasn't actually landed yet unless it's landed since like 11 o'clock last night when I wrote this talk it's not true but yet if you get the edge build of the community edition of docker then you'll be able to do this multistage build thing and it is very cool my favorite thing about it is that if you wanted to do a multi-stage build you actually had to use a proper complicated continuous integration process like Jenkins or keen city or something like that because you had these two docker commands and two separate docker files now you've got this multistage build within a single docker file you can just attach your github repository to your docker hub repository and set the automated build process so that when you do a commit to a particular branch on your github repository the docker hub will do a pull and docker build the docker file and then that just becomes the new image so that combination of github and docker hub just becomes your entire CI process obviously you still wanna run some tests oh yes okay so now I've got I've got my docker file in my rocket stop things here so here's my docker file for my main application and here is my docker file for my other application and the only difference there is that this one's running a different DLL down at the bottom here okay so I want to start these up and what I would normally do for that is I would do SRC backslash rocket stop so to build this manually I would say docker build - T manual RS dot okay and that'll go through and this is running it's running dotnet restore over the over the NDC shared Wi-Fi so everybody hates me now and it's it's creating this docker container it's done done the restore and slightly concerned that I've got Forza horizon 3 running in the background somewhere and then I would start this up with docker run oh you couldn't connect to the server oh that's good I may possibly be about to heat my life okay let's just kill that then right it doesn't matter because I've already got them built so that's okay right so then I would run that with docker run and if I go to docker images you can see that I have got an image here right so this none here is the one that I've just killed and you can see that's two point one eight gigabytes this is one I prepared earlier which is local flash IRS which is my local copy of rocket stop and that's three hundred and five megabytes which is pretty small for a docker container and you may be thinking you know what two gigabytes isn't that big for a docker container but bear in mind that one of the places you can run docker is on Internet of Things Micro Devices you can have like a Raspberry Pi model B and you might have a four gigabyte SD micro card stuffed in there so actually two gigabytes is quite a lot and there is a big difference if you're deploying to these things and Doc are really pushing on the internet of things use case as a way of deploying their code so it is it is worth saving off those 1.8 gigabytes when you do your build step ok so then I would say docker run - P 8080 colon 80 local slash RS and this will start up my my container and so you can see that's now running and it's listening on port 80 okay so the asp net core base image sets that asp net core underscore urls environment variable 280 which is obviously the port you want a web application to be listening on it's going to run all the applications in all the dr containers are all going to be listening on port 80 but they're listening on port 80 in the container not on the host machine that - P command that I passed in there so I've told that to around 80 80 didn't I so here the localhost 8080 and that's the code running inside the docker container so I found port 80 inside that container - port 8080 on my actual host machine it's nice and easy that way you can just publish all your asp net core containers you know that they're all going to listen on port 80 and then you can just do whatever you want with that when you're binding them to your actual host machine okay but now I would need to spin up the other docker container with my rocket stop docking service inside it and then find a way to maybe find that to a different port bind it to 80 81 but actually the process running inside the docker container can't see any of the stuff on the host machine so I have to figure that out so I am going to use docker compose okay so docker compose is yet another layer of abstraction if you like docker compose let's you say this is what my entire cluster of microservices looks like all the dependencies all the databases everything and all my services I'm going to spin up with a single command and you have a docker compose file which is a yeah mol file so I told you you can make your lonely app yeah Mel so this is docker compose and you basically set up all the services that you need and all the networks that you need as well so you create virtual networks between the different docker containers so that they can see each other so in this instance I'm creating a network called RS net rocket stop net and the driver is bridge if I was to take this and run it in production I would spin up a docker swarm somewhere in a data center and use something similar to this but I would use an overlay network which would allow all the containers on the swarm to see each other so my docker compose file is down at the bottom here and it looks like this can't see it because I've still yes okay got it right so this is my dr. Campos file so up at the top I've got my sequel server Linux this is going to spin up a new sequel server Linux box and I've got a rocket stop underscore docking and I've got I'm saying this is the image if you can find the image just run the image if you can't find the image use this to build a new copy of the image I've got the docker plugin for visual studio code which is why I'm getting these nice pop-ups telling me what the hell I'm doing which is very handy because I don't know I'm going to set an environment variable so asp net cause configuration builds up in layers so you've got your app settings jason then app settings development jason and then environment variables and then command line so i can override what's in connection strings docking DB by setting this environment variable that's a double underscore as the separator on those so that will tell the docking service to use ms sequel as its data source okay and you can see we've got the password secret squirrel there that data source name is the name of this service here okay so that's going to get hooked together the dns server that provided why docker on the bridge network when you look for MS sequel will return the IP address of that MS sequel docker container then rocket stop I'm going to set its services docking to point to HTTP colon rocket stop docking so that we'll be able to see this container once I bring that up and then down at the bottom here I've got a whole load of other stuff as well for influx and graph on ax which we're going to be looking at in a moment okay so let's stop that and will do docker - compose RM just to clean up any containers I may have had already and then I'm just going to say docker - compose up okay so that's spinning up all these different docker containers this is creating me an entire micro service cluster on my development machine okay and everything's running here and I've got a thing saying logging failed for user SA and failed to exploit there but then yes so you get into this interesting thing I'm going to show you some of the code that does this in a moment when you're bringing up this micro service cluster you might have your application starts and tries to run the migrations before the MS sequel container has actually spun up and started accepting incoming requests so you end up writing an awful lot of code like this where's my data docking context in it so because I'm spinning up a brand new sequel server instance every time I run this I'm spinning up a brand new sequel server instance so my database doesn't exist it's not like when you've got MS sequel whatever the application are coming where what the applications called studio management studio if you can't do that and create the database and then run the migrations and everything you having to run the migrations as part of the startup so I got this thing that retries for a minute waiting for the database to start up and I have to pull in the connection string and then change the database to master because obviously docking DB doesn't exist yet so yes there's there's lots of fun going on in there so we retry six times and wait ten seconds between each one but now I've got that up and running and if I go back to my docker file we can see that I told it to bind rocket stop to port 8081 so that should now be showing on there okay yes it is and if I go to docking and I can dock a ship and put it in base C and we'll call it Vogon constructor and dock that one and so that's going in there as well okay so that's everything is all just brought up and running and I can check this code into github and somebody else as long as they've got docker installed on their machine and running can do a git clone docker compose up and get an entire micro service architecture with all the dependencies up and running if docker can't find the image that's in the docker compose file it will try and pull it from the repository if it's not on the repository and you've got a build thing it will build it for you okay so this makes micro services really really really easy to work with except you've still got a whole bunch of other stuff that you have to worry about when you're building this sort of thing so I took a look once I started getting into this whole kind of thought works Martin Fowler buzzwords thing I thought I'd go and look at what's happening in other ecosystems that have been doing this architecture for a bit longer and I found Java has a couple of framework kit things called spring boot and Rock wizard which have got a whole bunch of bits and pieces but very useful for doing micro services go if you like that sort of thing my microphone feels like it's falling apart go has something called go kit and so I was looking into what these things have in them and the big thing that that becomes important with micro services if you've got one application and everything is included in that one monolithic application is very easy to know if everything is ok you hit the application and it's either there or it's not once you've decompose that into a hundred micro services you've got a hundred things to keep an eye on and so you need some sort of monitoring in there and there are various aspects to monitoring you've got to make sure that all these services are healthy that they're all running right that they're all working at the right time seriously this is falling off ok so you need something for logging and actually for asp net core for logging we've got logging there's AI lager factory and you can attach things to it and you can attach their log and log4net to it and then type the content onto as your table storage or a sequel database or elastic search or Splunk or wherever you want it to go okay so logging is taken care of but you also need metrics and metrics is not the same as logging logging is information about stuff that has happened okay metrics is statistical information about how long it took how often it's happening is that metrics are the sort of things that you can do lovely pretty dashboards from logs are the things that you go in searching when something's gone wrong to figure out what it was okay so you've got metrics and then you've also got health checks so you need to be able to go to a service and say are you healthy and that doesn't just mean are you running can I ping you it means if I ask you to do something to the database are you going to be able to do it so the health check endpoint has to say right I've got a date I know this is my database connection string when you hit the health check endpoint I'm going to open that connection and return okay as long as the connection opened okay if the connection doesn't open I'm going to return an error code and you will know not to direct any more requests to me okay insanely complicated it turns out these micro services and then you've got circuit breakers and circuit breakers the wrappers around the API boundaries between the different services and the circuit breaker pattern says you might sort of in background just ping a health check on one of your dependent services and say are you there are you there or you might have a circuit breaker that you wrap around an actual operation and you say if that operation fails twice stop trying it for five minutes and then pick up trying again so you don't end up spamming all your other services with potential denial of service attacks okay so these are all the different aspects of monitoring logging like I say we've got covered with Microsoft of extension logging I was went looking for for the metric side of things and there were a few ports of the drop Wizard metrics library the direct ports of the Java library and they've all various stages of abandonment and so forth and then I found at metrics and the website for that is at metrics two i/o and at metrics guy called Alan Hardy does this it's bonkers it's one of the best bits of dotnet open-source I have ever seen in my entire life okay so remember that address I'm going to demo the stuff that's in there now so we've got these are your metric types these are the types of data that you're going to pull out and the first one is called act decks and this is application performance index this is an industry standard and it says is this service performing within expectations you can set what those expectations are but it's like a user satisfaction index if you put this on your website it will say everything's okay if you're serving your kind of main web page out in under 100 milliseconds if it's taking over half a second it'll say no that useless if it's gone down so apdex is the dashboards that you use understand app decks and you can create dashboards based on it then you've got counters which is just how often has this thing happened or how how many times as this thing happened you've got gauges which is just a single number that changes over time so you might say how many users are currently active on the system would be a gauge so I've got you know every time someone logs in you add one to this number and every time someone logs out you take one away and so you can map that over time as well you've got histograms which is when you say so there's less than 100 milliseconds less than half a second less than a second more than a second and get a distribution of values over there you got a meter which is how frequently something is happening so how many times per minute and how many times does it happened in total and then you've got a timer which is like a meter and and how long it took at the same time so you can see this is happening this often in on average it takes this amount of time you can link that up with a histogram it's all incredibly complicated okay at matrix ads endpoints to your application so that you can pull this data out and the three simplest ones are slash metrics slash health and slash ping flash ping just returns pom and it just goes yes this process is running we're going to take a look at metrics and health but before we do that I'm just going to show you how easy it is to include this in your project and why I am so impressed with it so we have four references that I've added to my application the at metrics call this it doesn't specifically say it'll work with a net core app - but it will it's not touching any of the internal stuff or none of the stuff that it's touching internally appears to have changed in a way that stops it from working then we got the MVC extension which means it knows about the asp net core MVC framework and can hook into various things in that we've got an influx DB reporter which we'll look at in a short li and we've got the JSON format ER which is the thing that's going to hit these endpoints okay and then to hook this up it's a big chunk of code to hook the middleware up to do this okay that's it's not that it's this okay so we've got our configure metrics function here but the thing is this will be essentially the same for pretty much every application you create so I unn tempted to kind of create a new get package of my own that just has my standard set up for this has all the dependencies for those for things like that I can just install my new get package and just have this configure metrics the only thing that's really going to change from from use to use is where my influx DB is that I'm going to be reporting stuff to and my default context label which is essentially what I'm going to mark all my metrics with so I got this I got my service collection here which is where on registering my dependencies and I just say services don't add metrics I can set various options here so I can set my server to the machine name which will be the daka machine name when it's running inside docker the app is going to be my assembly name so Rockets DLL and I can use to get my asp net core environment and say either prod or staging or development then I'm going to add various types of output serialization text serialization health serialization health checks add the metrics middleware add the reporting to send it out to influx these are the influx settings which we'll look at in a moment and then I just say apps don't use metrics and apps use metrics reporting ok so it's a lot of boilerplate code but like I say is the same boilerplate code every time you can either copy and paste it every time you create a new service or you can stick it in there I've put it between the bit that says use the developer exception page and use static file so it's going to tell me how long it takes to serve static files but it won't tell me how long it takes to serve error pages I don't want it to do that because those error pages will catch errors and then the metrics won't get to see the errors so when you're doing this configure application it's in kind of reverse order so it'll do the metrics and then if it was an error it'll carry on reporting that up to the error page and then down at the bottom I've got my use MVC there okay so if I go back to my docking thing here and we go to metrics you can see that this is picked up this is just a JSON output for everything that's happened so far okay and if I go to health you can see my health checks that are in there now if I thank you so yes that tells me which health checks are healthy and which are degraded and so forth so if I've got monitoring I can be pinging that health check every five seconds and setting up alerts on batch boards or sending people text messages or whatever if something goes down and it gives you details on what actually has gone down so if you've got a micro service that depends on two databases and some kind of cloud service it'll be able to tell you which of those three things has become unavailable or unhealthy so you get your health check out of there to implement the health check all you have to do is create a class that inherits from the at metrics health tech base class it'll work with any dependency injection you've set up so you don't need to worry about where you're getting your connection strings or anything from and you can just say do my connection string and set the connect timeout to five seconds try and open that and if it works then return healthy if it doesn't work at metrics we'll pick up the exception and incorporate that into the error report on the unhealthy status okay so we've got our health check there just by creating that you can have as many of those as you want and it will report them all healthy or all unhealthy when you get into the thing and if we just do ping then you just get pong back which says yes this process is running no idea whether it's healthy or not but it's quite handy to know whether it's there at all so so that's fun but what you really want when you're doing monitoring and metrics and all this other good stuff is reporting you want lovely dashboard sending you want those things that you see on radiators and so forth in in like modern development shops and so forth now those it's easy enough to set it up we saw the the chunk of code that was pushing stuff out to influx TV an influx TV is the time series database is specifically designed just to record values over time useless for anything else fantastic for monitoring and metrics and app metrics will push stuff out to that it also push it out to graphite and Prometheus and various other sources if you want it to to actually look at what's in your in flux DB you need to use graph honor and so I'm going to go to Grif go back to my browser and I'm going to go to port 3000 because that's where I bound graph honor to and I'm going to go to admin admin and log in there and then I'm going to add my data source and that's going to be HTTP colon in flux BB through on 808 6 and it's not graphite it's in flux DB and the database is at metrics okay and this is going to be my default data source and that's added and that hasn't worked Oh - yes sorry in flux actually now let's call it at metrics I put the host in the wrong place right okay success datasource is working so now in my lovely new world I go to here and it says oh you need to create a new dashboard and I go how do I create it and then it's got this thing here and then I'm going to say I want to graph how do I add things to this graph graph on is quite complicated okay so now this is my favorite thing about this app metrics project the people who make graph on ax run a kind of repository where you can put templates for dashboards okay and the guy who's done the app metrics open source library has created a template specifically for the output of the app matrix library and so I can go to dashboards and say import and I'm going to ignore my changes there so I can either upload a JSON file which I've exported from somewhere else or I can get one from the graph on a dashboard and the graph on a dashboard so go to graph on alabs here and you can see we've got the app metrics web monitoring in flux DB and we've got a dashboard and it's identity is 2 1 to 5 so I can go back to here and just say 2 1 to 5 and it goes off and it finds it and I'm going to say import that and I immediately get this amazing like brilliant dashboard where it's saying things are healthy and if I go back to my rocket stop and I say docker ship and why have you crown oh that's not the right one that's why localhost 8080 1 and go to slash docking and then I can say docker ship and put it in Bay F and I can dock the Millenium Falcon in there because I don't care I'm going to mix Hitchhiker's Guide to the galaxy and Star Wars and if you don't like it you can well okay and then I can go back to my metrics dashboard here and I've got my overview thing at the top here that's showing these are how many errors I've got these are my requests per minute it's like a proper dashboard I just had to type into one two five to get it out and now I can stick this actually what I can do now is I can play Forza horizon 3 for the rest of the day and then at 525 I can put this up on the big screen and pretend this is what I've been doing all day so that's that's how I work does anyone want to hire me by the way I know you're mean so anyway and all the instructions to do that are on the site app - metrics - IO so that's that's the kind of that's the visual bit of what I could demonstrate the stuff that is also outstanding that you need to worry about and I'm going to recommend a couple of things to look at but I haven't got time for the demos because the rest of it was so cool so but yes the rabbit-hole goes really really really deep ok so there's a service discovery once you've got all these services up and running you need to be able to find all the different services it's ok for all the stuff you've got running inside docker whether you're running in swarm or kubernetes or whatever because they provide DNS so you just use the name of the service and you can see it but if you've got things outside like with 12 factor apps you're talking to win as your sequel database or a poster is there or something external then you need to put the URLs for that so you need a distributed service discovery system something like console is great that's from Hoshi core that xcd there's zookeeper if you really want to just piss off your ops people say get a zookeeper cluster up and keep it running that'll just take one of them completely out of the running so yes and for working with console and SCD there's a library called microphone that you can use which is a little bit abandoned but it works ok you just haven't updated it for dotnet core 2.0 yet the circuit breaker pattern there's a couple of libraries for that there's poly which is not just circuit bro it does the whole kind of exponential retries for kind of temporary partitions and so forth there's also circuit breaker net which is a more kind of literal circuit breaker library there are other protocols that you're going to have to start looking at using so HTTP is not necessarily the best most efficient protocols for services in a micro service cluster to communicate there are things like G RPC and thrift and there are libraries for running G RPC and thrift protocols client and server in c-sharp but according to David Fowler in the talk on asp net futures last night one of the things they're looking to do with kestrel is abstract the concept of HTTP away from kestrels so kestrel becomes like a networking server more like libuv and then you'll be able to run G RPC or thrift kind of directly on top of kestrel so that's going to be cool there's actors so once you get into this thing we got hundreds of micro services and I'm distributing a job and pushing it out and I want all of you to just solve different bits of this and then we'll pull it all back in again and turn it into a response so then you start looking at actor framework so you've got a code net which is a literal port of the Java akka framework you've got proto actor which is a kind of what would happen if we start a bit from scratch you've got all lens which is Microsoft's version of this thing and so you need to start looking at those things as well you've got distributed code so you need things like buses and rabbitmq and 0nq and Redis for storing data that's being worked on by a whole bunch of different actors so that they can all pull it out and push it back in again and create collaborated results the monitoring thing which I don't have time to show you which is Zipkin zip kin wolves marker request when it comes into one service and then you wrap it around your requests to other services and it creates a distributed trace so you get like a Gantt chart that says okay the overall requests this long but this request within it took this long and this request within that took that long and it's like one of those function timer things that you get when you profile a dotnet application except it's run across the entire thing it's based on a white paper by Google but it was actually created by Twitter and there's a fee sharp library for it zip King fournette is the one that works best at the moment so there's all this sort of stuff to come as well in this new net world and that's without even thinking that maybe now we've got this proper modern lightweight open source distributed code factory that we can use maybe we can create something new like the Java guys have been doing and the go guys have been doing and we can do it in dotnet and we can stick it out there so go and do it thank you very much [Applause] [Applause]

Channel: NDC Conferences

Views: 1,446

Keywords: NDC, NDC Oslo, Mark Rendle, .NET

Id: 5sEqLS65Rk8

Length: 60min 21sec (3621 seconds)

Published: Wed Jul 05 2017