Creating Dashboards with .NET 8’s New Metrics!

hello everybody I'm Nick and in this video I'm going to introduce you to the brand new metrics that come built in in.net 8 and how you can use them to scrape them and build very pretty dashboards like this one not only that I'm also going to show you how you can create your own custom metrics using the new apis that were added in dot net 8. if you let out of content and you want to see more make you subscribers for more training check out my courses on domtrain.com alright so make sure you don't have here I have a simple weather API here but I have changed a little bit this is not exactly the built-in weather API so here we accept a city and a number of days that I want to get the weather for and how long I want the forecast to be in the future and it still generates in-memory fake weathers but it has a bit more functionality it also adds a bit of a fake delay over here to simulate a real API call to maybe some other service if I quickly go ahead and I run this let me just show you what you're gonna expect on the front end so I API is running and I have Postman over here so if I try to get the weather for Zurich for five days I can call that and I'm gonna get Zurich and then a bunch of forecasts and if I say for example at one then I'm going to get 1 and so on now the metrics we're going to take a look at this video is sort of an extension on what has been happening with the whole open Telemetry initiative so it's going to build on top of that now what I want is when I go to nuget and I will install two new packages since I want to get metrics for hosting the whole dotnet hosting framework and then asp.net core I'm going to install Opel Telemetry dot extensions.hosting which at this stage is pre-release so if you don't have pre-release enabled you're gonna see the old version here we have 1.6.0 alpha one and then because I want to scrape these metrics with something called Prometheus which is sort of a Time series database that can store this metrics and then expose them to other areas as data sources I'm going to go ahead and install the open telemetry.exporter.prometheus.a spinet core again this is pre-release and if you actually uncheck this at this point you won't see a non-pre-release version because this will be the first one to be published so I'm gonna go ahead and just install that as well and now with these two packets in place I'm gonna go to the program.cs and do all of my setup now the first thing I need to do is I want to say Builder dot Services dot add open Telemetry and then on that I'm going to say with metrics because we're going to be exposing metrics here and we're going to do some configuration since we will be using Prometheus to scrape them I'm going to say over here x dot add Prometheus exporter I will explain what that does in a second we're actually going to see exactly what it does now since I have the exporter in place I'm going to go ahead and add some meters so I'm going to say add meter into the list of subscribe meters and I'm going to add two of them the microsoft.asp.net core.hosting and the microsoft.server.chestral this should be enough and if I just go ahead and I break them so you can see them better this should be enough for what we're going to demonstrate here and then because I want to actually expose the duration of the request I'm going to be processing I'm going to say x dot add view over here I'm going to give a name to that view I'm going to call it request duration and then to configure that view I'm going to pass down and explicit bucket histogram configuration now this goes a bit maybe too deep into metrics and how they're exposed but basically what this will allow me to do is provide the boundaries I want for this histogram metric stream now with those in place I can go all the way down here before the direction and say app dot map Prometheus scraping endpoint this is an endpoint that our API will expose on the first last metrics by default to allow for Prometheus to scrape and get those metrics that are apis exposing so we're not really sending those metrics to Prometheus we just say if you want to take them they're here and then parameters will go and just get them and start storing them now with these two things in place I'm gonna just rerun the API and show you exactly what I mean so the API should be running I'm gonna go ahead and just call it a few times so I'm going to say one two three four five requests and then I'm going to go to the forward slash Matrix endpoint I'm going to call that and as you can see now that is mapped using this line over here and this will allow me to get things like carrying Castrol connections Castle connection duration things like handshake duration cute connections if I want them I have scrolled the way down current requests TLS handshakes we have tons and tons and tons of metrics we now Expose and Prometheus can come in and scrape them and store them and use them so we can do things with them analyze them and visualize them and now we're ready to start scraping those endpoints and start visualizing things in grafana now if you want the code in this video you can become a patreon and you can get the code it will have all the compose files all the grafana dashboards everything I'm going to be showing will be there now for me to run is what I'm going to do is I've already configured over here a Docker compose with the API itself a bunch of different things over here Prometheus the node Explorer we have um grafana and so on so what I'm going to do is I'm going to go here and I'm going to say Docker compose up and this will run my API Prometheus grafana the node Explorer a bunch of other things so we're going to start downloading all of those images as you can see over here here and build them once that is running you can actually go to localhost 1990 over here and see Prometheus and if we go to the status and then targets you're going to see that our metrics endpoint over here is being hit is being scraped so if I go and I call it the API is running and I can see all those requests all that data and it is being successfully scraped by Prometheus and it means I can go to grafana and I can say admin here and then full bar over here and login and once I do that I can go to explore and I can search for a metric and if I search for example for Kestrel I'm going to see all the cast related metrics and do well anything I want with them for example if I want to say current connections I can do that I can run the query and I can see over here I had three over here two over here and if I go ahead and I keep spamming this maybe I'm gonna get more connections and I've configure this to scrape those metrics every five seconds so if I go here and I run this again we might see a connection increase but keep in mind those connections are opening and closing so we might not really really see much of a difference as you can see it was three and now it is two but this is not everything I want to show you in fact I want to go to dashboards and I want to import a couple of dashboards that you can grab and use for your own purposes just as a getting started mechanism and these actually come from um James the person who also made um json.net so I modified them a bit to make them work with the latest preview and we're gonna go ahead and import them if you want to grab them if you grab the code you're going to find them and the dashboards so I'm gonna grab the first one over here and just import it so I'm going to select the Prometheus source to basically say this is where you can get the data from and import that we're going to talk about what we see here in a second and then I'm going to add this endpoint one as well so back to dashboards import paste this here select the source and then import it so let's go to the main dashboard this is all driven based of data that is coming from that running API now because if I just call this API a few times and as you can see most of the endpoints are calling the metrics endpoint I want really have enough data to show you exactly what's going on so what I'm going to do is I'm going to go to this test.json file over here which is a very simple k6 performance test but it's more of a smoke test to just generate some traffic so we can take a look at what we have after I'm going to run this for a minute so what I'm going to do is I'm going to go and run k6 run test.js and I'm going to generate some fake data and one out of every thousand requests I think here will also throw an exception so we should have a good variation of successful requests and maybe a few unsuccessful requests and as you can see one of them did in fact fail so let's wait for a minute for me a minute for you just a millisecond and let's take a look what we have at the end of this test so the test has now completed we had two exceptions we should be able to see them in the dashboard so I'm going to go back here hit refresh and then we can immediately see some of the error rate over here 0.02 percent in fact some of the traffic over here the distribution between percentiles and how long they take and things like current connections so we did Peak at 100 and maybe two yes and two open connections those were the performance tests and the content scraping that's going on and then you can see the https over here you can see which endpoints fail so in this case the weather City endpoint and you can see the successful one of the top requested endpoints as well and if we click on this because we have this other endpoint dashboard we can actually see more details so for example the type of exceptions occurred how many requests the status the status from 500 as well over here which my face should be hiding so let me just move it around and so on so this will give you a nice way to get started and if you want to search for things like post requests other routes you can have that all nice and customizable as well now if you want me to make a dedicated grafana video showing you how you can build all this from scratch then please leave a comment down below I want to keep this video just on the metrics themselves so now we have scraping and we have dashboards but what I want to do is I want to see how we can actually have our own custom metrics in here because all that is just built in through a package but what if I wanna for example select how many times a city is being requested when I can actually do that with again a new thing that was adding.net 8 about metrics what you want to do is go here and say builder.services.add metrics over here and once you do that you'll be able to inject in your classes the private read-only I meter Factory so let's go ahead and inject that and once you have that you can go ahead and do things like this you can say meterofactory.create and you can create emitter and we can give it a name over here so I'm going to call that weather app and if you want by the way you can use the overload that uses meter options and this won't only allow you to provide the name but it will allow you to provide a bunch of other things like tags version scope and so on this is getting a bit too advanced so I'm not going to go that deep so I'm just going to say whether app over here or whether API even and once I have the meter I can go ahead and say instrument over here equals meter dot create and I can create a histogram a counter up down counters or something that increases and decreases like the connections for example I can have an observable counter a gauge you can have different things I'm going to create a simple counter over here to count the weather request I'm going to create the city counter request over here now you can do many fancy things with providing tags and so on I'm just going to leave it very very simple and I'm going to say instrument dot add and I'm going to add once so the city was requested once and so on now just creating the meter in here won't actually be enough to expose it what you want to do is you're going to go to the name of the meter over here so you want to grab the weather API and we want to go over here to the add meter and we want to add it using the name we used over here to be able to expose it otherwise it won't really work so remember to add it using the add meter method as well and now that we have that I'm gonna go ahead and just restart all the applications in Docker and I will will run the exact same sort of fake smoke test so that should be running over here yes it is running and I'm gonna go ahead and just run the test for a minute and see what we get back so requests are now done we had a few exceptions but let's go ahead and take a look at grafana this time so I'll go to the Explorer Tab and I should be able to search for City counter you can see it over here and if I say run query you can see all the city counter requests over here 5703 and those are all driven by that thing that I added over here so I said create a counter increase it and then go ahead and expose it this is the very first view to what is coming and I'm sure we're gonna get more and more of those and if you want a more in-depth video on Prometheus grafana or how to make pretty dashboards then please leave a comment down below thank you very much for watching and as always keep coding

Channel: Nick Chapsas

Views: 66,860

Keywords: Elfocrash, elfo, coding, .netcore, dot net, core, C#, how to code, tutorial, development, software engineering, microsoft, microsoft mvp, .net core, nick chapsas, chapsas, dotnet, .net, .net 7, .net 8, .net 8 metrics, grafana, prometheus, grafana dashboard, .net dashboard, grafana prometheus, grafana .net, Creating Dashboards with .NET 8’s New Metrics!

Id: A2pKhNQoQUU

Length: 11min 43sec (703 seconds)

Published: Mon Jul 24 2023