Getting Started with OpenTelemetry in .NET

hello everybody I'm Nick and in this video I'm going to show you how you can get started with open Telemetry in net and in this specific example I'm going to use grafana and Prometheus however keep in mind these are not the only ways you can use open Telemetry with in fact open Telemetry is just the standard for collecting all those stats about our application and then how we export them and how we visualize them can change in many different ways I'm going to choose free and open source options for you just because I want you to be able to grab the code from the description down below and start immediately pushing stuff into grafana so you can visualize them if you want to and both of those tools are being used in production extensively but know that they're also paid vales that are managed and they way prettier with the way they visualize stuff and the way they collect stuff but they're also more expensive now open Telemetry is the way you should be doing Telemetry in your application so if you're using something like app metrics or any other way of doing this really what you're going to see in this video is how you want to move forward doing met Tri in your applications because that's the agreed Cloud standard keep in mind this is not a Microsoft specific thing any application can use on elemetry and any application can do metrics and spans and stats in general in the exact same way which is the whole point of having this open Telemetry standard if you like type of content and you want to see more make sure you subscribe for more training check out my courses on doet tr.com okay so let me see what I have here I have a net API which is a weather API generating some built-in fake weathers and then returning them called in the weather forecast endpoint and then I have a weather app. web service which is a blazure application that allows me to visualize that weather and as you might have noticed already I also have two extra projects for net Aspire now if you don't know what net Aspire is that's fine you don't really need to know to watch or use any of that the reason why I'm going to use Aspire is because I want to have a distributed systems with metrics from multiple applications and do Aspire allows me to do that very very very easily and it is sort of the future of net in terms of running net applications so if you want to learn more about that I'm going to put a link in the description down below where I talk about it extensively just in case you haven't caught up and if you want to run a DOT aspired system which means both of my applications that communicate with each other all I have to do is run this apphost and again the code is down below if you need anything you'll see in this video it will be down below so I'm going to go ahead and just run it and what you're going to see by default is that net Aspire comes with this nice dashboard all I care about is running my front end so here you can see my front end and I get the weather this weather is from the API and if I keep refreshing I have a r cach so the weather is cached for a few seconds and then if I refresh enough times the weather will change and as you're able to see here I have some traces I have some metrics but none of that is actually populated that's because we're going to populate all that from scratch together now one thing I want to mention is you w't learn how to do this to visualize it with a DOT aspired dashboard because the do aspired dashboard is not deployed to production it's meant to be a local thing in production you're going to have your metrics exported with Prometheus and you're going to have them visualized with grafana and we're going to run all of them using kubernetes because I want to show you how in a distributed system you can have both your applications and your monitoring or visualization technology all together without having to go through the trouble of installing multiple things so let me show you how you can get started with this now do Aspire comes with this service default project which has this extensions. CS file in it and all we really do with this is we call it at the top of each application we want to use some service defaults in this case the service default we care about in this specific example is register open Telemetry for my application so this ad service default call on the web app and the equivalent of this on the API will contain for both of my applic ation now the configure open Telemetry call now as you can see this configure open Telemetry call is empty we're going to write all of that from scratch now what do you need to get started with open Telemetry well you need a few n packages now the way the open Telemetry n package is broken down is that if I search for open Telemetry yes you can have the core library in the API but really those don't really have anything other than abstractions what you want to use is use a combination of instrumentation packages which allow you to instrument parts of your application to start collecting some metrics also exporters to export your metrics in some way for example dou elmary protocol Jagger console and in memory a bunch of other options and also you can have extensions on existing uh things for example the hosting library because we're running a web host because those are both web service well I can get started by adding this package over here the open extensions. hosting now what do I want to instrument like I said both of them are ASP net core so I want to have the open Telemetry do instrumentation do ASP net core then I'm using an HTTP client to communicate from the web app to the API so I'm going to use the open Telemetry do instrumentation HTTP that allows me to have metrics for the HTTP client then I'm also going to use the runtime to get some metrics about the net runtime and then I'm going to get the default exporter of open Telemetry protocol so sort of the generic one now if you're using something like grpc and you want to instrument your grpc client you would install the grpc one which is in preview for now because GPC is not as popular as let's say rest for example but you can add that and so on and now what we haven't added is an exporter for Prometheus because we want to export those metrics to Prometheus which then graan will go ahead and use to display everything and collect everything in the dashboard or in in any other way to visualize or raise alerts or whatever so there's a specific Prometheus exporter for aspet core that is still in preview but it's pretty good it's an RC actually so open Telemetry exporter prometheus. ASP netcore and that wraps all the new get packages we will need I know it is a lot if we take a look at the Cs Pro you'll see there's plenty but once you install them you don't need to attach them ever again unless you need to upgrade them of course and then you can go ahead and start configuring open Telemetry so where can we add open lry well we can actually instrument the I logger to collect traces and spans appropriate to the log messages we collect so what we can say is builder. login. ADD open Telemetry and then we can configure a few things about that for example I want to include Scopes in my logger and I also want to include formatted messages here and that's it now our logger provider for open Telemetry is registered this means every time I do logger or ilogger do log information log warning whatever that information will be captured in an open Telemetry fashion and then exported wherever I need it then we can go to the core of the open Telemetry registration which is the builder. services. add open Telemetry a top level where we're going to add all the necessary services and we're going to say with metrics because in open Telemetry you have metrics and tracing so we're going to configure metrics first you have things like span and so on logs you have plenty of things but we're going to start with metric which is probably what you're most familiar with especially if you used something like app metric before we're going to say x here and then we're going to add the runtime instrumentation that comes from that runtime package and then we going to also say add meter and we're going to add some custom meters we want because we have to be explicit about which meters we want to start consuming or exposing now those meters will be the following we would need to have the as core. hosting one to get all the hosting related stuff all the hosting related metrics then we're interested in the server itself so in this case it's Cal so we're going to add the Cal metrics we're also interested in any HTTP requests being fired and for that we're going to use the system.net HTTP which is coming from that HTTP package we added and then in the end because I actually have some custom metrics in my weather API and I'm doing that using as you can see over here uh the I meter Factory that was introduced in net 8 or 7 I think it is 8 and I have a dedicated video on how we did all that so again description down below then I'm going to also expose those in here and then what I want to do is I want to say with tracing so let's add some tracing configuration as well so we can say add ASP net core instrumentation we can also say add grpc instrumentation if you're using grpc and we can also say add HTTP client instrumentation as well so both metrics and tracing now if you're in development and you can afford to do this mainly for debug reasons what you can do is you can say Builder environment. is development and you can say set sampler always on sampler now this would be very expensive to do in production but for here this is fine this will make your developer experience better so after we configured all that what we need to configure here is the exporters the way to export those metrics and for that I'm going to say add open Telemetry exporters which is a method right below this one and we're going to start writing the code for it now the way default do aspir does this is a bit interesting because what it does is it checks if we have a configuration parameter called otel which stands for open Telemetry otel ltp end point and if it doesn't exist then we don't use the open Telemetry exporter but if it does exist we do use it so because we're going to run this through do Aspire we're going to have it so what I'm going to say is go ahead and if you use the open Telemetry exporter configure the open Telemetry logger options the meter provider and the Tracer provider so we can have everything wired up correctly and really this is something you should be using but the thing is we will also use promethus on top of that so if you haven't done this already up here where you say with metric and you can say add promethus exporter what you can do is you can go down here and say builder. services. add open Telemetry don't worry I know we've called this already up here this will not override any of the configuration we will just add on top of it and we're going to say with metrics and then add Prometheus exporter to export those metrics to Prometheus and then the last thing we need to do here is actually map an endpoint to export and allow the scraping to happen for those Prometheus metrics we can do that by going at the very top over here of this map default endpoints method which is called in both the API and the web app and say app. map Prometheus scraping and poter we're going to use default for everything so believe it or not with that in place we have open Telemetry wired up what I'm going to do first is I'm going to run the application and show all that in the built-in dashboard not graphine Prometheus yet but just a built-in functionality which is using that open Telemetry thing that we registered over here so if I go to the dashboard you're going to see that now in traces I have two options front end and web API and if I select any of them of course I will need a few requests on them so let's go ahead and call them a few times here we go so if we go back to traces now you can see all those traces you can view any of them and you can see exactly what happened a redish cash call here a redish cach let's see what is here set X to set a value get a value get with the forecast and so on structure logs all of them are here and then all the metrics for each application with nice overtime graphs all that is done possible through the open lry exporter which we added and Aspire will use however depending on how you run this in production you might not have that you might only have the Prometheus one so how would we run this in a production like environment well if I was to run an Aspire application I would run it like I run all of my applications in kubernetes because that's sort of the standard for running containerized applications through an orchestrator and the way I'm going to do this is I have mini Cube installed on my machine and I have a cluster running on my machine so I can deploy in this cluster with which you can see over here in lens I have no applications now but I can deploy and I can deploy that by using aspirate which is a project that allows me to generate all the necessary kubernetes files and then push all that into kubernetes very easily with a single command in fact if I want to deploy everything you see right now all I'm going to say is aspirate and then apply I've already generated everything and I already have a video on this topic on how to deploy using aspirate so you want to check that I'm going to put a link in the description down below check that out I just deployed three applications so as you can see here I have three pods and I can click on the front end this is running in kubernetes again and if I forward this end point as you can see this is now running in kubernetes with the red cash and an API I'm going to quickly just say destroy and take down all of those applications and provision them so yeah let's go ahead and do that now you don't have to know kubernetes to follow along and you don't have to use kubernetes to follow along on any of this but it is a skill I think every developer should have and that's why we just launched on Dom train a brand new course by Dan clar who did the docker cost previously if you haven't taken that I highly recommend you get up to speed with Docker with that but now Dan released his kubernetes for developer course covering everything there is to know for kubernetes as the developer and you can go from knowing nothing about kubernetes all the way to professional level proficiency to go in a company and actually use kubernetes and apply and deploy immediately it's an amazing cost I'm going to put a link in the description and the first 200 of you can use discount code Cube 20 at checkout to get a 20% discount don't miss this opportunity this course is honestly fantastic and it's how I refreshed my memory and actually learned new things on kubernetes highly recommended but now if I want to add extra containers in net Aspire which I need because I need to run Prometheus which is service and grafana which is another service how do I do this well I'll show you technique that was shown in the net BL log initially that allows us to mix those containers and those services in here using the app host of do net Aspire so what I want to show you is I actually have two folders over here that you can see in the solution a Prometheus one and the grafana one one has the dashboard and configuration about grafana and the other one has the promethus configuration again you can grab all that from the description down below but this means I can now push them into the respective containers in Aspire and have all the configuration done by default including my Dash boards now to do this I will do the following in Aspire I'm going to say grafana over here and then I'm going to say builder. add container because we don't have an add grafana call so add container so which container we're going to call it grafana over here and then the image name is grafana grafana then what I'm going to do is mount a few things so the first thing I need is with volume Mount the config folder into the grafana folder and then the dashboard folder into the grafana dashboard folder and then to wrap it all up I'm going to say with endpoint and I'm going to say container Port 3000 hostport 3000 name grafana HTTP and then scheme here we go and then I'm going to do the same thing for Prometheus so add the container add the volume and configure the endpoint and that is it now I can take this grafana reference and I can add it to whatever project I want in this case I'm going to add in both you might not need in both this basically will just add an inment variable of where the graph dashboard is located and I'm going to get the endpoint using the name of my grafana endpoint which as you can see is this one over here so pretty pretty neat now I could just run this as a normal dot ASP application so I can go here and say go ahead and run the app host directly and if I do that I still get the nice dashboard and as you can see now we have a Prometheus and a grafana container here so if I go to grafana or actually let's go to the front end first and run a few requests here we go and then I can go to grafana which is where I'm visualizing those metrics and as you can see now I can see everything about this API if I keep hitting it you can see I have the front end actually here and the back end I can see things about them how many connections I have which end points are being hit the most now we call the metric end point which is exposed through well the whole configuration we just did and it's being called once every second that is why you see it so frequently called over here you can see all the HTP requests you can see the error rate you can see the percentile of the request durations and you can configure everything and you can always go and explore metrics on your own so you can see all of the metrics of our applications both the one I added the weather API request count for example is coming in here I can come here and do whatever I want so say run query and you can see I've called this API twice because it was cached before so if I call it again over here I should see another one over here after a second let's wait for a second here you go another request now over here so as you can see we have everything in grafana using Prometheus using Prometheus but what if you don't have the net aspired dashboard and all that well you can still run this in kubernetes with aspirate so what I'm going to do is I'm going to say go ahead and generate me a new aspirate configuration with all of these components I'm going to use red grafana Prometheus weather API and front end again if you don't know what that is I have a dedicated video explaining everything about about this and we have the cost on D train trust me when I say you really really want to get out of speed with kubernetes and there's no better and more upate results to do that so all that now is generated and I'm going to go ahead here and say aspirate apply I'm going to provide the password for the secrets and that is it everything we deployed now on Mini Cube as you can see it has I can see everything here containers are starting everything has started and I can now go ahead and say go ahead and expose my front first because I want to make some API calls so here we go the front is running I'm going to call it a few times here you go and then I'm going to say hey I want to see grafana so let's click on that let's say expose grafana as well open it here you go it's opened it's admin admin by default so let's skip that and go ahead in the dashboards and now you can go ahead if you want to and you can configure everything about your dashboards as you can see we have no dashboards the dashboards were not picked up in this case and pushed into grafana running when when is however what you can is you can create a dashboard and we can actually import a dashboard using adjacent file again if you want to see all that description down below you're going to find all of these Json files we're going to go ahead and import it that does mean however you're going to have to configure your promethus and kubernetes end points a bit but depending on how you run this this can be very very easy and what you can also do with aspirate actually is you can use the compos formatter so we can create a dock compose with all of these services so once we do that what we're able to do is get a Docker composed. yaml file so I can say aspirate output and I can say Docker compose up so I don't even need kubernetes if you don't want to use it don't use it this will configure everything and now as you can see here we have red this frontend weather API promethus and grafana so if you want any of this code you can get it for free from the description down below well that's all I had for you for this video thank you very much for watching and as always keep coding

Channel: Nick Chapsas

Views: 36,591

Keywords: Elfocrash, elfo, coding, .netcore, dot net, core, C#, how to code, tutorial, development, software engineering, microsoft, microsoft mvp, .net core, nick chapsas, chapsas, dotnet, .net, c# new features, c#, .net new features, .net core new features, .net 8 features, c# 12, c# .net 8 features, .net aspire, what is .net aspire, dotnet aspire, cloud native, cloud-native, distributed apps, .net 8 aspire, grafana, prometheus, grafana .net, prometheus .net, otel, opentelemetry .net, opentelemetry

Id: nFU-hcHyl2s

Length: 19min 55sec (1195 seconds)

Published: Mon Mar 11 2024