OpenTelemetry Collector on Kubernetes | How to Setup OpenTelemetry Collector on Kubernetes | #k8s

foreign Telemetry concept which is a very popular concept for monitoring your applications so yeah before we start uh onto this topic please like And subscribe this channel if you have not done it uh this will help us we help you to understand or get the latest videos related to kubernetes devops cicd monitoring open Telemetry and so on so yeah so let's discuss about this concept what is this and why it is so becoming so popular in today's world so you might have on the left hand side can have any kind of an application it can be a java.net base or Java node based node.js based application that needs to be monitored properly so what this open Telemetry it will act as a collector so rather than sending those entry directly to your monitoring agent so it will first of all collect the matrices and then we'll send to the relevant uh the back end it can be time series database it can be traced database it can be anything so why it is so popular because it is you know vendor agnostic when I when I spent agnostic it is I mean you can easily migrate from today's suppose your doing monitoring using grappana or app Dynamics or dynatory so tomorrow you can simply move from one vendor to another Vendor by simply changing the you know link so Hotel collector will remain same so application will continue sending your matrices and traces to the original plan to the relevant backend now what additional things you can do it you can get the infra level uh details also the kubernetes API level details or any Cloud related methods can also be added to what we collector and finally it can send it to the uh to the end result so just to see a complete holistic picture we'll be doing a quick you know installation of first of all a micro Services application onto the cluster and then we'll install a collector which will receive the matrices and traces from this app to this second section and then we'll use Jaeger for tracing or tracing the entire transactions which will take the data from collector basically this is the third section and then we'll have a media for getting the matrices so yeah this was all about Theory let's quickly jump on to the Practical implementation of it so I've created a GitHub page for this I will follow that page for our today's demo and then you know let's let's quickly get started into the the installation of this entire piece so this is the readme I'll be following the readme documentation so specifically updated all these folders and in in each and every folder we have development manifests that are needed for installation of this entire open Telemetry framework okay so first of all you need to install and configure Microsoft application so it is an open Telemetry app so basically you will find in this folder uh deployment and a service so basically there are a lot of endpoints that are there uh those endpoints are deployed as a deployment and the same thing is being exposed as a service now you will see in any any deployment file you'll see the matrices are being sent to this hotel exporter Hotel endpoint which is the one I'm talking about the centerpiece okay so first of all the met the application send data to The Collector and then collector will send to Jaeger we'll follow the route and then first couple we will install the open Telemetry micro process application so I'll go to this folder after doing a you know a git clone of that repository okay I'm into that folder now I'll create a namespace if it is not there this is this is a brand new cluster so hopefully yeah the name Hotel membership has been created now I'll run uh this command which will install all the manifests which are there in the current folder okay so let me see DNS oh I have installed everything into the let me see just again let me explain the Hotel demo kgp Oh by default I have installed everything into this world no problem let me just quickly change it to kns default my bad just give me a moment let me just quickly Cube CTL delete minus f I don't know how to install anything into the default namespace so we will install uh everything into the um foreign is getting terminated I'll pause the video for a second okay so I'm into the hotel name space the you know name space and now nothing is there so I'll run this command again and this will install everything to this namespace perfect so now my microsources application should be running here also it must be coming up yeah it is coming up and I'm into this namespace a what this will do it will install this micro sources based application uh we will first of all quickly see that application how we can you know do a quick login it is installing a postgres database and a queue for storing the data so let me just quickly see uh the app is slowly coming up yeah it's almost up raw detection service is let me just quickly do a hit service key put forward I'll do a front-end proxy 8080 oh this is the URL of your application basically yeah so this is my application which is you know already up so there's a wrapper of documentation which is already already there you can just follow this documentation for any kind of other understanding so my application is up let me just quickly do some uh Plug and Play I'm able to do some dummy orders okay so my application is up and running so now for sending this application data to collector I need to install the open Telemetry collector so I will come back to here okay and then I'll open Telemetry collector I'll go to this folder I'm inside this folder now what this is doing is simply installing a doing a deployment pulling an image and then it will just capture all the data which is being sent by the Microsoft service application and ultimately to assigned to the Jaeger port and all those things that will come in the third step first of all it will capture the data so qptl apply f so now this will install your collector I'll just go to the checking tab yeah you can see the open telemetric collector container is getting created I can see the events yeah it is coming up okay my collector is up and running now and we see the logs also for the collector I'm just taking little bit time to explain each and everything so that it becomes clear for understanding so see now there are no error in The Collector logs it is able to capture the data from the tracing okay punch for your next episode come on to the second step third step is relying to install Jaeger now what this Jager will do it will just see the deployment file okay it will capture the data and then it will push it to the Prometheus so first of all let me install Jaeger and then we'll install you know Prometheus Next Step okay folder Cube CTL apply f the Jaeger has been installed let me hit the cube it will get power that is getting created see so first of all we install application then we install collector which contain the details from application to collector and then while you're installing in a Jager perfect so Jaeger is already also installed and let us see uh then let's install the course over the primitius that will actually capture the matrices related to all that application so we'll go to Prometheus folder and then we'll do a cubital player now this will install Prometheus server only the same namespace and then we can see the kgp so let us DD open television documentation in parallel so we have already installed the application let me do a port forwarding again here ah this is the front end clock C port forwarding so you can see the application is up and running let me do some more hits on this application again to cart place order continue shopping this is this now we will just open the prepana is not installed here so I'll just open the Jager home page let me see if there's any kind of data which is coming here is also installed and the traces are coming up let me see whether we have Prometheus installed okay Prometric is not there but I can do a port forwarding from here so just try to understand the concept first of all we have installed application and then collector and then Geiger and then Prometheus let me see since there are not visual server is there foreign foreign service port forwarding I'll come here okay I can do up let's see whether things are coming or not okay data is coming and the data is coming properly I can see the discovery also I have to targets Target point so yeah so permit it is also everything is up and running right now finally we need to install good corner to see the dashboard because that will also only make you know some significant so if you come onto the grafana folder we have a deployment file that will install you know grafana and it will have the admin user ID and password and then we are having certain dashboards the predefined dashboards related to open Telemetry collector that we are installing you can just simply do copy waste on that we have a secret that is nothing but admin user ID admin password so this secret is used for logging to grappana and the config map that is used to test the graph that is up yeah so that is that is finally the installing grafana so there is the final piece yeah so I'll come back here CD dot dot grafana and I'll do a cube CTL apply as okay you can ignore this because the test scrapana is trying to run onto the port 8080 which is already being used here let me just quickly cancel this and then I love it and run this yeah so that is fine I'll do a kgp okay starting and grafana test is just you know creating the container Okay so I'll just Hep clear GP yeah forget about this death dot board and it's just you know doing a hit on this uh you can simply delete spot that is not needed what does it is a testing code fine so my grafana is already so up let me see the grafana service as we see um service so I'll do a key port forward foreign as I said the username is admin and the password is also admined okay I don't change the password now just you can ignore this piece you can simply come here you can see explore the option and then you see Diego there are certain so traces are coming properly you can see now let's see something else the let's see the Matrix is also from Prometheus you can see a lot of matrices are coming let me just to open okay one more uh two run one query to see how many matrices are coming yeah the query that I'll run to see what kind of matrices are coming basically how many matrices per job is coming yeah so I can see yeah so all these are endpoints that have been uh been monitored using open Telemetry only and yeah so all these endpoints are nothing but your uh application endpoint you can see card service checkout service all those things that's part of the reason I want to run this piece I want to show it to you now let Theory finally the dashboards I'll click on the dashboards there are certain dashboard that we executed when we you know installed grafana so those dashboards are nothing but it is in this folder uh foreign counts okay navigating some data to the back pan processor Also let's see this mother dashboard open television collector yeah yes we have started looking at some data onto the open Telemetry collector dashboard also okay now but ultimately collector data flow see so you have only one instance because it is a single node you know k3d cluster so CPU memory you can see the uh the node graph also right and you can see the other related details like it's the pure open Telemetry collector dashboard and then finally see this pan metric stash code also you have so nutshell you're getting data everywhere the proper linking has done so so the reason behind this the complete demos to we do uh understanding how it works for example we have installed uh application so you can see the application is sending details to this open Telemetry collector which is nothing but your this hotel call endpoint this and then it is sending it to four three one seven if you see any deployment sending data to this uh four three one seven now if I come here collector this is the collector that I'm talking about this is the four three one seven is the one grp OTL otlp Port okay so here the entire data is coming back and then this collector is sending data in a Jager at this board one four two five zero if expand is eager deployment file you will find this one core uh the recipient host code and yeah it should be there yeah so all those ports are there and ultimately it is eager is also sending data to over Media Services this is the one and then Prometheus server is getting the data for all these endpoints and it has a config map config map is nothing but the scraping thing yeah it is scrape interval not scrap interviews it is scraping that is happening it is Handy having the name space and the relabel configs and finally SD grafana where it is having all these final details about the dashboard and the uh so basically what we say in a nutshell we have installed how many tools the factor for the application and then the collector second third post is simply five uh you can see five different charts of file five kind of bundle many class we have installed so yeah I think we should wrap up this video we should not stretch it further so as a summary what we have done we have understood with a working example what is open Telemetry how we actually collect the traces from a working microsystem with application and how collector is sending data to a current parameters at the same time and now we can see the data back to the katana in terms of passwords so I think that is for now maybe we'll have some more open Telemetry use cases in future so if you have not likened the families do that from here so that you can get similar kind of Channel sessions in future okay I would wrap up the session now thank you for now thank you okay bye

Channel: Bhoopesh Sharma

Views: 1,776

Keywords: cloud monitoring, grafana, how to use opentelemetry, jaeger, monitoring, observability, observability aws, opentelemetry, opentelemetry .net, opentelemetry .net 5, opentelemetry .net core example, opentelemetry .net example, opentelemetry asp.net, opentelemetry asp.net core, opentelemetry c#, opentelemetry collector, opentelemetry distributed tracing, opentelemetry dotnet, opentelemetry kubernetes, opentelemetry tutorial, prometheus, what is opentelemetry

Id: _UwbFYLev-8

Length: 19min 58sec (1198 seconds)

Published: Thu Sep 28 2023