Introduction to OpenTelemetry

uh [Music] welcome to is it observable the main objectives of is it observable is to provide tutorials on how to observe a given technology today's episode is part of an existent series that we already started on a very popular cloud navy technology i'm referring to communities of course so we already explained in this series how to retrieve metrics with the help of prometheus uh how to retrieve vlogs and we cover by in fact two episodes so one with loki and promtel and the other one with flynn bit and last recently they cover how to collect events and represented two open source project and the one that introduced the technology today was k-span so today i'm gonna reveal which technology i'm gonna cover today we are going to cover very useful angle uh in the observability it's how to collect traces using a standard called open telemetry so if you are looking to understand why your application is slow why you're getting errors where you're spending time then this episode will be very useful for your project so if you enjoy today's content don't forget to like and subscribe to the channel so what are you going to learn out of this episode so first we need to do an introduction on open telemetry then we will cover the various data collections of open telemetry which are traces metrics and logs we will then touch base on the various components of open telemetry the collector and the various client library available today we won't do any tutorial because we will have a dedicated episode explaining your journey with open telemetry so let's start with the introduction from the moment our application are more complex using several services technology database various cloud provider it becomes very difficult and critical to understand how a business transaction is going through our architecture especially if you're have a part in your system then you're moving on to another platform another product uh you're losing track once it has left your environment so we want to understand where where it goes and we are more interesting to understand where we are spending time which service which components are slowing down our users open to entry will allow us to collect traces metrics and more you see logs from our system and export it to a solution of your choice and that will help you to analyze traces metrics and logs so opponent imagery is providing what we call distribute interesting so technically it's the technology that will allow us to have the exact level of details that we're looking for to drill down and understand where we spend time just by looking at the name open temperature so you have open open stands for open source and then you have telemetry and telemetry come from greek world word which means tele means remote and metry measurement so open telemetry it's an open source technology allowing you to collect remote measurements open telemetry is in fact the comes from two existing open source project that just merged and it's open tracing and open senses open telemetry it doesn't provide any observability back-end solution like prometheus or jager it's defining a standards so if you decide today to collect traces and send it over to solution a it won't be a problem to move to solution b because it's the same agent the same protocol of communication same standard it's just that the solution that will store your traces will be different so much more easier to transition from one tool to another so that's a great news when you're dealing with open termitry it requires two steps basically first we need to generate data from our applications so we'll see that and second we need to store and consume the data collected open temperature to be honest doesn't bring any technical revolution for us i mean most of us have been dealing with observable the platform and so and those platform were already providing the ability to drill down in our code so if you take a step back in the past the solution will name apm and if you remember well there was a solution called ca wiley and c.a wiley has the ability to draw down in java code and then other solutions came out and most of those solutions has the ability to drill down so it's the case of data trades newer relic and many others open telemetry doesn't use the property formats that other tools does it provides a standard so which everyone use that standard which means i can start a trace from my app but from the moment i move to an application i don't handle i may be able to also continue the trace and understand if i been slowed down by my third party application all right so let's see the open telemetry data collection so first traces so what is a trace a trace is the ability to drill down we all know that so to explain what is the trace let's take a short example with uh the application that we've used since the beginning of visit observable the google hipster shop so when you are doing a checkout in the googie shop the front end calls the checkout services the checkout services will call the cart services to check to get the cart and the cartridge they will interact with the database so when the checkout services receive the call from the front end and just before interacting with the card services the checkout services will look at the http request headers to get the trace context so the trace context hold the tracing information to glue all the spans together the checkout services can either utilize if there is an existing trace context at the existing one or it will basically create a new one with a new span and a new span will have a name and a timestamp so here we have the checkout api call at that given timestamp and then it will be added to the trace context in the http request that we will send to the card services the card services receive the request collect the trace context and it will also add its own span so whatever the work a given service or method has to do you can create as as many sub or child spans to attach the sub tasks so for example here we have the cart details with that timestamps and then we can track the interaction we'll do with the database so here we have a database call and then after we receive the database information we need to process the data in order to do stuff and send back the response to the checkout services the trace context is the information that helps to link all the span together to create at the end a trace so span has a name start time and an end time you can also attach to the span attributes that will give you more details related to the actions the technology that you're dealing with so and they are predefined attributes depending on the technology you use so for http for example there is http.method http.statuscode for database there is the db type db instance db statement and then for the messaging technology you can also measure systems destinations and so on and so forth and that information is usually usually very in important to understand the context of the request the host name the machine and then the requests uh the customer in the fire um anything related to kubernetes as well so new spans may be created to represent work being done by a sub-component of a service with the service when the service makes a remote call to another service the current span context is serialized and forwarded to the next services by injecting the spam context into the headers or if using message messaging technology will be part of the message envelope the next data collection is open telemetry metric so a metric is simply a measurement that you can capture during runtime and there are three type of metrics with opportunities so this counter it's very much similar to prometheus counters it's a value that will sum and that will just grow over time there is a measure it's a value that will be aggregated over time and last there is observer so we capture a current set of values at a given time like a gauge in prometheus similar to traces metric will have context and it will also be important for the metric information so metric has a name a description a unit a kind so the kind is the type so counter observer measure a label the type of aggregation and the time last data collection is logs logs is also a dimension that is going to be part open telemetry but it will see it's pretty much under construction as of now let's jump into the open telemetry components so there is first the open telemetry collector so the open telemetry collector is not necessarily required uh you will see depending on your your solution that will basically ingest your traces or metrics you may not use the collector but sometimes the collector makes your uh deployment much more easier and faster to change your destination solution that will ingest your traces so you have several way of deploying this collector you can either deploy it as an agent so basically the collector is in instance running with our application on the same host so it could be either in a all the fashion technology in the same machine or it could be in a con in a container world like a side car container or we can also deploy it as a daemon set in each nodes we will be close to the workload and it will be acting like an agent then you got the getaway uh so deployment so one or you can basically have one or more collector that are running as a service so one per cluster per data center per region you're very flexible in fact the recommend a recommended deployment is if you have a fresh new app then yeah choose the agent if it's an existing one it's more may become more complicated to add the agent everywhere so you can deploy it as a getaway in the case of communities it's gonna be an agent because we're gonna use daemon sets so it's make complete sense to use the agent mode within the collector there are several components so we have the receiver that will get the data into the collector it could be either send through push or pull then you have the processor so basically you have received some some some data and you can decide to do to transform it or do whatever you want with it once it's been processed then we probably want to export the data to several solutions so not only one you can have several solutions similar to what we've seen during the flint bit episode and basically you can send it either through pull or push and everything that we described so receiver exporter and processor they are defined through a collector pipeline you can create a pipeline to process transform and then send it over to an exporter and in the top of a collector we have extensions so in terms of extensions for traces um there are the receiver and the exporters will be able to receive open telemetry of course jaeger and zipkin the processor there are several ones see the attributes patch retry resource sampling span for metrics you will have a receivers for open telemetry in hosts and prometheus of course and same thing for as an exporter you can either send it to open telemetry format or to prometheus and then you you can have also community based component where you could basically build your own extensions that will enhance the collector pipeline so it's similar to what we've seen with the flintbit we can imagine that we can create a dedicated exporter for our solution of our choice that will receive and ingest the traces the metrics and the logs the pipeline is basically defined through a configuration file so it's the hotel collector configuration file in the case of communities like usual that configuration file will be stored in a config map so here is the pipeline that we used uh on the previous episodes for communities events uh in order to send it to dynatrace we uh k-span were sending the open telemetry traces uh to an open temperature collector and that collector will stand over two tana trays so here you can see that we have a receiver uh which is otlp the protocol you can define if it's the grpc or the http protocol and then you can define your exporter so in my case in our example it was the http uh open tomato protocol and we were interacting with the uh termetry ingest api of datatrace but you can see which is interesting is in this example we were using the http protocol exporter but also you can generate logs so you can also if you want to for debugging purposes you want to send to a log you can also do that in terms of communication protocol you will see in this example we see the receiver and the exporter has a notion of protocol so there are currently two type of communication protocols supported the open temperature http which is in fact protobuf and the other one is grpc so depending on the protocol supported by your third party solution that will store the traces you will probably need to utilize the collector to convert the the format and the protocol to be able to ingest it on your target solution the second component of open telemetry is the instrumentation library so application needs to have an open temperature library to collect either manual or automatic instrumentation so currently open symmetry has various languages supported in terms of libraries so we have c plus plus net alang go java javascript node.js php python ruby and rust and swift which is important is that most of the those library they support automatic instrumentation so it means that you will have to load that dependency or like the library or let it go open temperature and you will have to configure the connection to your collector through environment variables or system variables in the case of java so if you want to launch the your java files you can say java with your java files and you can add then the properties to attach your open telemetry library and the various configuration to attach to define the destination of your traces for most of the language that is supported automatic instrumentation means that it will attach a library to the runtime of our application and inject by code to capture open telemetry from a number of popular libraries and framework so what does it mean well it means that it won't instrument your code automatically it will basically all the well-known frameworks uh of the markets they will be instrumented because we know that so if you want to know if you're the framework that you use is supported i would recommend to go to the open temperature websites which is here and you will see all the various uh frameworks supported per type per library so the manual instrumentation so basically you will code the settings to interact with your collector so you will have to precise the data source the exporter propagator and so on so basically you add the dependencies like if you take the example of java we'll add a maven dependency or gradle and then in your code you will basically have a new library that allows to one configure uh the config the communication with your collector and two what are the traces that you like to collect what are the methods that you would like to generate traces and so on and so forth so here is a table reminding the current support for each languages remember there is first data data collection so traces metrics and log as you can see here uh most of the uh the languages and they're either in a pre-alpha or a better version in traces and metric and few of them have started to have an implementation usually in experimental for logging and net has a stable version for traces metric is alpha and log is better and java you have a stable release as well for traces metric is alpha and experimental for logging so depending on the language check out uh this the the open temperature websites this table has been updated so you will be aware if you will be able to generate metrics logs as the traces so traces is a good starting point but keep in mind that you can also collect metrics as well which is a cool feature and last the logging will be also interesting as well so that's it for today's episode uh related to open telemetry uh like mentioned here there won't be any tutorials because we will have two distinct episodes where we have tutorials and we will do the journey uh related to open telemetry so first one episode on how to instrument how to get started with putting traces and instrumenting your code and the second one is how to utilize the traces that you have ingested so if you like today's content don't forget to like and subscribe to the channel and we will build other content to help you extend the observability in your environment so thank you thank you for watching stay tuned and see you soon for another episode bye [Music] you

Channel: Is it Observable

Views: 353

Keywords: OpenTelemetry, Distributed tracing, OTLP, observability, Opentracing, OpenCensus

Id: dfpGU9al_i4

Length: 23min 26sec (1406 seconds)

Published: Thu Oct 28 2021