What is OpenTelemetry?

for the most part our back ends and our apis are black boxes we only see what goes in and out of them we don't actually know what's happening inside open Telemetry is a really important and interesting project it helps us get information about our application at runtime that gives us more insight into what might be slow where errors are coming from how our users actually use our application open Telemetry is a project by the cncf the cloud native Computing foundation and it's actually a combination of two previous cncf projects open tracing and open sensus now these tools help us understand our applications and by combining them together we get a lot of the power of vendor agnostic tooling from open census and the specifications of what messages should be passed as open Telemetry so by creating this as a thing it is a stable project that means it's an incubating project it means it's stable enough to use in production and plenty of companies are using open Telemetry in production today when we talk about Telemetry in the context of open Telemetry there's four things that we really need to understand that make up the Telemetry that we're tracking for the applications that we're looking at so so those four concepts are traces which are like the big picture of our application they're like one request response cycle and they're made up of smaller spans that make up maybe smaller parts of that request response cycle so maybe a database request versus a fetch to a third party API things like that the next thing that we're going to be talking about is metric metrics is basically the performance characteristics like your CPU or Ram it might be Network latency it might be all sorts of other stuff where it's just like a number that we're tracking in the context of a trace after that we have the idea of logs I think most of us understand what logs are but logs are basically a timestamped record that we can use to understand the flow through our application where we maybe just wanted to like log something out at a specific point in time and then finally there's one more thing called baggage baggage is basically just contextual information that we might want to pass from span to span throughout the context of a trace or maybe even through uh from Trace to trace throughout the context of our application so baggage is just data that we want to pass around to make sure that we understand the broader scope of things you might be wondering why open Telemetry why does it exist to understand that first we need to understand why Telemetry exists for the most part our backends and our apis are black boxes we only see what goes in and out of them we don't actually know what's happening inside with t we can actually instrument things like fetch requests like database drivers like various parts of your hot path in your application maybe authentication Etc all of these things are then individually instrumental and then you can actually see which parts are slower or faster which add up to the greater time that a request and response takes the next thing to understand about why open Telemetry is that we need to understand that the world before open Telemetry did have Telemetry there were vendors providing it New Relic data dog things like that but they were often their own very specific implementations they had their own sdks they had their own apis they had their own specifications on what the messages should be made up of it meant that if you wanted to change vendors it was actually very timec consuming and expensive in terms of developer hours just to like migrate from one to the other so you ended up getting locked into to a vendor with open Telemetry you have way less vendor lock in because they all speak the same language of sdks and also just like the message formats that they pass so you could actually take your data from one vendors another and theoretically their graphs and logs and all that stuff would still work so earlier I mentioned the cncf that's the cloud native Computing foundation and it's a really cool project it's a really cool Foundation they actually stewarded a bunch of really important tools like kubernetes like joerger uh for tracing there's a bunch of projects that the cncf has adopted to make sure that they are not preferring one company the other it means that these companies can pull resources together for their development and it's a you know a third party that helps maintain these things without outside influence as much before we move on too far into the actual steps to get open Telemetry working in your project we need to maybe point out some other terms first so those terms are you know going to be useful I'm going to mention them later and without defining them ahead of time you might be you know a little unfamiliar so the first term is OTP OTP is the open Telemetry protocol it is a Proto buff schema that is also then usually sent over grpc but you can also send it over HTTP and it basically just defines a bunch of props or you know Fields within this message that we're sending to our open Telemetry collector and then onto the rest of our measurement stuff like our databases such as grafana and jger or grafana and Tempo and various things like that so it's basically just the common message format that we care about and there are stable Fields as experimental fields are added and vetted and approved they then become stable fields and you can make sure that your application isn't going to die from one of these stable Fields disappearing after OTP we need to understand what the collector is and The Collector is its own service it's its own tool that you will spin up and use to take in data from the rest of your application and pass it onto your uh databases in the cloud such as Prometheus or uh you know Tempo or Loki or whatever right on graphon cloud or wherever you want to put it basically The Collector sits between your applications and your services and your back end and the databases where you want to store this data why is that necessary well the collector can aggregate data together it can scrub personally identifiable information out of that it can batch requests to the third parties to keep you from getting uh rate limited it's a very useful thing and you know it's better than taking all of that logic that it's going to do and putting that inside of your applications because then that means that you're going to end up duplicating a lot of effort and you know just having that in one place with the collector saves you from a lot of duplication and a lot of just redundant uh coding and setup and last of the terms that we care about is instrumentation so there's multiple types of instrumentation there's automatic or no code instrumentation where you just bring in a library and then you can maybe get Pino which is a logging tool wired up and sending logs to your collector you can get HTTP requests from like the node HTTP Library automatically wired up without you having to wrap it there's all sorts of things that come with the auto instrumentation node uh library or pack or whatever you want to call it that help us just get a lot more insight into our application without us having to do that ourselves then there's the idea of custom instrumentation custom instrumentation might be something where you're going to create a span for a very specific part of your code such as parsing a CSV or things like that right like there's so much that you could do with it that I'm not going to be able to cover but just recognize it as parts of your application that are unique to your application that you want to actually have measured and sent out to your databases for analyzing later and then finally there's the idea of Library instrumentation and that's where you might have a database driver like uh you know the database driver you might be using some other things where they would most likely want to provide some way of measuring their performance without you having to do that yourself and that is what a library instrumentation is in this case so instrumentation very important it's how we actually measure the smaller parts of a request and response rather than just the full full thing now that we have the vocabulary out of the way for you know a lot of the broader terms like OTL and what the collector even is what we need to do is start setting up our collector so the collector is a 264 megabyte binary of go code you can also install it via a Docker container and things like that and basically it's going to allow us to aggregate things like I mentioned maybe scrub personal information and just have one place that we're doing a lot of this processing rather than putting that into your own applications for one backend API service sure that's not a big deal you might not need the collector but as your application grows and you add more services it becomes even more beneficial to just make sure there's one place that's doing that if you don't want to take the time to configure a collector there's actually a really nice Docker container from versell that Docker container actually includes a preconfigured collector it has Jagger for tracing it has uh Zipkin for logging and it has Prometheus for metrics so all of these databases are set up ahead of time for wired up to the collector and able to be inspected and proxied to maybe a third party service if we want later but what fun is that I want to set this up myself and I want to show you how to set up your own collector too so there's actually five things that you can configure within a collector there are receivers which take in data and I'll we'll be talking more about these things in depth here in a second but there's receivers there are processors there are extensions exporters and pipelines and then finally there's another thing called connectors that's basically where you just glue together multiple pipelines for you know processing them together for composability we're not going to cover connectors very much here but the first five are very important receivers are how The Collector takes in data from your application it's basically going to spin up some ports to listen on and you're going to send data to those ports from your services and that could be you know with each Port you might have a different protocol that you're listening for might be TCP might be HTTP might be grpc the receiver basically is just the the entry point to this greater collector pipeline that we're going to have once the collector receives some data it then passes it to processors and processors might batch things together processors might also make sure that the uh various pipeline isn't consuming too much RAM they might also scrub for personal information the processor is what is basically transforming the messages as The Collector takes them in before they get exported out to a third party like Zipkin or Prometheus or whatever extensions are extra things that aren't directly related to Telemetry right they're more related to The Collector itself so you might have an extension monitoring the health of the collector itself you might also uh do some other stuff one of the things I found useful was making it be one place where I could Define reusable off information right so if you're going to have the same user ID for a bunch of different databases you're sending data to that might be one place you define it to prevent from repeating yourself so extensions might not be as common as the other things you're going to configure but they are still incredibly useful after processing our data we actually need to export it to somewhere cuz if it's not going anywhere what are we doing with it so the exporters are where we Define how The Collector sends things along through the pipeline to our third parties our databases wherever uh so you might Define an exporter for your logs like Loki you might Define an exporter for Tempo or Jagger for your traces you might Define one for your metrics like Prometheus it's just simply the place where your data after being processed then gets pushed off to and there's also the idea of another exporter called the debug exporter and this is going to be more useful for when you're just doing your own testing and fiding with things because it'll actually just log out to the terminal or you know the console basically when you are just trying to understand like what data is coming in are you even seeing any responses and it helps you maybe like debug if the collector's seeing stuff but maybe your third parties aren't so very useful exporters are simply where the data goes after it's been processed pipelines are the final configuration step and pipelines are pretty much just composed of the previous configs that we defined so you can Define extensions or uh you know processors or receivers specific for each pipeline so you might not be taking in TCP data for your traces and that might just be logs like CIS log stuff you might only be doing logs through your log pipeline so you don't care about the traces being there so you can mix and match and compose these but you can still reuse some of the processing that you care about so making sure that user emails aren't making it into your final data that would happen for all of your pipelines probably not just one or the other so you can kind of mix and match and compose these things together and that's really like the power of this configuration just being able to compose from the rest of it and that basically brings us to the end of what I wanted to cover with this video I wanted to make sure I covered some of the basics about open Telemetry I wanted to cover uh some of the history and why open Telemetry exists and I also wanted to dig into the vocabulary and terms that are going to make sense for other learning you might do when you're looking into open Telemetry yourself in the future I plan on covering nextjs and how to wire that up to open Telemetry so look forward to that and in the meantime maybe like And subscribe and check out some of the links Below in the description for more information thanks

Channel: Highlight

Views: 1,504

Keywords: highlight, observability, coder, software engineer

Id: ASgosEzG4Pw

Length: 12min 55sec (775 seconds)

Published: Fri Jul 12 2024