Grafana OnCall: Templates & Grouping

hello this is mate and in this video I will tell you how to connect new monitoring systems to graph alarm call using templating how to adjust existing templates how to put important data to your templates and how to change grouping of incoming alerts graph an uncle is open source product which is also available as part of graphene Cloud I will launch open source graph addon call in my local machine so writing Docker compose okay cool checking as you see it's running locally it's empty nothing here yet connected and let's imagine we want to connect it to some monitoring system but unfortunately it's not yet supported no worries because we will Deep dive into templating so I'm creating a webhook integration where phone call gives me URL and I will simulate requests from my monitoring system using Postman so going to postman I have some payload here this is alert manager payload by the way and let's send the request oh it was registered as you see grafana call was not able to parse title it was not able to parse buddy and if I will send more requests and refresh the page it grouped all of them we didn't know how did it decide to group everything so we wanted to have a control what is getting grouped and what is not getting grouped let's go to the configuration uh graph and on call process all payload which is going to it using templates and all those templates are editable through this UI here you could find different templates of those who are responsible for rendering for in-bap those which are responsible for rendering in slack and also there are some templates responsible for how graph and Uncle understand the payload let's start with web first thing we want to do is to change how graphene on call understand the title of alert we have example of payload here and let's think what could be a title looks like this field labels alert name we find a template called title and type payload labels alert name let's check one more time yeah it's a dictionary so it should be accessible this way and we see that it's rendered here we also could type something whatever we want here and it will also be added to that title but we are good for now let's save the template and send one more alert and you see the title is changed so graphene call is now able to parse title properly let's go back to integration settings and change how how does it render our message what do we want to know we want to Maybe let's imagine we want to see all labels so here we could use any Ginger instruction one of them is for Loop so we could iterate over labels and render them in some fancy way let's do it [Music] so I want to get labels first that's payload Dot dot labels yeah I see labels rendered here and now we go to trade through them and render them somehow uh for this we need instruction called four or label in payload.labels and instruction ends with N4 I see no error here so it looks like when the right way and let's render label yep all our labels are here those are keys in this dictionary so let's Draw Something Like This [Music] yep we see all labels rendered now let's maybe remove the new line double new line cool so saving template going to alert groups let's resolve the cellular group and let's send a new alert and here is how it's rendered remember before it was just raw Json now let's think about grouping so now everything will be just grouped in one alert group and it's not what we want so as you see seven alerts grouped here let's group based by cluster name so all alerts from the same cluster will go to the same group I am going back to Integrations settings alert Behavior grouping ID and I want to group not by the whole payload so all alerts with the same payload will be grouped now I want to group by cluster name so payload labels cluster and now let's resolve this one foreign two more alerts with this cluster name and change cluster name here to Dev and send three more alerts with devname one two three as you see they are getting grouped now to two different groups that's what we expected uh jinji is very powerful instrument which allows you to do a lot like almost whole python here possible thank you for watching see you

Channel: Matvey Kukuy

Views: 2,989

Keywords:

Id: S6Is8hhyCos

Length: 7min 59sec (479 seconds)

Published: Wed Dec 21 2022