Mastering Azure Monitor (April 2020)

hello everybody and welcome to the first remote UK show you scoop and as you can see this it's been a little bit of a difficult time for everybody I'm sure you all aware it's a little bit strange we're running a user group online without but without senior need you find people I hope you all keep you safe and keep them well and that everything is going as well as possible in this sort of difficult isolated time there's a big community and one of the big parts of the occasional user group is and has always been that we help each other app and that's we give each other advice and make each other care and our time so I hope that by carrying on these groups remotely we're still getting on a form to reach out to everybody and to try and make those connections and to try and find the way through all the challenges that we're facing with our lives this is a challenging time for me and rich as well mainly with this technology which is a little bit new to us trying to do a live stream trying to get to work it's working pretty well from mine and Rich's perspective but the other Richard Richard Ben well who so he's here to do our talk tonight is struggling to connect so maybe two or three in bed song in that and I guess that means that we probably all just go on me for a brief while and and we'll try and get the connections set up and get Richard all connected if not then we will try our best to exhaust every single Avenue we've got for getting all this connected but you know it's it's going to be a great talker thinking about monitor and looking after all the assets that we've got individual it's an interesting and difficult thing that one of the interesting things about the cloud is what you lose when you go to the cloud right the ability to physically go and touch one of the machines that used to run watch those little lights flashing to make sure all the thing it's still going has been a you few companies I've worked for in the past word literally if that hard drive lights weren't fast enough they were a support ticket so it's a interesting time free live you don't disturb that's really interesting how that's just gone let's come back well isn't this enjoyable they kick that was amazing for a second and back again may be detected too much waffle hi I will take that as a cue to require you okay wonderful everybody hopefully you can you can hear us all can I just can I just get some feedback in Q&A if you can if you can hear us all anyone you you okay just need just need a little bit of feedback from one of you just to let us know that you can you can hear the sound now I can eat Oh fantastic okay all right low defeat yeah okay that's brilliant I'm so sorry we're off to a late start we we had a few few technical issues with teams it's the first time we've tried to use live events I'm gonna I'm gonna natter on for a little bit while and I'll speak Richard BEM well I'm going to introduce shortly loads content up hopefully won't take more than a few minutes so just to just to give you a bit of a background I I met Richard initially I think it was about a year and a half or two years ago at the thames valley group yeah which he started the guys have squared up have put on some really good talks and went which offered to give us a talk about actual monitor which is one of the mainstays of squared up i thought you know we've never had anything like this in the user group and it just it just so happened that the week before i got to use the the actual monitor HTTP data collector api so i figured it would be really good to start learning about the features since it's it's a it's everywhere in the shore now so i'll i'll hand over to rich and let him introduce himself and looking forward to a great talk tonight sometimes rich yet my name is Richard as well let me just get set up with a presenter screen I have realized some people talk about drinking beer and I'm by missing a beer which would have just rounded this off just personally but I have to do without just the water so hopefully they'll let me know if you can see that content being presented should be I've seen my squaring editor yes I believe everyone Casino Richard right all right all right I'll get going then so welcome yeah my name is Richard Ben well and thank you for joining we've got a really good turnout which is fantastic to see I'm co-founder of squared up as rich Conway they were saying before starting squared up back in 2011 I'd previously worked for Citrix developing their monitoring and management tools and I left Citrix to bring essentially to bring modern data visualization to the world of the variety operations Scott up is now a global company as we said we focus on dashboarding both systems center operations manager and I shall monitor where headquarters and all proactive elements here in the UK just outside of London in in Maidenhead and despite the economic conditions we are still growing quickly and still hiring which which is good news for us I'm just going to check or soak up something telling me I've got bad quality hopefully you guys can still hear me yeah can you fine um so before I get going maybe 7 hours worth of content makes a half slice half half demos I wanted to just actually put this up before I forget before before you sort of hang up from the cool just there's a really nice of timely event coming up which is next week so next Tuesday there's this agile monitor virtual series from Microsoft themselves so that's fever there who's essentially the top dog when it comes to our monitor in Redmond and based on the content there that's it's going to be a really good event to follow I'm going to give you my sort of definitely unofficial take on agile monitor so if you want the official take on agile monitor Nyree suggest marking that out in your diary that looks like over three hours well worth well worth it another one just before I forget is just on our blog we have a nine part blog series covering pretty much every aspect of perversion monitor with a cover techyv angie lists there so if my presentation today kind of leaves you wanting wanting more has a kind of a couple of gaps and please do you go head over to that with but sort of take an approach there of the sort of zero to hero so for a lot of us we were focusing on system sensor traditionally so so a lot of us patter to move over to agile monitor and there's a bit of a sort of learning path there that we shared for everyone else as well so hopefully those two resources will be pretty useful and I'll mention those at the end as well when I've given this sort of talk or a very short version of this talk at a couple of events of ignite Orlando and ignite right London earlier this year I'd normally do a bit of a hands up poll Aussie that doesn't work very well over teams which is a bit of a shame but I thought I still asked those questions and sort of show you the sort of response I get so the first question I ask is who here is using Azure in production and you know this the sort of answer I get this sort of reflects most the people on this call today right this is usual UK user group but expect most of you are using Azure in production right so that's great everyone's got a genre that's pretty good news don't ask a follow-up question which is who's mastered their actual monitoring and I've asked this like time time again of the events now and this is the response I get it's it's really I guess concerning I guess it's why a lot of people come along to to these talks and sort of want to learn about them wanted to say hopefully together we can address this and get on top of our erosion monitoring a little bit more than we are today why do we struggle without your monitoring a sort of there's a few different reasons and I guess unfortunately pretty much every single talk I've given and and I've seen about monitoring always has to say like why should we monitor why should we bother monitor and that just reflects the fact that we're not necessarily always doing is much of it because we should do and not necessarily doing it the way we should do it it's not sort of working for us but there are you know really really important reasons to think about monitoring I think of it a little bit like this just in terms of some dependencies and order order of events obviously you've got some sort of goals with with your application you got you guys set out to achieve something by building that application and running in the first place and so with those goals in mind you actually want to be in control so taking actions you really take actions in response to information right you can't sort of drive blind you need to be aware about your what the current situation is and take action in response to that which of those goals so awareness is around knowing what's happening and observability is actually the underlying sort of data that you can get awareness from so subject some dependencies going from sort of goals all the way through to observability which is a bit more of a buzz buzz word these days than monitoring we provide the context of monitoring really we're talking about our awareness and observability so observability being the underlying data that is available about what's going on awareness is then surfacing that that date in a useful way and also that allows you then to control and take take take actions in response to achieve those goals so you know classics have a kind of corny car example right we're driving a car to arrive our destination generally with some some different sort of qualities here we ought to drive right there safely efficiently comfortably quickly and so on we're also driving to a changing of a change of direction maybe stopping to refuel servicing if it gets really bad to do those we absolutely need to know what's going on and we can't drive without any kind of awareness of what's going on around us it would be kind of ridiculous so dashboard yet and of course a windscreen knowing on what sort of going going on there in the real world and that's all underpinned by observability right so you can't build a car an engineer building a car you'll be a terrible car if it just had no observability you know what speed or direction or whether you needed to change gear or whether you're running out of fuel right don't be ridiculous to have a car without that observability so sort of how apps that are kind of mapped onto a real-world situation in an IT sense we typically Express goals like this sort of uptime performance you know success all sort of failure rates for our users security costs and they might all be wrapped up in a NSA layer a service level agreement obviously we're all living in a world of managing IT so we're all use the control bit right we're going to scale up we're gonna reconfigure we're gonna deploy a new version with some bug fixes we're gonna make you sort of do a failover or restore some data so we're very used to the control bit but really again you can only really be properly in control if you have that awareness of what's going on and typically in an IT world and it with monitoring that's that's we dashboards to really understand the big picture alerts to be notified of a change and some analysis or reports or allow you to to dig in and understand exactly what's going on an underpinning about those dashboards is again observability right so this is our underlying raw data that's coming out like easier application have metrics that can be collected and observed and surfaced into dashboards does it have logs tell me what's going on can you trace something end-to-end so we hope to do we need monitoring if you have you built an application and you know you can't monitor it you're already in control and you probably not gonna achieve your goals with that the other part of the problem with monitoring of course is why should you be in particularly care and this is the classic problem with monitoring is that no one really does care and a four is in charge of it if you're the business obviously care about the goals and if you're a really truly our DevOps team you're in great shape because as a DevOps team you're a single owner that's a whole point of the DevOps team right you have to deliver this for the business you are in control you've got to go do the monitoring you'll build that awareness and the observability into your into your processes and your stack so that's why sort of that really kind of pure DevOps team and does work so well typically though often we're still broken down to those classic separations have sort of ops or reliability engineers maybe who really need to work backwards from those business needs right you're trying to deliver this application and make sure it is those SLA s even though work backwards and make sure you have the information you want out of the underlying platform often the real problem and I can say this cuz I was one of the developers right cuz they love writing awesome code Chucky's out it works works brilliantly they then sort of forget about this really important bit right yeah you can't just develop an application that does something wizzy without it being observable and surfacing that awareness of what's going on so you can hit skulls so the message for developer is do you already write what encode great but be more awesome by making it observable as well and that's just something that hopefully so the developers can start to pick up especially with monitoring built into the platform like it is with agile a last just another persona who might be on the call today we want to be a developer or an SRE or something like that you may be responsible for multiple applications running on on say your your Azure platform and in that case you're typically going to be something like a cloud architect right where you have similar concerns but they're across multiple applications on behalf of the business are you in control your costs across all your applications are there security performing well are you being able to do this efficiently and consistently across all your applications so I expect many of you will probably submit yourselves in that that category as well we're not necessary responsible for one application you are needing to to sort of surface this information across many applications so that's my little pep talk about why we need monitoring hope regrets that are teed up there so I guess I think this question is why why is your monitor there are other monetary tools out there so looking at your monitor the benefits you know the reasons why is it's best for a job right it's it's the monitoring tool make it into the platform designed for as you're very specifically tonight interaction and all the bullets are platform features given how quickly things change in agile you know the most likely place you're going to find support for new services that come into our door is going to be agile monitor it's also cost-effective it's not entirely free I'll touch will cost a little bit is but it study a lot more cost-effective generally than a than a separate third-party commercial tool so that you do have third-party tools they they can help you with your own monitoring the benefits here are typically that you probably already have them you might have already one of these tools or a similar tool in your organization so you're probably thinking about whether you should use that instead their clock cross platforms they have that nice feeling of maybe being able to do more than just this at one one situation on a show and they're often more feature-rich than the more you gonna find make it in without your monitor there is a lot going on with our monitor you'll probably find a little bit more going on and some of these dedicated tools but again not not in the context of just a sure it's not really a either either or this one yeah it's not as a monitor or or that tool you already have you are going to need as your monitor because it is going to be performing that platform level monitoring it's the monitoring that's baked in that's where the data's going to come from you might need to send that somewhere else you're certainly going to need it and you're going to be utilizing those built in agile monitoring features you may also then choose to you know sort of use your your third-party tools right especially if they're sort of end a neutral or get that sort of overall monitoring solution that you need the decision point for me is probably that that area in the middle where as you wanted to do some of those things that those other tools do perhaps not in a you know as a as feature-rich solution or not able to do it across different stacks but certainly can do it so two in particular that I'll touch on today is sort of a p.m. type features so application performance monitoring and log analytics so as your monitor has those capabilities they're baked in I'll leave you to decide whether you want to go down that route or a third party brutal or maybe a mix of both all right so that's that's sort of the preamble out of the way we'll get sort of hands-on now a bit with with agile monitor this is the slide you'll see in the documentation or at the diagram you'll see in the documentation it's pretty much today diagram you see in every single as your monitor talk someone's going to put this up in front of you it captures what is there I always find it a little bit confusing it's the wildering there's lots of different things on the right hand side lots of different things to learn about obstacle metrics and logs underpinning it and there is a lot so as your monitor the logs and the metrics are our fundamental I'll touch on those more later really essentially everything on the right is underpinned by either a collection of logs or the collection of metrics from the agile platform and your application on the right hand side there's a lot there I don't necessarily see all of those as monitoring so Watto scale for example I feel is a little more than that kind of control right you're automating your control in response to to monitoring that the sort of the big dawgs that the really important features in in as your monitor here or application inside super powerful log analytics the a massively powerful the metric analysis is just sort of pervasive so that's that's being able to you know graph and analyze the performance metrics out of the platform so those are the three really important features I want to really kind of call out from the slide be of course there are others that are important to monitoring so you want to receive alerts about this stuff you want to - before this and workbooks are released an interesting alternative to the dashboard we'll look at so there's a there's a lot going on already on on that slide now my job isn't actually to use actual monitor date today in my role I normally sort of dip into it every one game for demos looking new features and so on and actually that can I helped me approach agile monitor with a fresh pair of eyes often every time I look at it I sort of have a have a fresh look at it and I find navigating agile monitor really really hard I find it hard both in terms of sort of conceptually and and the documentation but also within the yeah job portal it can be absolutely a bit of a maze and you can get very easy lost I guess that's true of the edge report for me in general but certainly without your monitor there's so many different pieces as as rich Conway mentioned a you know it is it is popping up everywhere and you can get very lost trying to find what's what's where and what kind of what does what its changing all the time this maze is sort of changing all the time things change name a lot without your monitor it has to be said they can be moved around they can be consolidated and often every time I dip back into the azure portal things are different or I get a little bit confused about about what's where there are absolutely dangers in this mace you can very easily spend budget and I'll also cover a couple of those when we said navigate through this but there is an amazing world out there so used to do navigate this maze there are some some great benefits as a just job it just it just a small request from someone to minimize the the team's window just to maximize your screen real estate brilliant thank you so when I think about navigating agile monitor psyche bit of maze it's changing with time definitely dangers but there's some great stuff there and if you're thinking that sounds familiar you're not alone and you're absolutely right so today we are going to be running the as your monitor maze it's a it's a very similar we've got different zones there's there's a lot to go on there and some great things to find and there's some great dangers but we are going to be running the azure monitor maze today so what do we got up insights fantastic we'll cover off that dashboards we'll look at the platform metrics and logs will look at workbooks will touch on blog analytics will look at in detail insights we won't cover so much I'll just point that out just let me call around here monitor an output monitor here which is great because the whole thing's the agile monitor maze there is a theme portal called monitor so we're gonna still work out what that is and why we should go to that to the different zone and we have alerts of course so now a bit of fun we're going to run the main these I'll take some different zones will we'll find out sort of how to navigate through them how to get to those to the gems that are lying there and avoid those dangers the best way to navigate amazes to to understand how it's built and what was the sort of thinking behind the maze and so what a minute it's just on this slides give a very short history of our monitor and that little caveat there is this is totally unofficial this is just build from my so the personal observations I've been looking this stuff yeah for a decade I remember when there as a report that was the Silverlight portal and so on so when this has just built up from what I've seen both in Asia and and other products that are going on in Microsoft so yeah close to my heart where we're actually squared up started so before what usual was was widely adopted we started building dashboards on top of the system center operations manager so that the monitoring tool that everyone has for on Prem but what happened with the Microsoft is that the team builds calm moved on to build something that time it's called off insights and that was really sort of a standalone service that really was something very similar looking to Splunk at the time but because it was a evolution in some ways from also in the team behind it were coming from INSCOM world it was very very orientated towards server monitoring and it was even pitched at some point as a replacement to SCORM which it really wasn't it was it was much more of this sort of Splunk like log analytics database so that came out of the service I'm not sure of the time frame I've got this other notes there and actually in parallel and completely separately as far as I'm aware Microsoft came up with another interesting service which was app insights and that was done independently and again on a very different portal and was focused on really the APM type functionality so looking at your application in detail end to end from the the browser code all the way through to your web code through to the database then you had a third different today within Microsoft who were the really agile platform team who are obviously baking in some some monitoring to the platform so if you went to the agile portal you would see some some monitoring baked in and not been sites and that mean sites were we're totally separate open sites got renamed to OMS operations management suite and sort of gained a little bit of momentum in terms of the number of different things they're trying to build into this and this really was the sort of oregon's I guess and what would become as your monitoring but certainly they were thinking very much about sort of monitoring within our ms and they rolled it some other things like automation back up and I was all part of this operations management suite meanwhile be the platform team came out with a nice new shiny portal so we're starting to see graphing and some other pieces built into the portal there and the app insights team eventually integrated him to be the azure portal so app insight started to appear in the portal and their separate portal was was deprecated and the same thing happened with OMS they sort of let him kill OMS they they repurposed you know some of the some of the pieces there what happened is they moved the logging piece of OMS into the agile portal but there's a point I want to emphasize here is this stuff was happening individually separately they you know in parallel and that really flows through today into what you see in the portal which is it explains why some of it feels still very fragmented as it's come from three different teams and three different places because because it has a really interesting thing happened with a pin sites which is they developed an amazing database technology for massive scale logging and data collection and so I think partly automation there but this this takes place called cousteau proved to be very very capable they used it within Microsoft for massive applications that they ran within Microsoft I'm not sure whether says want me to keep it you think things like Xbox Live or hotmail and they were using some app insights on on those sorts of services and it was proven to be very successful so they actually took that data platform whose Data Platform and made that the foundation for log analytics as well rather than the EMS platform so you're starting to get some some some columns consolidation and integration and then find leader this was I remember 2018 so the Ignite 2018 they finally announced as a monitor as a as a brand really first they announced data monitor as a brand and then you know a really sort of drove the integration in the portal under that brand and so really it's only being probably you know years you know just through outs of 18 months that they've actually been building something called as your monitor and really working towards that integration there have been some great steps so alerting for example is now integrated across the board you're still going to see the the separation of those sort of three different aspects of monitoring and that explains as I say a lot of the a lot of a maze and a lot of the inconsistencies we see so hopefully that gives a little bit of useful background and obviously there's some exciting things ahead with with agile monitor as it does get more consolidated and mature as I say underpinning pretty much everything we're going to see today is the logs database that originated I believe from from that up insights team it was a new database technology they developed designed originally fir for a pin sites you know think Splunk elastic so on it's a sort of sequel like it's massive scale it's a really really interesting database technology and I don't know about you I started developing when really it was only sequel Oracle and MySQL were the only three database design you about these days you know there are there are dozens of databases to choose from and then Cousteau is is one that that has proven very very robust for these particular applications in fact they you know it's such a great technology as well as underpinning now everything in agile monitor it's available as a completely separate as your service as your bait to Explorer I'll show you a demo about just very very briefly just to kind of give you a sense that this is a really important technology if you're developing an application you need a large-scale database you might want to look at agitator Explorer but certainly it's going to underpin everything with it with agile monitor there is a new query language for Cousteau hence what's called KQ Elsa Cousteau was the codename the project name for this new database don't think it's official anymore officially I think it is called agitated Explorer the Cousteau is still sort of lives on as a as a sort of codename and comes through into that query language called kql I really like a key well I am familiar PowerShell I'm familiar with sequel and KQ ologist is like this beautiful blend of the two it's really really nice I think if you do so try it out you already enjoy it I think it's probably one of the best things they've done in this area is build this database and build this language on top of it it's very very nice to use great developer experience and very powerful there was a blog on skype calm introducing you to kql so it's designed for anyone with a PowerShell or a sequel background and it kind of just takes you through how kql compares with the sequent PowerShell but you really is worth developing that skill and you'll find that a lot in the actual portal now I'm was going to do is just give you a quick demo actually I'm just going to show you very simple kql query so let me just hop over first of all to me I'll portal I'm not going to do this but just if I type in as your data Explorer so you see we've got your data Explorer clusters there so we can go and create one of these databases complete standalone as a separate as your service even though it's the same technology that underpins a little monitor so completely outside of agile monitor we can we can create one in two databases and use it within our application they've got some beta to explore adagio comm is a nice place both as a UI to express queries and run them against as your datum to our databases but they also just have this nice public sample database where we can we can experiment with it this is a a mock-up mug database covering sessions at ignite 2018 because see you've got nice typical tables with fields most these fields are within within a table here's kql in action it's typically within a UI like this has a bad episode of really nice intellisense so you don't have to put too much effort or too much thinking into running any of these queries so for example if I start writing conference is going to say you want a conference sessions table yet it does this pipe character so that's going to say take all the data in conference sessions and do the next thing so it's going to aware clause so there's a bit like PowerShell but sort of sequel like operations so where and now I can pick a compound at the table so where conference equals and just be the value so as to ignite the 2018 and again very familiar if I want then border that I pipe that through to another operation which is order by suggests that's the next keyword and then says what you want to order it by so as to section title it seems like the obvious one so very very easy to create these these queries just run there and across the large amount of data I get those 2018 sessions coming back so hopefully not too overwhelming I'm not sure how many of you are probably familiar with that either Rhian as your monitor because you're actually using it within your application but I absolutely recommend checking that out a little bit more sir that's a really nice technology so it's a data explorer and an Cousteau so we've looking at the history version monitor we looked at what it's built with which is which is Cousteau it's now time to run the maze and we'll check out some different zones and I'll take you on a bit of a at all we've got probably what like just going to right now it to do this so I've kind of try and try and move as fast as possible so the first section we're going to go over to looking at what's built in the platform from an agile monitor point of view and we'll just I'm going to spend just a minute - poor governments on dashboards are not going to do too much but just many focus on on what's built into the platform when it comes to as your monitor so this is probably what you're almost familiar with so if you browse to a virtual machine or an app service you're going to see this view here let you select select metrics the great thing about these metrics and dogs is it's free and I mean that in two ways it's free as in it's no effort you just get it you don't have to set anything up you don't have to configure anything you don't have to Craney storage accounts it's just there so given lots of different resources in our show you're going to find that that monitoring power band those metrics available it's also free in the sense of no charge so you don't get charged for this is a built into the cost of any resource it is only short-term retention it does change I'm not sure what exactly the retention level is here is sort of in order of the kind of days or a couple of weeks so it's not something you can use for long term so reporting or analysis you'll find this is mostly metrics might even get confused myself just about what logging is built in and what sort of magically happening by a biased of log analytics but I'm not trying to point that out as we go along and sort of two classic examples I'm going to show you just in a demo are an app service so is running as a service and virtual machines I'm going to do this in the demo go just call this out on the screenshot because this is part of navigating the maze what we're going to look at is a resource so this is not on a blog analytics or app insights resource this is on just a regular as a resource so an app service for example and the key place to go is this metrics you'll find a lot of some of the other links either you do need to set something up or they're there perhaps not where you're expecting them to be so metrics is that it is the reliable place to go find this information on any resource so let's just switch back over to the portal and what I'm going to do first of all is just come in to do a VM how many of you are still working with VMs or whether you move on to containers or ID service now but at least sort of a VM is its familiar to us all so we drove down into at the N there is the monitoring section here that I'm sure you're all familiar with what the Microsoft thing we've tried to do is sort of can't broadly make everything available there everywhere you go which can be confusing because what everything necessary is set up right now and is available so as I said if you go into metrics you will find the metrics that are baked into the platform you're not paying for this you're not having to set this up they are just there so they click on metrics it's scoped to the resource the namespace for the metric is Virtual Machine hosts that's essentially a sort of a platform namespace and then we can pick out any of the metrics and these will be generally very familiar right so in the case of VM these are all VM level metrics we're not looking at the metrics within the OS itself so we're not looking a windows level metrics this is the metrics for the VM so the kind of course look at CPU so let's just pull that up I'm not going to do too much on the metric interface here this is of something you can click around and explore it's it's it's pretty nice you can have multiple metrics different little types of visualizations and so on and you kind of drill down so it's quite quite intuitive looking at those metrics if I switch over to logs which you might be tempted to go great I also want to look at the logs for this VM you get this screen and this is where it starts to get a bit more complicated so you do not have o s level logs baked into the platform you don't get that for free you don't get it out of the box in order to do that I'm just gonna click enable they're not going to do this I'm just going to show you the experience in order to do that it needs to check that the Microsoft monitoring agent running in the VM is available I think that was just a pause they're just making sure the Microsoft monitoring agent was um was accessible and then it's going to ask you well if you do want to collect logs you need to now setup log analytics I'll come back to that one of those zones so those are the two main areas you'll probably look at when it comes to monitoring in a resource like like at the end saying to examine box logs actually piers here is part of log analytics I'm going to zip over to a resource group that contains a share it's going to contain whoops parts Unlimited and get familiar with some of the example ok let's come over to this I'm going to come over to this app service running a little slow so this is another agile resource so here we're just looking at yeah typical as your resource of like an app service and you know you're familiar with these lots of views again if we come down to monitoring metrics you get out of the box so here we're going to see metrics for this app service so again the the namespace here is app service standard metrics which kind of means the baked in plan for metrics and you're going to get things like this so you can look at the CPU time of this app service the logging is a slightly different picture with that services because there is some logging I believe some baked baked in so if you're logging from your application that's going to be collectible here so we can specify that we do want to collect application logging this is all a platform level feature so not necessarily part of agile monitor in a sense of the log analytics of beta platform you can collect logs from your application and put them either too short on file system or long term blob storage you may be familiar with them putting those logs out by a say something like log stream what you can do it like with agile so with the virtual machines and log analytics if you do want to store these and you can you can send them to log analytics so here we can set up what's called a diagnostic setting to essentially pump these these logs over to unlock analytics database where we can then store them for for some length of time and do some some analysis on them using agile data Explorer coming back to metrics and just finishing off the tour of this zone we're just going to quickly pop into the adjacent dashboard zone so do I just grab a graph like this that might be useful for you just to keep an eye on on CPU what I'm going to do is pin this to a dashboard so I'm just going to click pin to dashboard this I I'm not totally sure about I think what happens is it it's gonna store the most recent batch board you sort of visited so it kind of knows what your current dashboard is but I don't like to rely on that just because I'm not quite sure exactly which dashboards in context it has saved in my browser so I'm going to just pin to another dashboard and I'll use this an example of creating a new dashboard come out to create new name you can build in a subscription obviously and then interesting checkbox here it's actually sort of makes a default recommendation say let's publish this to the dashboards resource groups and a built in dashboards resource group you can do that so your dashboards are in the same place I quite like the idea of putting the dashboard actually in the same resource group as your as your application I think actually parts unlimited you if you deploy that out at the arm template because a dashboard is just a magical resource like anything else it can be deployed out with an arm template and deployed into a resource group so I quite like the idea of keeping that self contained right just want this resource group and the dashboard you've got for this to the gratification get swiped as well quite quite like that that relationship so we're going to create a new dashboard on UK within the same resource group where where my parts unlimited resources are and hit apply this confuse me the first time that it's created it and I think it has also added the the tile onto it so I'm just gonna come here I need to refresh now it doesn't look like it's got its opportunity to refresh come up to the containing resource group oh come on agile it's gonna make the demos a little bitter little interest there we go dashboards appear as that one you can see that and hasn't appeared yet hmm I'm gonna come back that find out where that is so just crash what I'll do with other parts of the demos publish more things to that the same same dashboards hopefully we'll learn we'll build this dashboard out let's see if we can get it created first yeah there yeah so it's in this resource group with all of the other parts unlimited resources and it's just a resource a natural resource like any other as your resource in fact if you go to it you kind of get this quite ugly view and then you have to switch to the dashboard view so there's our new dashboard as a UK and that's the graph that we thought was quite interesting so we've pinned it on to the dashboard so nothing too too much whatever what we'll do is we'll build out this dashboard a little bit with some of those other demos so that's the what you get at the box so everything I've shown you do neftis have been able anything new or start paying anymore right you can start dashboarding for example with those metrics that are baked into the platform what I'm gonna do now is run over to another part of the maze and have a look at lap insights and we'll have a quick look at alerts as we do this with our pin size is one of I think the most exciting ones are features I think it's where you get most or at least claim to value so you get to get a lot of value from a pin sites very very quickly not a huge learning curve I really really like it it is quite simple to to get up and running with this you need to create an app insights instance so that's different to this baked in platform stuff you do need to kind of create now a new agile resource which is called an applications insights resource what it does is monitor web applications HTTP based applications so you can monitor traditional is running on VMs you can monitor app services or you monitor service functions across a typical array of languages so support for each one of these it provides you with application forms management features so like some of those APM tools we see from external vendors so things like being able to check the availability of your application I love this I'm going to show you a demo this one in particular I think is probably their the number one thing you should do when when monitoring an application is just to make sure it is available and you you're the first to know when it's not you can also then inject JavaScript into the front-end web application and essentially we run that within the users browser so you get what's called real user monitoring you can actually see load time is in the browser so not just the backend you actually see what your user experiences you can then monitor obviously what's coming in through your HTTP requests it can then do code level monitoring so if you have you know application for example it can give you the stack traces and the time spent in and each method and it can also tell you about the queries the outbound queries you're making to databases so given an incoming slow query you know an HTTP request takes five seconds you can very easily then identify the bit of code or the database query that's taking taking a long time this is a huge avaria i'm not going to spend too much on this but i'm just going to point you again in the right direction to show you what you get very quickly the screen show on the right I'll show this in the demo but just to point out we're now going to now be looking at a specific agile monitor resource which is an application insights resource so this is not now looking at the app service itself or the VM or the function this is a switch obturaculum insights workspace the two key pieces although there's quite a long menu there but there are around the edge of portal the two ones I really wants to highlight is availability which is this this URL pings but again checking availability of your application from from really any other Azure data data center and the alerts metrics and logs that's where I think that's what underpins the entire of app insights the rest is actually derived from those so let's say something like failures for example there's actually just giving you a prettier view on top of your logs you know performance is doing the same thing as giving a pretty-pretty overview on top your locks so availability is worth looking at and then those three alerts metrics and logs that are independent so let's have a quick look at this as well as we get a chance so I'm just going to come back to my parts unlimited resource group to where we have a power app service and that's where we also created our dashboard if you want to create an app insights instance you can do that within any resource group I suggest creating an app insights instant for one particular application it's I think it's kind of true true of a number of different things the easier to focus this entirely on one application rather than have one app insights trying to manage different disconnected applications which can which can be quite hard so create an instance I'm just going to do this just to show you what you get out of the box if you won't take too long I did say richer Andy I'm not looking at a queue and Isis if anything's coming up from the QA please do a surfaced and I'll try and answer any questions that I can do while waiting so let's just wait this to come up this is being very slow sounds good does anybody does anybody have any questions while we wait for sure must be my no I think I think we've all been doing a lot of waiting especially on North Europe over the last few weeks months all right so this is my sandbox subscription here the resource group which actually contains the application so I'm going to put my tap insights instance into the same resource group is my application and I'm going to give it a name that I think kind of in spaces and somewhere more sensible let's just have crate on there okay so we'll come back to that I should appear in a second then we all then we'll see what we get a box with it yeah great all right so how'd we do Ken so we've got a napkin slice instance now within this resource group where we have our application something I think when I started using agile monitor I got really frustrated with was you create something like this you're really excited to see all these APN features and what you do is you click around and you you don't see anything with no data it's all blank and you it's quite hard you sort of lose enthusiasm for it just because you don't you know get to dive in and have a play with all this I would recommend something like say that parts Unlimited arm template where you you talk pre-configured for you so you sort of set that up and back running for a day you'll come in and you'll see the loads of great data in a pin sites so you don't get much out of a box it's not collecting anything it's not connected to any app services it doesn't automatically start monitoring anything it's just now a an instance of app inside sitting one thing you can do very very quickly I'm just going to show you how to do this he's adding available variability tests so my parts Unlimited is this this website here grab that URL so before I even monitor anything kind of on the backend what I'm going to do is just click add tests I'm going to monitor this application from the outside in this is just going to be a an HTTP request against any endpoint so really anything that has an HTTP endpoint you can you can you can check or URL ping test great what's the URL it's that skip skip tested from five different locations maybe where my my branch offices are hit create and just so I write the bottom they just said alerts enabled so what's also going to do when it creates this is create an alert for me as well so actually just by doing that I'm now going to be alerted when my when my application goes down or the response time is too high so that's all being I've been created if I just jump over to alerts and all will talk to an alerts and then we'll look at the rest of the APM features quickly so don't have any alerts make sure you click on the alert rules that's where that's where actually your lurtz are defined so what we should find here is we have the alert rule to say give my website is down then send the notification so here's the alert rule for that availability test we just just created I come in and give you a quick overview so probably all I'm going to say on alerts I've got one slide to the later we if we sort of wrap up after the demos I'll just give you a quick sort of flavor what is really nice and said about the alerting in agile monitor is it is now consistent so anything in sends that has this interface and you can manage what you learn from me across across anything that's creating another in a shell which was not always the case your alerting was very fragmented as well so it is a it is a place where they were really invested in in making this consistent so there's a question while we're waiting and one up somebody's asking one app insights per application what are we defining as an application if I have an architecture with ten micro services as function apps is that okay to have all those using a single app insights yeah exactly never never a right answer I'm not not sure what the right answer is to be honest I mean I I would be tempted to to have individual app insights instances just because actually one benefit of the kql and the azure data explorer querying is that you can query across pack insights instances so you can pick data route and say okay you know what we are in this timeframe show me you know showing me data from multiple different app insights so I can I can get a sense of what we're going across on across those micro-services the downside obviously is just the overhead of management of each one of those but again just like that - water quality idea that if you were to just deprecated remove the need for one of those particular micro-services iraq insights gets removed out as well right you don't have to worry about then having the inconsistent data you know in a centralized that incites instance first are typically the that i write the overhead of managing these things versus actually having these these discrete databases of information but as i say one one thing that they have got right is being able to query across different data sources so don't feel like you'll be stuck only ever looking at sort of one bit of data if you do that do you got one more question and then and then i'll let you all get on leave you in peace and let you carry on would you recommend using a pin sites for debugging an application ie if running an API locally would it be ideal to send the telemetry to app insights or send logs locally is that locally as in on-prem or within within an application sort of app service or something B I think what we're talking is interesting to you you typically get dev level of logging right so the developers of logging already understand exactly what's going on with the application making sure it's working as expected and then you have the need for logging actually when it's in production you really need to troubleshoot a particular issue and understand what's going on and yeah for that latter case you need to be make sure that you are surfacing the right logs into something like app insights so your s Ari's or even sort of you know high levels of support can very easily then troubleshoot issues from those logs without being searched you absolutely can collect logs from from anywhere but probably running in actual so if you do have is running on Prem for example you can upload those logs to app insights as well typically there is a developer you probably is going to go to the Thank You space you can find those logs just just when you're focusing on an issue when something's happened in production and you want to find out what happened retrospectively then then typically going to need them in app insights I've got you yeah someone else is just asking this is a quick one I assume that the ping test can be to any HTTP endpoint or does it have to be hosted in the short URL you want to monitor the uptime of Microsoft comm or whatever the go-ahead that's what I quite like about it so even if you're you know you're not ready to do APM and you're you know you're not necessarily even using agile right now for for those sorts of applications you can start to use app insights has a I don't know whether you're familiar with sort of Pingdom or alerts and I taught things like that so you start doing availability monitoring against your your application raise alerts get emails push that into some other alerting system like pager duty or ServiceNow or something like that so you can use it as a standalone monitoring it all up to do a PD all right facto alerts this is how Allah look three different things what's the resource you want to alert on so in this case it's in this particular case it's the availability monitor which had been created as a new as your resource in our resource group so when this availability monitor has a condition then you write the condition so whenever the failed locations is greater than or equal to two okay so a little bit of them a bit of redundancy there right so one test from 100 Atta Center failed we're not gonna make our CD application is down if it fails from two different locations then we want to be alerted new course you can configure that and configure a lot of different conditions there and then action groups the one thing the box here will no actually group select its you will need to go and create something to do with this this alert and then this is a space where they're they're developing it quite quickly so there are some some new features in here like per subscription you can have a like an admin action group right so it's just going to go to the admins of of that subscription for example here I've got our we test its my email address rights this is just going to go straight to to my square common email address or I could hit a web hook or order us of anything else that I need some agile automation for example to respond to this for this email I started to this earth alert so there you go so I've got my when my variability test has two or more locations fail then essentially send me an email and you can you can modify the details of that email and then save so that's just touching on email sorry alerting essentially all alerting anywhere you are and as your monitor is going to be the same right for a resource by the specific resource like an app service or a VM all for something like a log analytics or in our app insights workspace detect a condition and then hit some actions alright let's come back to slice but no not slice I'm showing you the best bit of happenstance let's come back to I created this app insights instance what I want to do is just show you one that has been connected to an app service so very simply you can connect just by coming to my my up in sites resource here this is a good example where I just said very simply and I'm just gotten where you were where you can see it ah Diane where is it where you configure a pin sites someone tell me on check you know completely just must come from my mind I'll come back to that but one of those is where you where you set up the app insights but I've completely blanked on it and we come back to that's why it wasn't the wrong thing sorry I was opening up inside run the app service that's why I've been in the app service is straightforward to essentially connect it to this app insights instance once you do you then start to get the really juicy features of that insights so what you get as I say is the blogs so I just have a quick look at what we have in logs here so this is now going to be logging everything that's happening with your application depending on what you've enabled so this will be every single HTTP request that's coming in so you can familiar with something like is logs it's logging every single HTTP request that's going to go into this log file the so is code level instrumentation if you enable it so every single method cool how long it's taking every single sequel query you're firing off to your back-end how long that's taking so you can really really build out an amazing view of what's going on in your application another one as well as looking at load going through I'm going to look at failures so exceptions that are coming out of your no js' function or your c-sharp application so we have a number of different tables here that we're going to start getting once we've enabled app insights so just as a fairly obvious one if I do request oops give ID requests run there that's just going to show me all of the HTTP requests and come in and essentially sort of showing that's enough is log but I can obvious then start to query by by failures or response times and so on so that's the underlying data we just come back up here what you get in this investigates section is pre-canned kql queries so further requests count so when it failures happening you see here this a very well should do it's just a sort of template application although we you know weirdly we did have a failure here you can show the kql so I can save human logs so I would like to see you know how to query the the failure is or the response codes or something you see that the response count some lawfare l request count so if I want to show how that graph was built I just click view and logs it's going to come over to just that previous view with Priya pre-populated kql query so great what way of sort of finding out what you duplicate well is is using this approach that's just coming in to the right here slowly there you go so that's the kql query they're using to generate that graph that we have we see any investigate panel and of course you can modify modify the kql if you want and and do some more research if you don't want to take something like this and pin it to the dashboard we do the same thing so anyway you see a pin you just kind of click it it will put it on to that dashboard so I'm just going to this time say we should be just current dashboard and that should appear on a dashboard in a second come on all right so recent dashboards Asia UK great so that's [Music] all right so that's that graph is now going to be on that dashboard it's gonna take a little long times I'm gonna come back dashboard later now you'll see some more stuff on it so hope you can get an idea of what you've enabled this you're going to start to see start to see a lot of information that you can use to troubleshoot your application also to understand what's happening as well as monitoring to what's going on on the backend there you also get sort of yeah with with athletes eyes come out of the box once you've done this is your analytics on your usage so if any of you are familiar with Google Analytics where it's telling you about how many users you've got arriving on your website how many are new users how many returning users what their flow through the application is so you can set up here essentially steps in the funnel so you can see whether they're sort of logging on signing up you're clicking on a link and so on you can start to understand your your users usage of that application so much more sort of kind of business level view of this application and whether it's achieving well what is set out to achieve not from a performance point of view but from a functionality point of view so you get that as well with without insights so it's a very very powerful I'd say what you need to do is is enable that on an app service service so enable that on this app service for example then you're going to get a lot of value that you can then start some pinning onto that dashboard this is what I was missing so I've come down to settings and app insights this is where it should say I'm already that it's attached to that happen sites instance right so actually in this case it's enabled through my essential man I think my web config file so in that case is enabled sort of by the developer there's a put into the web config otherwise you can just click on this button say tell an application insights and I really like that about the app insights you can just browse the resource and you said ok so monitor but it's a function or an app service just say turn on that insights and it all just works so I'm so you kind of recommend experiment and turn with a pin sites is one of the first things you do if you do have a wave-like application which we've got another question if your I - yeah to answer this we go along dippin is asking what is the rate at which the database is able to ingest log data good question they do have essays for agile monitoring analytics an app inside its I'm not sure that there is a certain latency for for logs like that it's not higher I that's very efficient where where log analytics nap insights are a bit frustrating is there is a there's quite big latency on those alerts that I showed you so unfortunately there is there is some some delay you know can be minutes before alerts are fired generally though the logs are pretty fast and very high scale there is some documentation I don't know the numbers but there is some documentation where where Microsoft say you know this is what you can expect in terms of kind of latency and unscalable logging cool okay take another question now yeah okay well things like funnels and user flows work if you have services split into multiple app insights instances yeah good question I don't know the answers I don't actually use that part of app insights terribly I'm much more familiar with with Google Analytics typically what you would do with the funnels that's your your user experience right so typically what you're gonna do there is put that on your your front-end yeah your your ultimate front end even though that's leveraging you know 10 back-end micro services so it's not looking at your your flows through the Microsoft sees that those funnels are looking through your your sort of front end application experience if you have multiple sort of front-end applications and user logs onto your corporate website and then comes off into a partner you know website portal and then goes off somewhere else my feet you will lose visibility there I don't know whether you can query across multiple instances there per typically you're going to be looking at things like they come to your website they go to the product page great that's really good news they watch the video that's great news then they sign out sign up that's great news and then they click through and they you know they do do something else they download that that white paper or something so those are the sort of funnels that you typically use there we're good questions great questions I'll move on I hope you can get through in the next next sort of ten or fifteen minutes log analytics it's the big one in national monitor it's the Beast very powerful use with care and there's a couple of massive kind of coaches here like app insights you created as a as a new agile resource so you create an instance of raga analytics workspace is the name of the resource and it can collect data from a number different places I'll show you this when we just create one quickly you'll see that it's very very VM centric very very orientated towards the ends that's what that sort of where originated so instant it's going to ask you you know what becomes you want to monitor whether you want to add additional solutions to collect more data from those VMs and so on so it's really primarily designed they're originally designed to collect logs from within your VM and performance metrics on the OS level of course remember with platform metrics you don't get any OS level metrics you can't tell you about what's going on inside the VM it can just tell you what's what's happening from the VM point of view all those very server orientated vm orientated it can now start to collect logs from other places in the agile platform so storage logs want to see was having a storage or auditing activity logs is great if you want to actually now audit what's going on across your agile environment you can set up centralized activity logging and there's the essentially additional logs being added all the time so if example containers now keep any T's you could push logs through into log analytics you you do have to be careful with with log analytics one in particular I'll sort of show you when we just look at the portal the I don't know whether this has changed it could have could have changed yesterday for all I know but the last time I looked the configuration of what data you collect is set globally sort of for that instance so if you say I do want to collect is logs or I want to collect my system a log from from Windows you're going to do that for every single this is reporting to this log analytics instance and with many of our enterprise customers who have been been looking at either using this for on-prem monitoring of monitoring their their on Prem servers and/or VMs lift and shift IBM's in an agile or they've been very badly burned by this because they they they experiment with it they turn on log collection of performance data collection they you know they want to collect cpu that they do not want to collect cpu from every single server across their environment you know the sort of the the default rate so the last time I looked you weren't able to to configure it honest VM by the EM basis or or a group of VMs so be very careful about enabling logging especially in large environments you can also connect this to scum so if you do have some running on Prem Thor on Prem vm's you can essentially leveraging leverage scum to send your data up to our Shore just once through through that scum to the log analytics collection a connection rather than independently from each VM when you come into the log analytics instance this is a resource in in a ashore it's funny because there's a lot of things again in the side panel the one that's absolutely critical which is basically the whole resource but doesn't stand out necessarily is this nobodies log analytics it's bad thing there is is what log analytics is is it yes you're a database with every single kind of log coming from over anywhere you want in that one single database so game very confusing if you just browsing around you click one of these things you you create a log analytics workspace it can be a bit deflating because you're not quite sure what you're looking at but it's that one one section there on that that panel which essentially underpins what log analytics is so let's come quickly back across I'm not going to create the legality to workspace now it's a straight forward just like we did with with app insights of the create an instance again you might decide we've depending on your use case here so going back to those intro slides if you're a part of a DevOps team you probably are looking at creating one of these again per application you again might want to do it by the purpur microservice all sort of one for kind of a logical application of grouping microservices it's meant to register where you're where you're sort of management boundaries are and in general you will be able to query across instances so again you have that but but you might want to send to set it up so once per per logical application even though that cloud architect and you're responsible for what's going on across your entire environment you might be thinking about more of a centralized logging so if you want to collect activity logs from everywhere in actual so you can you can audit who did what and when you probably want to set up a some more kind of global instance here one one subscription or tenant and put the data in like that so again depending on your use case but be careful because it's not very fine-grained configuration so if you think I'm just gonna have one of these and it's gonna it's going to monitor all of my applications you're probably gonna get stuck or it's going to monitor all your VMs you're probably going to hit it sort of massive costs so I'm going to come over to so here's a log analytics workspace again kind of confusing so on sidebar here activity log I am tags right that's sort of just your general staff locks no that's not interesting either quickster not really not really not really looks right this is the thing you're interested in this is this is where all your log data ends up you can see here like I showed with data explorer there's a number of tables in log analytics what you sort of get however the box is this one so this is the one for all of your VMs so you're going to start collecting data from various different VMs and stories and these different tables you know event being probably to the Bur yeah the classic one rice is going to be your event logs coming in for Windows or and also from from Linux as well you can add in additional solutions I'm not going to go into that today but there are this idea of sort of plugins here add-ons that know how to collect certain data so they they give them Microsoft monitoring a agent additional rules for collecting data and it will end up in your database but I won't go into those today you come down to virtual machines this is where you can see which virtual machines are plugged into your log log and little workspace so with a vanilla workspace you're not gonna be collecting any data and this is going to show that none of your virtual machines are connected this is quite nice what allows you to do is see every single virtual machine in your in your tenon head for the docks you buy by subscription and you can see whether it's connected to this workspace or actually whether the VM is connected to another workspace this is a little confusing at a reagent level you can get a VM to report to multiple different workspaces but it's not what the URI tells you the UI makes you makes you think you can only do one I think I can explain that later why they've done that so it can essentially they really only want you to have a VM reporting to 1 log analytics workspace although it is possible to have multi multihoming there again you can do things like which storage accounts you're putting logs in from and what other activity logs you're collecting as well ok so there's a three sort of classic sort of sources data sources for Falak analytics so that's looking at your data sources that's where they end up the other one I'm going to highlight which again coming back to the maze you just wouldn't guess it's here you're going to set up these virtual machines to report to log analytics you're going to start getting data and then you're gonna say well why is it collecting these ten metrics where do I do I configure that yes yes I want CPU but I don't want now then wants a dis space you know I want I want another another metric or which Windows been logs is it collecting and it's not obvious where you go for that you certainly can't do that if you go down to the end or in this view here and actually it's tucked away in this advanced setting so this is the other panel that I'd point me towards when you do do log analytics it's a bit quirky the last time I tried this on Chrome it it didn't work it was it was only working in edge and it's because what they're doing here is they're embedding that old OMS UI into the new agile portal so this is actually you know an old bit of UI it came from the the OMS days and they've just ported over Bible embedding into the portal here so what we can see is we've got different types of connected data sources you can go and manually download the agent and configure it to report to this workspace so if it's still casual it's quite easy to set up at the end to report to this workspace if it's INSCOM you can just do a one-click report everything from SCORM through into log analytics or you can manually download and do this and this is the one I really want to point you towards is the data so this is where you find out what it's collecting there's not an obvious place to look this is this is really where you can figuring what logs and date you want to put in from your data sources so with Windows Event log this is where you would add in different event logs you'd like to collect so put in the name there can Plus this is also where you'll see what performance metrics is collecting and how often so as I say if I go and add in a hundred different VMs here across my subscription I will be collecting these metrics every minute for some of the ends and you really can spend a lot to be a bit careful tune that down decide what you want and use log analytics carefully to two segments ideally your VMs and your data collection same with is logs so your do you want is logs yeah cool sounds great that's now going to be enabled for every single VM I'm gonna collect is logs from from everywhere and things like you know domain controllers you start collecting generally logs with the main controllers not necessary is but also they've been logged you're going to end up filling you're filling your database and spending a lot of money so that's where to explore that and to configure that this does have a lot of data in it's pretty pretty smart a little bit like a pin size you're playing directly with kql is quite hard I don't want to recent tie powder you know if I take you I'll query here just off the top of my head is quite hard to do that so if you want to find some pre-canned queries and some nice pretty results the place to look here is workbooks so we've we looked at dashboards briefly we look for alerts workbooks are great kind of really interesting there but somewhere in between a dashboard and a report and a sort of your view you know they're just available here so for example here I am in workbooks for for this and I can look at some of these you know some of these workbooks that are that are out of the box just going to choose agent health and a little bit like a dashboard it's going to have a number of tiles or driven by kql queries against against those logs we should get some interesting data come on you hmm so a summary I guess and then we should have something like this so you can start to do some really interesting visualizations appearing well you know I've got a little at the ends and I've got my key key key key metrics and properties there's a lot of different workbooks and you can also just like dashboards create these through through arm templates they're relatively sort of new and building in into the visibility and interest in the community so you'll see a lot more coming out again just like with anything else if you want to pin that it says to the current dashboard so that's where hopefully it's gone to the right dashboard so I'm gonna go click view Doh dashboard and I should be in this dashboard we created earlier yeah so if we have the CPU we've got let's just click oops edit so the CPU from earlier we did that sort of the first time and here's now our workbook so when you see some really fancy dashboards on time and as I was not sort of working behaving maybe nicely enough me to do this so say for example in the meetup I think rich Conway there who added in a screenshot of a dashboard you can get this is this is sort of how you build them and generally there's some really nice visuals coming out of the run box so that's a nice hexagonal status map for example is is a particular rum book so you start picking out some really nice visualizations from from the drug books and building out in these dashboards I'm just gonna switch over to show you the dashboard you get for example when you deploy parts Unlimited just to give you an ID yeah we can do with dashboards so this is sort of other box without arm template when I create this I get this dashboard and this is then you know the key metrics for for that parts unlimited resource group that we saw so requests response time you see that when I mouse over I can see I had high response time and failed requests at the same time and so I can start to start to kind of correlate what was going on and you can see they've used just another little panels to give them give them titles and so on so there's quite nicely started building these dashboards I find it a bit clunky hard to navigate around but the great thing is that they're built into the agile portal there so that was a brief look at mug analytics what you do sorry that we did instead of do do more there but kind of hopefully gives you a flavor of what you can what you can collect last one who's going to wrap it all off zone six monitor just monitor insights monitor is an interesting one confuse me the first time I saw it you can just search for monitor in your agile portal it is a a global UI right you just go to one monitor it's not a resource it's just a thing bake it into the agile portal so you go to monitor and you have one UI now across everything we just see so 1 UI across all of those metrics logs alerts workbooks and so on across all of your resources so whether that's the baked in platform metrics or your logs coming out about insights or your log analytics data you're going to be able to query that in one place so let me just give you a quick orientation of that so we've been digging around and looking at different resources it can be confusing if you come in to monitor so there's only one just type in monitor we're now just in a UI here that's going to be very familiar it's some of the UI we looked at across there's other resources there's the one place now to look at everything we might have set up so alerts is going to show me the alerts I've created anywhere in my my Asha person descriptions you know whether that's an alert on a pin sites or an alert log analytics I'm going to be able to to see here and manage those manage those rules if I want to report on some metrics I can do it here it's going to ask me for a scope because I'm I'm just a global UI at this point right there's no what I want to look at so it's going to say select scope so this is where we need to be a bit more careful do we want to select a particular resource and pull out those platform metrics because maybe we haven't enabled blog analytics or do we want to point it at a log analytics or app insights works basically we do want to get those those richer long term values so for example if I search for that VM we looked at so this is not enrolled in in log analytics so I can select that VM hit apply and you see you have multiple scopes there and I'm back to where we started right I'm now looking at a platform metrics for this particular at the end but equivalently I could have pointed it at a log analytics instance and look at the performance metrics coming out of a lot of analytics instead there's longer term same with logs if I click on logs here I can now connect to anywhere where logs are stored so that's going to be any insights instance or any log analytics instance where we have logs it's going to ask me to do the same thing selector selector scope so here it's connected to sandbox logs that I could switch over to my log analytics instance and query out of that as well and just to finish off sort of an area an area that I'm not going to really go much more into is we looked at the history of agile monitoring we saw there's something called app insights now this is one of the main things we looked at today they've taken that insights brand and created other different solutions so app insights is what we looked at earlier and it's not to create out other insight solutions like virtual machines insights containers insights and so on I'm not going to go into them today but that's where they're sort of trying to build out much more sort of resource specific monitoring so with virtual machines hopefully what I'll see here is the sort of health stats that I'd want from a from a virtual machine you know is it you know what what is the CPU memory disk right without having to collect that in a in a log I just want to get some basic information from it we also want a rollup of Health for example so you're going to start to see more and more solutions coming out under these insights brands you know the other one there was containers so you want just kind of more out-of-the-box monitoring for your communities you can head over and look at insights there and start adding in those solutions so there's lots to add your monitor we haven't even touched on on service map um last of all just in slide where we looked at this sort of overall UI from monitor I'm just gonna kind of recap on alerts the one great thing about agile monitoring is there is unified alerting we've recapped on this before so for any resource watcher conditions which will rules and what action group do you want where that's an email or or some sort of automation or other connection really great resource for that can you believe it a 12-part blog series and each blog is not short I can tell you that's over a cloud administrator dotnet I'd absolutely recommend checking that out if you want to know everything about Azure alerting really really really thorough and then just finally I noticed a few minutes over time and just going to wrap up with with dashboard because going back to to the opening slides we've talked about a lot about the observability and kind of collecting the data they were really neat to know make people aware of what's going on using that data and hopefully drive actions we've looked at the agile monitor dashboards just briefly because we'll see a nicer version of what you can achieve there the other one you're probably familiar with this graph on a-- and the third one I've gotta mention it is his squared off and I just wanted to kind of give you a quick quick idea of how these different and where you should go like with a lot of the agile monitoring features is the benefit of the the dashboards there is built in so you live in the console you you know you want to just refer to data very quickly we're having to dig around what we've just done I'd recommend building you know one of those ones dashboards or maybe those there's workbooks grifone that generally is known for its graphing ability so if you just want much higher resolution graphing and there's interactive graphing and graph honor is is a great option they do have a passion water plugin so you can point graph on or a tile monitor and get out some some basic information there so that the metrics and the logs and then squared out well we really focus on is is sharing which is sort of going like that stacked or the control old and the the goals so what we generally find with the two on the left is that they're great maybe for the developer or the the sre who knows what's going on you know inside out with that application and just really needs to monitor every single detail what they're not good at is surfacing that information to two other teams within an organization you know whether it's your support desk or your escalations team or your management team or your cloud architects for example who want to see what's going on across across the organization so that's where that comes in I'm going to just take the liberty of showing you just very briefly what square up and do one thing in particular which is very different to anything we've just seen we mention in the first few slides and haven't mentioned it since which is cost one thing as your monitor does not do is tell you about where you're spending your money which arguably is kind of one of the most sort of valuable things literally in terms of in terms of your business what we do is squared up as well as integrating with as your monitor out of the box we do integrate things like an Hydra cost management so you can very quickly see the breakdown of whether you're spending the money you're one of the largest resources you find in this the log analytics resource is going to be is going to be one of them so that's the log analytics Resource Group so that's where we're spending a lot of our money is just collecting logs so you see that very very quickly and the others the main thing we do is that roll up so you want to show the health of multiple different applications in one view maybe integrating data from your service desk and so on Kingdom then you can do that as well with drill down so this parts are limited that's the the application that we were looking at in a portal I can start to build dashboards very very fast intuitive nice easy to use dashboards and sort of have unlimited drill downs right so I can crawl down now into these different resources to troubleshoot and I can push these out to my to my other teams very easy to allow them to troubleshoot what's going on such as a little flavor of your dashboarding options and we're square that fits into all this so with that hopefully we've done a little bit of learning around as you want I know it's a lot I've gone very quickly there's many different elements to it I hope when you make you'll run into the maze some of these guidelines will give you just a little bit more chance of success so good luck with the house Rob one it's a maze if you joined late I'm just going to finish on this really worthwhile I don't want a virtual series I've given you my perspective on agile monitor if you want to hear it from the horse's mouth go go sign up for for that I'll paste this into the the chat I just want to finish the presentation here and if you want a lot more detail I give me one about kql Cousteau 101 blog I've used some about the alerts go check out this it covers every single aspect which it looked at but in much more depth thanks Fang around sorry I'm a bit late I hope you're all well and cozy at home it's got very dark here and just got a lamp on so I hope I haven't disappeared entirely but thank you very much listening there any other questions I will hang around and take a Q&A thanks a lot brilliant thanks ever so much rich great talk we've got a we've got a bunch of questions just so that I can do things in the right order and not forget I just announced the the two upcoming talks the next talk we've got next week for any web developers or and anybody interested in using a framework called blazer and how that works with us you're in signal arm real time applications and then I'm actually quite topic or given given your mention of as your data Explorer but we've managed to get the ory Barash the principal program manager of data explorer so he's going to give us a talk from xi probably probably the Wednesday of that week about some of the new up-and-coming things so if we could there's a there's a bunch of questions so maybe we can go through these quickly somebody's asked somebody's asked if if you get prompts on cost through the usual portal if you start using as your monitor no ending yeah wouldn't that be nice no I mean as far as I know there's no integration between our monitor and the cost management API so well what I'll show you squared up there was ice integrating directly with the cost management API depending on the the the subscription type you have without your so we're putting that data routes and visualizing it there are dedicated tools all around cost management a little bit sort of heavy weight but yeah wouldn't be a nicer version when I suggest told you though what you could be saving a lot of money I think I knew the answers okay so somebody else wants to know in a load balancer scenario do you recommend using the same app insights for all back-end pools I think in that case I would just because you don't want to be trying to work out where where to go oh right so that in that case you've got like that's more kind of a single logical application or service so yes I would use a single app in science for that okay so I've got a P I seem to reflect the confused heritage of the product is there a combined API coming out yeah good question not that I know of when you dig into API is where it's not just sort of the endpoints like each each endpoint has a slightly different characteristics things like selecting data you know there is only data available for seven days you ain't gonna get one API not not the other so as you can see from the portal it's built on top of a sort of fragmented API is very much I'm not sure where there's any consolidate API coming out I don't know the information to that you probably do more than me rich yeah I'd have to listen for that but no no no I'm confused as well but you know all of the API is converging quite nicely and at least they're there a single standard now I remember when when you used to have to post XML install and Jason and others so in the old days of classic so things are getting better so Babb wants to know how is squared ups cost management better than is yours cost management and billing section yeah visualizations they're the things like breaking it down by tags by the hierarchical view against the resource type or by debt tag allows you to to very easy to locate a couple of things that allows you to locate well your big spend is just visually I see it as a big blob it also what's quite nice about it in squared up as it brings the cost alongside performance data so what we like to do is point you towards things like zombie vm's right we all know them right we provision something we're playing around we've forgotten about it it's costing us clean it up so by showing you all sort of flatlining the ends using absolutely zero CPU but our costing you this much you can very easily make those savings as well and then the last piece I think is really relevant is you know we all know this as well which is you know no developer in their right mind is going to not go for the biggest possible you know resource and suicide yeah right you'll be an idiot to say no I only need like they're the smallest one so I'm fortunate what happens is you've got the dev teams generally over consuming you know over over provisioning and then you've got your centralized team trying to lock them down right and walk up to them say okay report this they'll be the potentially spending a lot so first we need to see it alongside the core state but also what you want to do is make it a little bit more collaborative and self-service so you want those individual dev teams to see their spend you know she's to see that against performance and everyone's got that same view so it's not necessarily about like what data can can't you see it's how you see it what context is now you can see it you're not going to get many developers looking at the cost management and billing sections in the azure portal I can tell you goichi Kasper wants to know what your recommendation is on reporting alerts to slack he says he's only been able to find guides on how to do it using logic ABS yeah good one I've actually literally been doing this just just earlier this week you've got the action group can post out to an HTTP endpoint typically knows a web hook writer just to post some data to some HTTP endpoint org think it off how has your automation on the sock side that's got nice it's just essentially you can have a web hook sitting there ready to receive a a message right you can just say send the message into into slack what you need to do is build a bit in between you can't typically have that web hook firing out of your monitor and it just landing directly into slack and something sensible happening right slack doesn't know about how to interpret those different different incoming query so you do need that glue in between and actually the answer is there's just like there's a hundred different ways of building that back that glue right if you're going to be an hour shore you can do it with without your automation you can do with an agile function right so your HTTP web hook is just read your function and your agile function is going to take that day reformat into a different bit of JSON and post it off to slack I actually tend to use their own quite like API was happier to know where you guys are familiar with it it's just super easy to use so that has a incoming web hook it's got a direct integration with slack so there you just say receive some data and it can show you your agile alert as it comes in and then you can construct your smack message you can post it to a particular person or Channel and so on and so that's where they've got sort of a bit more packaged up integration with slack that makes it a lot easier so I know it's just you know a particular a particular sort of been there in a particular sort of suggestion but it's a zapier is something I use and it's sort of free when you do it at a small level but you can probably get a better feeling of what you can do with that before then sort of main you always make money yourself as well okay so we've got an anonymous question someone wants to know what's your opinion on Microsoft's App Center and how that relates to shore up insights yeah I I don't I don't do so much on the security center side there is a lot of overlap so for example out of Sentinel my understanding is built on top of log analytics right so so actually the logs that are coming and you know as your Sentinel it are actually being underpinned by log analytics you can go and see them through log analytics right and you can build you know you can build on all the same kind of queries and stuff that you can do which are seen today so what you do generally find is that a lot of this some of the lines of platform technology is being shared but then she's being presented differently through the through the portal so I'm not so familiar with all the security center pieces but you certainly will find things like Sentinel are actually has a lot of lot of mythix kind of wrapped up with a different solution oh it's just they've just said App Center Oh app Center yeah sorry my bad yeah I thought it's not secure as well answered oh no I'm afraid that's a part of the maze that haven't yet explored yeah okay pair deuce sorry about that so and one other question which is what's the mo sorry that is the question that you answered what's the most relationship between as your monitor and as your security center yeah yeah yeah yeah the other one was was absent for an hour pin sites yeah no I can't I've never used an absence or so so again they'll actually history and on that insights was that it was my getting this right I have a feeling it was it was actually tied into Visual Studio online at some point but that's definitely where they're sort of come from right so they would be much more thinking about that Visual Studio developer experience so in visual Studios itself for example if you create a new asp.net web application there'll be a checkbox just to say and enable app insights right you just click that checkbox and it's going to be done done for you so that is typically the focus is those things are not for you last enter the code assume it's sort of tied in with that okay fantastic well I think we're all out of questions and we're all out of time but I really think there was a fantastic talk we've we've got this recorded now so we'll be putting this on the YouTube channel and I'll send the link around through meetup so you know if if anybody didn't catch it and hopefully they can catch it on the rerun or and what I'll do is I'll rich if you're okay I'll pass your slides and maybe your social media details so that people can get in touch with you directly if they want to talk to you about what you've spoken about tonight or squared up and yeah they'll be great fantastic well thank you very much thanks a lot guys say see definitely take care all right Cheers bye you

Channel: ukazure

Views: 7,354

Keywords:

Id: _MtrZOMQ18M

Length: 109min 38sec (6578 seconds)

Published: Wed Apr 22 2020