Hello World and Welcome to a brand new, Azure
vlog. We just hit a thousand subscribers on this
YouTube channel. I think this is amazing when I started this
YouTube channel, I wanted to combine two hobbies of mine, videography, but also my love for
Azure. And as I, see the increasing numbers of members
on the YouTube channel and also viewers on my video I can conclude that you also like
it, so thank you so much for that. This really motivates me to create more and
even better content. In this Azure vlog, I will talk about analytics
rules in Azure Sentinel, and this will be one of the longer vlogs. I will do a total story of creating an analytics
rule, defining the KQL query and all things that are required for that, and also apply
some automation. So without further ado, let's grab a, grab
a cup of coffee and I'll see you in a minute. [Music] That was a really nice cup of coffee. So let's now talk about detecting threats
because that's actually where, uh, Azure Sentinel, uh, analytics rules are made for, to detect
threats based on logs, uh, that you submit to Azure Sentinel. Uh, but how can we detect threats all, there
are a couple of methods we have, uh, out of the box detections will to help you to detect
threats and we can, uh, customly define our own, uh, logic to detect threats. Out of the box, uh, you will get alerts from
other Microsoft solutions that you connect. We have Azure Sentinel fusion, which is really
cool machine learning, uh, model that correlates between multiple alerts that are generated
to multiple sources. And we have rule templates available, which
are basically custom scheduled rules, where all the alert logic is already filled in for
you and I will discuss them in a minute. So, if you connect a, uh, other Microsoft
solution, the solution is capable of itself by generating alerts and normally these alerts
resigned only within that, uh, specific solution, but by connecting Azure Sentinel, we can configure,
uh, the connector to copy the alerts from the soft product into Azure Sentinel. Uh, so you have, uh, a holistic view of all
the alerts and, uh, things that are happening in your environment. This is really helpful if you want to go correlate
between a multiple alerts and that kind of stuff. As I mentioned earlier, we also have Azure
Sentinel fusion, which is a machine learning that is currently capable of detecting around
90, uh, multi-stage threats. Um, you need to have a lot of connectors,
uh, uh, configured for that. We need to have Azure active directory identity
protection. You need to have Microsoft cloud app security,
uh, configured, uh, Microsoft defender for endpoint is required as a defender and you
also, uh, optionally can, uh, configure a Palo Alto, uh, solution there. And it is really awesome if a, uh, alert happens
in identity protection and later you also, we see alert in defender for endpoint cloud
app security, etc. It's able to create one incident based off
those multiple, uh, alerts that are happening. So it's correlating between these alerts that
will generate one multi-stage incident for that you can enable Azure Sentinel fusion
by deploying the advanced multi-stage attack detection. You can only define one of these rules and
it will automatically do all the magic behind. It's what I call a black box rule. We can't see the, uh, the logic that's behind
the rule, uh, but it works awesome. Okay, let me quickly show you how you can
set up the Azure Sentinel fusion, uh, alert rule. If we go to analytics, you go to templated
rules, we can, uh, filter here on multi-stage and here we have advanced multi-stage attack
detection, which is based on a fusion. If you can click on this, you see a, what
connectors need to be configured in order to have this working. Uh, and the only thing you have to do in order
to bring this rule in place is click on create rule. Um, we can set up some automation, if you
would like. Click on review and create, and create. There you go your Azure Sentinel fusion rule
is now, uh, available. If you would like to disable it, uh, you can
click on disabled over here. If you would like to alter the automated response
that can be done over here, just like a, a normal, uh, scheduled rule. Next, there are also rule templates and they
are a really good point, uh, for starting. It's basically the scheduled analytic rule,
um, but with all the logic filled in for you. So the KQL query, uh, the alert, logic, all
the settings are visible for you and you can change them to your own needs. So it's a really good point to start with. If you were getting a lot of noise, you can
change the, uh, alert logic and make sure it, uh, it fits to your needs, depending on
the template rule that you deploy. Uh, certain connectors need to be set up. There are, uh, detections available, which
only requires you to set up one connector, but there are also, uh, detections available,
which correlate over multiple sources and of course, then you need to have all the required
sources configured as connectors. Let me give you a brief demo of the, uh, template
rules, which are available. If you go to analytics, its Rule templates
over here, here you have the list of all the rule templates, which are currently available,
and these are multiple pages. You'll see there are a lot of, uh, of rule
templates available. Um, what I would recommend you? Is to filter on data sources, for example,
if, uh, Azure activity is one thing that you would like to build, use cases or analytic
rules for, um, filter on it, here we have a, a rare subscription level operations in
Azure. Uh, if we can click on create rule at all,
brings us to the, uh, rule creation wizard, which is actually the same as for the, uh,
custom scheduled rules, but all the fields are filled in for you right now. Uh, you can see they have the query, uh, visible
over here. You can change it if, uh, if needed. Um, the entities are over here. Uh, the scheduling has already set up, uh,
event grouping, etc. So this is really helpful if you click on,
uh, create over here, this template rule is now converted into a, Azure Sentinel, a rule,
which you can also, uh, edit and still the, uh, the rule logic, uh, is available for you. So, um, what I recommend you is start with
a, a template rule and alter it to your needs. Uh, so it doesn't, uh, bring you any noise
or alert, fatigue altered is to, uh, to your environment. So, we also have, uh, uh, scheduled rules,
um, basically the custom analytics rules where you can define all logic by yourself. Um, so what does a scheduled analytics rule
consist of ? Of first of all, we have the detection logic. Secondly, we have the classification settings,
we can set a severity, uh, and the micro tactics, which will be visible, uh, when the, uh, detection
rule triggers a alert. We have scheduling information available,
It's a scheduled rule. So Azure Sentinel is a query based SIEM. Um, that means that within a certain interval,
we run some alert logic and based on that alert logic, we generate, uh, incidents. So as part of your analytics rule you have
to define the scheduling, uh, uh, information. We also have entity mapping, which is part
of the, uh, analytic rule. Um, you can imagine you can ingest a lot of
data into Azure Sentinel, and depending on the log source that you are using, uh, fields
can differ from name, for example, we can have a table where, with an account, a column,
a, and we can also have a table where we have a user name, a column basically means the
same. And by using entity mapping, we can map those
columns to a type of entity. We also have grouping information available. Um, if a alert triggers a lot of time, we
can group all those alerts together in one incident. And lastly, we have a automated response,
which is based on the logic apps. So, the base of an analytics rule is a Kusto
uh, query and these Kusto queries are there to filter down in the, uh, event data that's
in, Azure Sentinel and also correlate in, uh, this data between multiple events that
are happening. I get a lot of questions about the Kusto query
language. How you can start with it, where can we learn
etc. I have one lesson of here, just start by doing,
uh, just deploy a sandbox Azure Sentinel environment, configure a connector to connect it with your
Azure subscription. So the Azure activity log is available over
there, which is really easy to do and then just create some detections, just design a
scenario, and create the required Kusto query which is needed to, uh, filter and, uh, detect
all the events on which your, uh, analytic group will be based. There are a couple of sources, however, which
will, uh, help you in, um, in learning, uh, the Kusto query language. There is a really good Pluralsight course
available, it's about four hour and, uh, after you finished the course, um, you are a Kusto
hero. There's also Microsoft docs, all things that
you can type in a Kusto query, uh, you could define over there with a good explanation
and, uh, samples also available. You can also go to a medium.org and go to
my page. I, uh, now and then write blogs about, uh,
Kusto query detections. And lastly, I have a good colleague of mine
who is running the website kustoking.com, where we also discuss a lot of Kusto related
stuff. So just go and start by, uh, by doing and
make use of the sources that I've written down in the slide of here, and also, uh, have
them as a clickable link in the description of this video. So let's now talk about scheduling because
this is a really important, uh, thing and I see a lot of people making a mistake over
here, which can result in, uh, events that are not monitored, or, uh, a lot of, a lot
of noise. So let me try, uh, let me try to explain this
as clearly as possible. We can define a interval on which, uh, the
query logic is getting executed. In the example that you see over here, we
run that every five hours. And when that query gets executed, we look
back to a data set that is generated over the past five hours, as you can see, in my
example of a year, it runs at 10:00 AM, 3:00 PM, 8:00 PM, 1:00 AM 6:00 AM, and we continuously
look back to a period of five hours ago, and that is really good, depending on the detection
that you would like to do a common mistake that I see that's made to quite often is that
the query interval and, uh, the Lookback timeframe is not, uh, corresponding with the, uh, uh,
correlation between events. So, we have a query interval and, uh, a Lookback
timeframe, which corresponds with each other, but we also try to correlate events. And, um, in this case, we try to correlate
between two suspicious events, for example, a user gains admin permissions and create
an admin account within 12 hours does, or two events gaining, uh, admin permissions
and, uh, creating an admin account and it can be really suspicious, right, getting the
rights or directly creating a, an admin account. And we have a time span of 12 hours. So if you have a query interval of five hours
and you have a look back timeframe of five hours, and the time between those events can
be for example, 10 hours, uh, we will never detect this suspicious behavior. So make sure in this case that your look-back
data frame is set to 12 hours, so we can check for the correlation of these two events. So, let's now discuss Entity Mapping. Within Azure Sentinel, we can configure a
lot of data sources and they all use their own tables. Um, and those tables are actually not standardized. So we can have a table with authentication
events where a user principal name is set in a username field, and we can have a foul
access, uh, events table where all the, um, account names are set in the UPN table. So what we can do, uh, within the entity mapping,
uh, configuration of our rule is make sure that the, uh, username and the UPN always
will end up as a account, uh, entity on our alert. And we have a lot of these entities available
by default IP addresses, URLs, follow hashish accounts, Azure resources, cloud applications,
DNS, uh, hosts, and even your custom entities, uh, you can configure in Azure Sentinel. And this is really important because when
we want to investigate an alert, you want to map out the whole incident. So what's other alerts are happened over there? What accounts were being used? What host were being used? And we might see that in, in some, in some
cases that the same host or the same accounts were used, and it's really hard to create
that correlation if you have, uh, different, uh, entity types. So, as you mentioned earlier, we can group
incidents and we can configure incident creation by default for each alert, an incident is
created, and we can disable this, uh, incident creation. And we also have, uh, alert, grouping, uh,
available it's disabled by default, but I strongly recommend to, uh, enable it because
it can help you to reducing a lot of noise and even alert fatigue, alert grouping allows
us to, uh, group alerts based on, uh, their analytics rule, or we can group alerts based
on the selected entities or even all entities that were part of that alert. Let me give you an example of this for the
table below. I have configured to alert grouping to be
enabled. I have set up a grouping timeframe of one
hour and all entities that are part of an alert need to match. So if you see in the first two rows, we have
an account name over there with an IP address. And, uh, these events happened within one
hour. So Azure Sentinel will group them as a two
alerts and generate one incident out of them. In the third row, we have a account and an
IP address and a, a timeframe. And even though the account name is the same
as the two rows that, uh, that are listed before the IP address is different. And as all entities must match, um, this will
result in a new incident because the IP address is different, so not all entities match. On the fourth row. Uh, we have, uh, actually the, the same situation. We have an IP address that's being used earlier,
but a different account name. And also the time in which the event happened
is not within the same hour, the, uh, incidents that we saw before. So a new incident is getting created. On the last two rows, we have the same account
and the same IP, but the timeframe is different. The one, one occurs at 2:30 PM and the other
occurs at 8:30 PM. And the time span between those two events
it's bigger than one hour. So for those two events, a separate incident
will create it. So alert grouping gonna help you a lot by
reducing the amount of incidents that are getting created. And this is especially helpful, If we look
back in the scenario that I mentioned you earlier, where we have a query period, uh,
of for example, five hours, and we look back at a data frame of 12 hours. So we potentially have the scenario available
over there where we can generate double alerts and by using the grouping logic in Azure sentinel
we can grouped it, grouped them together. So they are becoming one incident. So, we also have automated response and I
created a video earlier, I will put the link in the description of this video, so you can
watch my video about automation rules over there. So let me give you a quick summarization of
that video. Within Azure Sentinel, we are now able to
set up automation rules and the automation rule holds the response logic, which can be
really simple by just changing severity fields or whatever fields you want to change, or
the more advanced scenarios where you run a logic app, which can do anything in your
environment. And we can connect these automation rules
to our analytic rules. So let me give you a couple of, uh, of last
tips over here before I'm going to show you how this is all working in Azure Sentinel. It all starts with a plan. You need to think of a scenario that you would
like to cover upfront. And it's really important to know where you
can find data. So you need to know what connectors need to
be connected, what data is getting through this connectors and even what events are getting
logged by the connectors really important to know. So, and also, I would like to encourage you
to not just go through Azure Sentinel and create a bunch of alerts based on what you
think is okay, because you probably will end up in, uh, a lot of alerts happening, which
could result in Alert fatigue, which is the situation where you have alerts than you can
handle. So normally there are a couple of drivers,
uh, on which you can decide to create a analytic rule. So one of the drivers can be incident driven. Incident happened last week, you investigated
that incident and you saw that a couple of events were, uh, crucial for that, uh, incident
to happen and you create an alert..alert logic around these events, when there are happening
again, uh, you will get a incident in Azure Sentinel. So we also have situation, uh, related drivers,
for example, you have crown jewels, so you want to protect them well, uh, as they are
your crown jewels, you know, what normal behavior is. And I will recommend over there to create
alert logic based on abnormal behavior that could happen over there. Lastly, uh, you have, uh, intelligence related
drivers, for example, a new exploit is available, or you saw something in the news that a lot
of attacks are happening. Um, these are also things that you can investigate
where most likely a lot of indicators of compromise are already, uh, shared through the internet. And you can create alert logic based on the,
uh, indicators of compromise that are shared. So, with that, it’s time for a demo and
I think of a scenario that I would like to, uh, demo, uh, um, an incident happened at
Demo Company. The attacker opened port 3389, which is the
port for RDP on an Azure VM. So he/she was able to use RDP, uh, changing
network security groups is only allowed by a network administrator in a maintenance window. The maintenance window of a demo company is
between 7:00 PM and 8:00 PM daily. We have a network administrator and his account
is stated over here, and we automatically want to disable the user account of the user,
which is executing the change on the network security group when the rule is triggered. So we now need to think of a plan and decide
what log sources are required. I think in this case, it's , uh, it's okay,
If we only use the Azure activity log, so we need to create the rule logic. So we want to filter out the, uh, events,
which are illegal. So we need to create a rule logic for, uh,
events that changes, uh, uh, network security groups and changes that are not being done
by the network admin or by the network admin and outside of the maintenance window. And lastly, we have to create a logic app,
which will automatically disable our user once a network security group is created. Okay, so let me now show you, uh, my workflow
of creating a new analytics rule. Before I started recording this video, I did
two things. One, I made sure that the events are already
in my log, analytics workspace. So I did go to a random VMM my MSDN subscription,
uh, changed the network security group. So the events are in there now. Secondly, I made sure that my connectors are
configured well. So for all my Azure subscriptions, uh, the
Azure activity log is, uh, getting forwarded to Azure Sentinel. So what I normally do is going to explore
the data. So if I go to Azure activity and, uh, limit
this 200, you see all events that are currently in my, uh, look, analytics workspace, and
that have been, uh, generated the last 24 hours. Um, it's, it's actually loss, but, on the
end over here, we have a column that's called created or updated security rule. And this is the operation that is part of
a changing a network security group. So if I go to the beginning and, uh, check
this time, uh, the times are in UTC. So depending on your time zone, you have to
add, or a subtract a couple of hours, but this corresponds with the moment when I, uh,
changed my, uh, network security group. Also, if you preparing your data set, like
I did be aware of the ingestion time that is coming with Azure Sentinel, I had to wait
for, I think, five or six minutes in order to have my event pop-up in a, in Azure Sentinel
so be aware of that. So I already have prepared a couple of queries
that we are going to discuss in a moment let's first, go back to the scenario because, we
are having a lot of constraints over here. So, we would like to create a, uh, analytics
rule that triggers an alert when, uh, something is happening on network security group outside
of the maintenance window. And if we are in the maintenance window only,
uh, network admin, uh, at my domain is, uh, is allowed to change network security groups. So I found out that it is quite hard to, uh,
have both these, these constraints in one analytics rules. So I figured out it's best to create two analytic
rules in order to, uh, uh, tackle this scenario. So let me show you our first scenario. And that is the scenario where a network security
group gets deployed outside of the maintenance window, whether it is a network security admin
or another user, in none of the cases that is a, uh, legal, uh, event to happen. So let's create an alert when these events
are happening. So, we have the Azure activity log, which
is the table where the events are a logged. We've filtered down, We filter down on, uh,
the events that are happened during the last eight hours, we uh, filter out, uh, the events
that are happened during the maintenance window. So between, uh, 7:00 PM and 8:00 PM, next,
we are going to filter on the operation name. In this case, we filter on the create or update
the security rule operation name. This is the operation of changing a network
security group. And lastly, I only want to filter on events
that are executed successfully, so the activity stages need to be succeeded. So if I click on this it will narrow the whole
thing down to a one event that happened, uh, and that was me generating, uh, a network
security group this morning. So this is the, uh, the first scenario. Let me now show you the second scenario where
I would like to rule out deployment of another user other than the network security admin
during maintenance window. So, what I did, I first created a list of
network administrators. In this case, it's only one administrator,
but in the future, we don't know what's going to happen. It might be that multiple users are responsible
for our network. So this list can grow. I would like to have this list outside of
my query so that I don't have to adjust the query logic. When I, uh, add a new network administrator. What I do, uh, next is getting all the events
out of the Azure activity table, filter them down to the last, uh, eight hours, so I only
have the events that are happens during the last eight hours. Then I filter, uh, only events that are only
happened during the maintenance window. So 7:00 PM till 8:00 PM, and the status of
the activity needs to be succeeded and the color should not be in network administrators. So if with an, a strange account, other than
a network admin, uh, I now create a, uh, or update a network security group inside of
the maintenance window. This, uh, alert rule will trigger. So in this case, there are no results available
as I did not, uh, do any, uh, deployment in, uh, in the maintenance time window. So that are my queries. Let's now go and set up decent analytics rules. So, if I go to analytics, I click on create
and create a new scheduled query rule, and, uh, let's give it a name, uh, Illegal Network
Security group deployment and, uh, this is the, uh, outside, uh, maintenance window that's
the name. So for description, I will leave that empty
right now, but I encourage you to give a good description over here. Uh, the tactic, um, I think it's persistent
because we can open a port so we can get in a later, if we are an attacker and the severity
medium is okay for me, the status can can be set to enabled. So let's drop in the KQL query over here,
and now we have to configure our entities or a couple of entities, which I think are
interesting. That is the IP address and the address and
that's called the Caller IP address, uh, in the, uh, in the, uh, Azure Sentinel table.. In the Azure activity table. Another, uh, thing that is the account, which
I saw this is also required when we are going to do a response on it. So, for the account, we also have, uh, the
name and that is the, uh, the caller in this case. So as you see, uh, account corresponds with
the column caller over here, next we have the, uh, the scheduling in this case it's
okay to run the query every eight hours, uh, lookup for a data, uh, set of, uh, of eight
hours as you saw in my query, I started with where time generated is smaller then eight
hours. And that basically means that we pick all
the events that are happen at eight hour from triggering this analytics room. So if we have more than one, if you have more
than zero results, uh, this analytics will trigger an alert. So here we have the incident settings, I would
like to create a incident when this alert is triggered. So we leave this on enabled, and I also would
like to, uh, enable alert grouping. Uh, for me, it's very helpful, uh, to have
just one single incident for attacker, uh, instead of just, uh, a bunch of incidents
for each network security group that he or she, uh, changed. So I will enable alert grouping, um, when
all entities match between my, my alerts. So next up is our automated response. I will leave this empty right now. We first have to define and create our logic
app that is responsible for the automated response before we can configure it over here. So I'll leave that empty. Next up we have the, uh, the review, and I
will create my rule right now. So as you can see, it's created over here. So, in the meantime, I have created my second
analytics rule, the rule that is responsible for, uh, checking the events inside of the
maintenance window. If I click on this, uh, you'll see the query
over here, this corresponds with the query that I showed you earlier. And the settings are the same as for the,
uh, the outside maintenance window analytics rule. So we have the analytic rules defined now,
uh, but we still have, have to do something with our automated response. So if I go over to my home, uh, page over
here, I have created a logic app. This logic app holds the logic to, to execute
the response when this, uh, types of incidents are happening. So let me make this a little bit bigger for
you. So what I did first is to set up the trigger. This logic app will trigger when an incident
is created in Azure Sentinel. Secondly, I collected all the account entities
from this incident For each of these entities, for each of these
accounts, sorry, I collected the user. I collected the user first because I had to
set up a couple of variables over here in order to update the user. What I did is, uh, at the setting for account
enabled, and I will set it to No, uh, when this, uh, when this response is getting executed. So I think this is basically the response
that was stated in the, uh, scenario earlier. We want to disable the account when a, uh,
illegal network security group changes made. Um, and that's what this, uh, logic app is
doing for us. So let me now to go back to Azure Sentinel,
uh, and we are now going to set up a automation rule, uh, that will hold, uh, this logic and
is bound to our, uh, analytics rules. So I will call this disable accounts so I
can reuse this automation rule, uh, for other incidents. I will select my illegal network security
group deployment, uh, analytic rules. And the actions that I want to, uh, do is
run a playbook. And the playbook that I would like to use
is the Auto response it's currently not, uh, selectable and there is stated no Azure Sentinel
permissions. What I can do over here is managed playbook
permissions and navigate to my, uh, research group where the logic app stated. It's the DEMO- SENTINEL, uh, resource group. Click on apply, just wait for a moment and
now we are able to select this one. We will not have a expiration date set up
on this rule, and it, uh, it is ordered as the, the, fourth rule to, uh, apply and for
me that's okay. So it's apply. We have the disabled account, uh, ultimation
rule over here. And if I now go to my analytics and we open,
uh, these rules and go to the, uh, automated response panel over here, you say that when
an incident is created, now automatically, uh, run the disable accounts, playbook, and
that's done because we have set up our, uh, automation rule to work with this uh, analytics
rule. So, as you can see, I'm currently locked in
with my demo account. So it's a safe to have that one, uh, disabled. If I go to my demo Logstash machine, go to
networking, I can change a network security group, and it's currently outside of the maintenance
window. And this user is not allowed to, uh, to change,
uh, any network security group. So, um, if everything went well, this usual
will gets disabled. So if I click on, add a, uh, inbound, uh,
security rule, uh, let's change this one to port 8081. So now port 8081 is also open for this virtual
machine. Um, as soon as this events, uh, gets into
Azure Sentinel, and my alert rule is triggered. Um, this will, uh, uh, disabled as user. So, and here we are back again, it's a couple
of hours later. I had to wait in order to, uh, have the data
ingested in Azure Sentinel and wait for the moment that my, uh, alert rule got triggered. Uh, but here we are, and let's have a look. Here we have the entities, the demo user,
which I, uh, used, uh, the alert, uh, on which the incident is, uh, is based. Um, and what we can do is go over here to,
uh, my logic app, and you'll see that it, uh, succeeded to run. In order to make this working, um, I had to
make a small change. We first use the Azure ide connector over
here, but I didn't got that working. So I changed it for the, uh, graph API. This is where we directly work with the, uh,
the graph API and disabled the account manually on HTTP request instead of using the activity
that's provided there by Microsoft. So if we now go over to the, a user, I have
the demo user over here, you can see that block sign in is now set to yes. So the user cannot log in anymore. So he will not be able to make any mess of
our network security groups anymore. So, in this Azure vlog you learnt about analytics
rule. We first had look in to the out of box solutions
that are available in Azure Sentinel, mainly the Azure Sentinel fusion analytics rule. This is an analytics rule which is powered
by machine learning and helps you correlating events that are happening in multiple alert
sources. The alert logic that is behind this rule is
machine learning and it’s not feasible for you. And we also had to look at template rules
whereas in the Azure Sentinel fusion rules the logic is hidden away from you, you can’t
see the logic that is available for a rule template. You can see the KQL query which is being used,
you can see all the classification settings etc. You can even change them to fit to your own
environment. Next we had to look at custom scheduled rules. These are rules where you can define everything
yourselves. We created our own rules. You saw how these rule were triggered and
how the logic app automatically disabled the accounts. I think this is really important when you
see attacks are happening more often and they are getting more complex it requires us to
automate a lot of incident response. Automation rules in combination with our analytic
rules helps us a lot with that. So, with that, I would like to end this Azure
vlog. I hope you liked this content. I hope you learned something if so, please
hit the thumbs up button. If you still have questions please use the
comment function of Youtube , I will answer them as soon as possible. Also subscribe to this channel and of course
I will see you in the next one. Bye. [Music]
