>> Did you know that you
can deploy Azure SQL anywhere with Azure
Arc-enabled data services? Learn more from Ben in this episode
of Data Exposed, MVP Edition. [MUSIC]. >> Hi, I'm Anna Hoffman. Welcome to this episode of
Data Exposed, MVP Edition. Today, I'm joined by Ben. Ben, thanks for joining
us today again. Can you start by telling us
a little bit about yourself? >> Thank you for having me again. I wasn't sure if I was ever
going to be invited again. But then again, that was my
best behavior last time. MVP from Nuremberg, Germany, mostly in the data
warehousing and BI field. Last time we spoke about
big data clusters. Well, now I guess I'm
here for something new. >> Yeah, we're excited to
be learning something new. It's this ever-evolving
thing and I'm excited to learn what you have to share
with us today about Azure Arc. Without too much preamble, I thought I'd just ask you, can you tell our viewers, and myself, what is Azure Arc and
how does it relate to things we know like
Azure SQL and SQL Server? >> Azure Arc is this real cool thing. It is bring your own datacenter, but still enjoy the luxury of Azure services in general,
I should rather say so. Azure Arc is not just
for the data world, but we're going to be focusing
mainly on the data world. The idea of Azure Arc
is you go to any Cloud with any infrastructure
and any Kubernetes. This can be Azure Kubernetes service, you can run your own Kubernetes. I'm basically running my own
Arc datacenter in my basement, which we're going to
see in that emulator. You can run it in any
of the other public or private Cloud, AWS, Google Cloud, you name them, which means wherever
you go, it doesn't matter. It can be on-premise, it can
be private, can be public. It can be connected and disconnected. We're also going to see
that in the other slide. There's data services, there's
our Kubernetes services. There's all kinds of Azure services that are
moving to other clouds. Again, on-premise is a
Cloud in that context. The idea for the data world here
is we get managed instances and PostgreSQL Hyperscale into
your own private estate. still having all the
luxuries that you would get from the classic Azure
SQL service world. Classic, I mean, they're
all brand new and still you get to see
them in the portal, you get to manage them still on-prem, you get manage backup, you get manage updates, all that. But all that without having to
be locked into the Azure Cloud, which is a big deal for
many people that either run multi-Cloud strategies
or that just say, "Well, I cannot move
away from on-prem with some of my data for
performance rate." You may just be sitting on
an oil rig and just not have the connectivity to everything to run into the Azure Cloud non-stop. What does that mean for the
Azure Arc-enabled data services? You start by just deploying
your Kubernetes cluster. I keep that rather gentle
because like I said, this can be Kubeadm, AKS, EKS, all kinds of flavors. On top of that, you deploy a
so-called Azure Arc Data Controller. This is like your main endpoint. If we remember big data clusters, we had the big data
cluster controller. This is the counterpart to
the Azure Arc if you want. One single point of entry, and that's the only thing that
you really have to deploy. This will take care of everything
else that's going on there. This is happening again
through the Azure Data CLI, through AC data, which we are
also using for big data clusters. Azure Arc or Azure
Arc-enabled data services, I should rather say, that is
the answer to that question, the end of the path where
people are wondering, "Well, SQL and Linux, SQL and Kubernetes, I see what it's doing. Why?" This is why, because now we get all
this in those containers, in those orchestrated containers. Just like with big
data clusters, again, we can either use AC data directly or we can use
Azure Data Studio, a tool that more and more people are using for exactly those
purposes: managing, deploying, and developing
their data solutions. We got it connected
to the Azure portal. Again, same logic. We will also be deploying
our data instances, again, like I said, Azure
SQL Managed Instance or Azure Database for PostgreSQL. Those are the two
offerings we would get in Arc-enabled data
services right now. By the way, Arc-enabled data
services are in public preview, so we can still get monthly releases with more and more
interesting features coming. Since when Azure Data Studio can even automate all that through
Git merges if you want, you can have automated, built and deployment pipelines
for your data estate; and you get the same Kibana
and Grafana interfaces that you already know from SQL on
Kubernetes and big data clusters; and we got the link
back to the portal. Like I said, depending
on where you are, you may have a direct
connection to the Internet, to the Azure Cloud or not. That's why there's basically
two ways of getting those metrics and log
files into the portal. You can do the direct mode, which means your data controller
will constantly upload your consumption data and log files for analysis
to the Azure portal; or you can use the indirect mode. Indirect mode basically means that you're running
a script every hour, every minute, every
week, every month, that will just push that
data to the portal so your resources will show up there
and are available for analysis. Let's take a quick look at
that in Azure Data Studio. This is just PowerShell
what I'm doing here, but I decided just to run
it in an ADS notebook, so we've got it all in one place. I already got my controller
here in my basement deployed. What I can do is I can see, well, I got already three
managed instances and one on PostgreSQL instance
deployed there. I can manage all that right here in Azure Data Studio but I can also just run a PowerShell
command where I can say, "Show me all those instances." Again, I can do that
through code or through the user interface
depending on what I prefer and I will see my three
managed instances showing up including
their external endpoints. Well, external from the
Kubernetes perspective, those are private IP addresses
because I'm running that in my own private basement datacenter. If I want to create a
new Managed Instance, all I would need to do is
run this command here, azdata arc sql mi create. If I would do this and
just do go-titans 01. I think Bob What he's a big fan of the Titans so I'm
doing this especially for him. >> Huge fan. >> Well, him and Buck, of course, because Buck is the biggest
sports fan that is out there. Again, if I would be preferring something with a graphical interface, I could have gone here
and say, "New Instance." Basically doing the same
thing so this would, exactly the user agreement. I could say, "Well, this will be my go-titans too," and I could either just script it to a
notebook that I could run later or deploy
it right from here. We're sticking to just running
it right here in PowerShell. What this will do is, in
my Kubernetes cluster, it will create another
part, go-titans-01 - zero. That is, by the way,
super-interesting in Azure arc. This will also be the
platform which allows me to run which is why it's
saying, "Dash zero." There could be a dash one, dash
two if I had multiple instances. This would be the way to bring SQL
Server Availability Groups into Kubernetes which is
currently not supported by the regular SQL Server,
quote, unquote. This is running, this is creating my Instance and if I would go
back here and refresh this, we would also see this
is being created. We see this Instant as Creating and once you've got your
first Instance created, all the consecutive one usually take about a minute to a
minute and a half. There we go, go-titans-01 is
ready and if I do a refresh here, I would see the Instance
just as well if I do the azdata arc sql mi list here, it would show up the Instance
including its endpoint and including its IP address and port so I
could connect to it. Pretty cool. >> Nice. This is pretty cool. It's a good bit faster than
installing SQL Server. >> Slightly. For those
trying this at home, the first time will take a
little bit longer because it actually needs to
download SQL and that's again the beauty of Kubernetes
once it's got all these images downloaded to its local
library if you want so. Ninety seconds for a SQL server, that's not just a regular
SQL Server because if we just go here and do a select, add, add version on one
of my Managed Instances, what you will see is this is not
just SQL Server or SQL Server on Linux but what I have here
running on my local premises, I got my Microsoft Azure SQL
Database Managed Instance Azure Arc. This is a Managed Instance. This would get automated updates, this would get automated backups, all that, if configured for
that specific instance. I could get all the local information that or in my Grafana dashboard. If I go here, where of course I
did not deploy a certificate, this would get me my
SQL Server metrics. I could go to all the
different instances that I got deployed here. How many transactions, how many
batch requests I could see, what's actually happening on my
host but I can also get that back to the Azure portal and this is where it gets super
cool if you ask me. Because that means in the end, no matter where my
databases are running, no matter where my
instances are running, I get them all in one single place. The first thing I'm doing is I'm
exporting the LogFiles here. This is something that
you should schedule in the real world so have that run every two
minutes, every two hours. Unless you're in direct
mode where like I said, this would be happening
automatically. What this will do is it will collect all the LogFiles and metrics
from all these instances. You can see, it already picked up my go-titan so it's not like
I have to configure anything, it will just do it for every single
Instance that I've deployed. Then the next step, I will just
upload those to the portal again using azdata so what this will do is will take the
files that I just created, the two JSON files. Push them into my Log Analytics
workspace in Azure and if I go into my Azure portal
and do a refresh here, you will see there's my local-mi-01 where I could now see the metrics for that specific instance right
here in the Azure portal. What happened to my CPU on
that On-premise SQL Instance? I see it in the portal. If I go to the overview,
I would also see, well, it's running in Ben's basement. This is my endpoint so this is where it's all
coming together and this is the beauty of Azure
Arc-enabled Data Services or Azure Arc in general. >> That's pretty cool
because that means wherever your SQL is running, if you have it connected
to the Azure portal, you're going to be able to manage and monitor them all from one place. Am I understanding that correctly? >> Correct. Without the worry of running your own
updates and all that. All the beauty that you
get from managed services on your own infrastructure or
somebody else's infrastructure. It's like I said, it's bring your
own data center if you want so. >> Love it. >> I think that is amazing
because it really gives you that flexibility no matter where
that restriction is coming from. >> Awesome, cool. Well, thanks, Ben. This has been really interesting
and for our viewers, you've actually been pretty busy with Azure Arc-enabled data services. I think you might have something you might want to announce or
share with our viewers. >> Indeed, my very good friend
Anthony Enosantino and myself, we wrote a quick book on Azure Arc-enabled
data services because I'm known for writing book on technology where you will know for sure that the
technology will change five-times until the
final product comes out. This is based on the
current public preview but it was big fun writing that. Trevis, the PM for Arc-Enabled data service
actually was nice enough to write us a great
forward for that book. If you want to find out more about Azure Arc-enabled data
services, how to deploy them, also more about the
technical background, make sure to check out our book, I hope it's worth it and
let us know what you think. >> Awesome. Thanks, Ben, and
I think for our viewers, it's a great call to action. You want to learn more about
Azure Arc-enabled data services, then definitely check out
Ben and Anthony's book. We'll put some links
in the description. Ben, again, thanks so much
for joining me today, and to our viewers,
thanks for tuning in. We hope to see you next
time on Data Exposed. [MUSIC]
