Hey guys, how's it going? Hope everybody's doing well out there
today. I've been using Proxmox for about a
year, give or take, plus or minus, whatever. And I really like it. There's a lot of things about it that I
really, really like. Spending up VMs and containers and the
backup strategy and the whole bit is just super, super
easy to do without much thought. There is one thing about it though that I've found that I don't necessarily
like and that is their reporting. If we jump over to my desktop here, we can take a look, this is my
production server. So if you go to any of my different
online assets that I've given access to, you're hitting this specific piece of
hardware. The problem that I have with their
reporting is that it happens once per minute. Like right now we're at 10:05 on
December 6th, and the next one back is 10:04, and 10:03 and 10:02, and so on. So when I'm trying to figure out what's
going on in real time, this isn't super helpful. And I decided to try something that I've
done in the past sort of, and that is, I installed a NetData on my Proxmox
server. Now we took a look at netdata almost
four years ago now, but I wanted to give it another look and see what it would be like to install
it on a Proxmox setup, kind of at the root of this setup, so that we can keep tabs on what's going
on across the entire server in real time. Now look, I know some of you are going
to ask, does this work with like a high
availability cluster? And the answer to that is I don't know. I don't have an HA cluster set up
anymore, so I haven't had the opportunity to test
that, but if you've done this and and know
whether or not it works on a high availability cluster, please let us know in the comment
section down below. So if we come back over to my desktop
here, we can see this is netdata running on
my production server. We can see Prox Proxmox production. I've named this the same, or it shows up
the same, so that's super easy to tell them apart. You know, we can scroll through, and of course, if you're familiar with
netdata, you'll know that you'll get a lot of
really good information about what's going on on your server. And what I really kind of dig about this, we come over here to the right hand side
of the page, and look for containers and VMs down
here on the right side. Here we can see we've got maximum CPU
utilization within limit and maximum CPU utilization and maximum memory utilization within
the limit and total RAM used. And here we can see kind of all of that
broken out by individual containers or VMs or
things like that. So this gives us more of a real time of view of what's going on. Now, when I say that, it's every two seconds versus every
minute, at least by default, and I find that to be far more
informative than getting just a one minute increment
update. I like seeing things far more regularly like it shows here on netdata. So what I want to do in this video is not only show you how to get netdata
installed very, very quickly and easily, but I also want to show you how you can
get notifications if the system notices any kind of
anomalies or anything like that. Maybe you've got a disk that is too full. Maybe you've got excessive RAM usage or
things like that. You can get notifications from netdata about your Proxmox setup. So we'll take a look at how to do that
as well. For the demonstration here, I'm going to show how to do that with
Discord, but there are lots of other options that we'll scroll through and take a
look at very briefly. So let's jump back over to my desktop
and take a look at how to get netdata set up to monitor
your Proxmox host. Okay, so here we are back on my desktop. And because I've already got this
installed on my production server, I'm actually going to just close those
first two tabs so we don't have to worry about getting
confused about anything because that happens to
me a lot. What I'm going to do though is come over
here to where we've got like my data center and then I've got my node under my data
center and I want to click shell right there. And here we are, we're logged into our
shell and now we can start running our
commands, which thankfully it's really only just
one command. And I will have links to all of this in
the video description if you'd like more information. So here is our command, right? Basically it's going to be, it's going
to run a bash of this script after it downloads it
over a secure connection. It's kind of in a weird order, but this is a simple command. They could have done a curl and then a
bash, but they just compacted it all into one
very brief command there. So what we're going to do at this point
is you take enter and now it's going to pull all the
resources it needs and get netdata installed for us. Now here at the bottom of the screen, hopefully you can see that it says, "Hey, this is going to take up an
additional 157 megabytes of additional disk space." Yours may be different than that. In fact, it will probably be different
than that, but that's what it's going to take for
me. So I'm just going to say, "Yes." Oops, I'm going to say, "Yes." Oops, come on, there we go. And I'm going to hit enter and now it's
going to go ahead and do everything it needs to do to get
this installed. And literally as I was saying that, it
finished up. So very, very quick there. So what we can do now is just kind of
grab our IP address here. We don't want to grab the HTTPS. This does not run on HTTPS by default, so we're not going to use that. But we're just going to go ahead and
paste that in. There are type of 1999 like that. And then we're going to go ahead and
say, "Continue." And we're going to give it just a second and there it is. Just that quickly, now we've got netdata up and running on our Proxmox. Server this again is PVE demo. It automatically named it. I thought that was the case, but I
couldn't remember. And here we can see our disk rights, our
total disk reads, our CPU, our RAM, all of this kind of
stuff. And again, if we come over here to the
bottom right-hand side and click on containers and VMs, there
we go. So again, here we can see all of the
different containers or VMs, whatever the case may be, on
your setup. You'll see both here. Well, we can see the value percentages, the actual value and megabytes. So it's just a quick, easy way to get
more information about your individual containers and VMs and that sort of thing. And I really, really dig just how easy
that was. Now we could easily spend a very, very
long time going through all of the different bits
and pieces and that sort of thing about netdata. But that would take an entire video or an entire video series in and of
itself to explain everything. Also, as I mentioned, about three, three
and a half, four years ago, I already made a video
about netdata. And there's a lot of really good
resources here on YouTube and across the internet about what
different things about netdata do and how to utilize them and that sort of thing. So I don't want to bog this video down with all of that information that may
not be relevant to the people watching this video. Again, if you'd like more information, there's lots of it out there. I'll try to link to some resources in
the description. But I wanted to show how quickly and
easily we could get netdata installed on our
Proxmox server to actually monitor individual bits of
our server to get more real time, close to real
time information about it. So we've done that and that's great. But what I want to show next is actually how to get some notifications set up. So let's jump back over to my desktop and take a look at getting notifications
set up and netdata for Proxmox. So here we are back on my desktop. What I want to do is actually come over to the actual Proxmox instance and come back over to the shell here. I'm going to go ahead and clear my
screen with, oops, with Control-L. If I can remember how to do that. Now we've got a clear screen going to be
much easier and I'll of course zoom in on this. So the first thing we need to do is
actually do a CD into the /etc/netdata folder. We're gonna hit Enter. And then we're going to modify a file
here. In fact, if we do an ls. just for the
giggles of it, here we can see the files that are in
there. The one we're looking for is the
health_alarm_notify.conf file, which is this one right there. That's the file that we're going to edit. I'm gonna use nano to do that. And I'm gonna do a health_alarm_notify
like so. Now, the reality is this may already be
pre-populated with my information, but we'll see what
happens. So what we're gonna do, there's a lot of
information here that you can modify. You can change the image-based URL and
the export URL and the date format. Whether or not you're using a fully
qualified domain name, you know, are you using external commands like send mail or curl or like there's
just a lot to scroll through here. Now we're starting to get into
notification requests though. This one is for email, global
notification options. And we're not gonna do that. We're gonna keep scrolling. Here is Dynatrace settings and then
we've got, well, that's more Dynatrace. We've got a go-to-fi, or I guess we've
got a go-to-fi. I've never actually thought about how
that's pronounced. We've got OPS or opsGenie, I guess,
send pushover. We've got pushover options there. Push bullet notifications, Twilio
notifications, messagebird notifications. A Kavenegar, Kavenegar. I'm not sure. I've never actually paid attention to
that. We've got telegram, we've got Slack, we've got Microsoft Teams, we've got
Rocket Chat, we've got Alerta, if you would use that. Flock and then here is Discord. Here we go. So what we're gonna do is actually
modify this file just a little bit, just add, modify the three lines that are there,
add a few more. And again, all of this will be linked in
the video description. If you wanna jump over and grab the stuff that I've already done here, so you don't have to type everything out
manually 'cause that would be really, really
tedious. Luckily, I managed to copy and paste it
from somewhere else and I'm just gonna make it available to
you. So what we're gonna do is we're gonna
come up here to where it says send Discord. And of course, we're gonna change that
to yes, just like so. Our next option here is for Discord web
hooks. So we need a Discord web hook URL in
order for our netdata, for Proxmox to communicate with Discord to send us the notifications that we
want to get. So next we need to go over to Discord to
grab that URL. Okay, so this is my Discord server and I've got a testing war room where I
do all of my bot testing and things like that. And this is where I'm going to put it, of course you'll wanna put it wherever
you have the authority to build in web hooks and that sort of
thing. But what I'm gonna do is I'm going to
right click my testing war zone and I'm gonna click
edit. Then in here I'm gonna go to integrations and then I've got web hooks right here and I'm going to click there. And then I'm going to say new web hook. And I'm gonna, this is the December 6th, that looks right. So what I'm gonna do is I'm gonna call
this netdata touch and it's gonna be in the testing war
zone. I'm gonna copy the web hook URL and
click save. So now I can come back over here and I
just paste that in, making sure that the web hook URL lands between those two sets of quotations. Next year, this I believe was already
filled in earlier by default. But what I need to do is actually change
this to the name of the discord channel. And I wanna make sure I get this right. I'm gonna come back, it's gonna be
testing war zone is going to be the name of that. And it's gonna be all lowercase with a
dash in between. So I'm gonna do this just like so. And at this point I can save with
control O and enter just to make sure if anything goes wrong, I don't lose my settings. Now there are some additional things
that we need to do, that we need to add to this as far as
the role recipients to make sure that we get all of the
different notifications that we want to get. So basically what I'm gonna do is I'm
just gonna drop down to a new line and I'm gonna paste all of
this in. And this is role recipients, Sys Admin,
Domain Admin, DBA Webmaster, Proxyass Admin, and Sites
Manager. And here are all of the different
notifications that they would get. So basically any time one of the
notifications for one of those things pops up, we should get a notification for that in
our discord server. So again, what I wanna do is do a
control O and an enter. And then a control X to save and exit. That's all we did there, control O and
enter and control X to save and exit. Simple enough. The next thing we wanna do is actually
make sure that we're gonna get our notifications. So what I'm gonna do, so I'm actually
going to do these side by side here for just a moment. And I've got a command, again,
everything will be linked in the video description. So what I'm gonna do is run this command
right here, which is gonna be
userlibexecnetdataplegons.d, alarm notifySH test. And I'm gonna hit enter. And there we go. There are the notifications that it said
that it sent. And right over here are the notifications that we just received at 11.06 AM. So when I first set this up yesterday or
the day before, or whenever it was, I honestly, I
thought, well yeah, the test works, but do the
actual notifications work? Are we going to get notifications for
these kinds of things when something actually happens? And it turns out that I did actually get
some notifications about an issue that I was having when I was trying to deploy a container. Okay, so here we are. We're back on my desktop taking a look at my testing Warzone area. And here we can see the on PVE demo. This was at 9.37 yesterday morning. This was basically the same test alarm that we just demonstrated a moment ago. Once I was kind of, I still was kind of
skeptical as to whether or not I would get the
notifications. And just as happenstance happens to be, I started to deploy a container over on
my PVE, or my demo server. And it hung up and it was taking forever. And I wasn't really sure what was going
on. But then I noticed I had a message in
Discord. So I came over to my Discord channel here and saw that PVE demo needs attention, disk-based storage, sub-volume 107. So basically, container or VM number 107 says that the storage space at 9.4% or 9.4% and then 9.4% which is reversing those numbers. And then 99.7%. And then I realized, oh, these
notifications are actually working. That's amazing. And let me know that my storage for that
particular container or VM, I don't know which one it was,
has maxed out. And that's why my installation got hung
up. So once I realized that that was the
issue, I was able to go back over to Proxmox, give it some more storage space and
redeploy. And then, just within a couple of
minutes, I think. So this was at 10.39 was the last
notification right here. And at 10.42, it actually, and it says
10.57, those numbers don't quite line up. But either way, the case is that it says
that it recovered and that the disk space is fine now. And it was a alarm raise for six minutes. And again, down here, same thing. But I got notifications letting me know that the issue had been resolved. So I love that you'll actually get a
notification saying, hey, there's an issue. And then when you fix it, I was like,
hey, there's no more issue. So I appreciate getting a notification on both sides of the issue, letting me
know that there is no longer an issue. I really, really dig really big that if I could get my words out. So that's how easy it is to get netdata
installed on your system so that you can more
closely monitor and get notifications about your Proxmox
setup. So that's all well and good. But let's say for some reason, you want to uninstall netdata from your
system. You don't want to use it anymore for
whatever reason. Now I want to show you how you can
quickly and easily remove netdata from your Proxmox instance with just a simple command. So we're back here on my desktop. Again, I'm going to clear my screen just so that it's easier to find out
what's going on here. So here is the command. And it is just curling that same
installation command that we ran earlier. But it's going to put it in a temp
folder. And then we're going to run that command
with the uninstall flag. And just like that, this is, are you
sure you want to remove? I'm going to say yes and hit enter. And we're going to give it just a moment
here. Do you want to remove the netdata repo
edge? Yeah, probably. And there it is. The bottom of the screen saying
everything is done. And now if I come over to here, all of a
sudden, we're not getting data anymore. And if I refresh this page, it's failed
out, which tells us that our netdata installation
has been removed. OK, guys, there you go. There is how to install netdata on your
Proxmox server. So you can get some more data about your
setup and get notifications if the system
notices anything going wrong. Hopefully you enjoyed the video. If you did, it would mean a lot to me. If you give the video a thumbs up. And maybe even let me a comment down
below, letting me know what your thoughts are
on this setup. Is this something you've tried? Is this something you're going to now
try? Let me know. Let me know all of that down there, of
course, in the comment section down below. As I've mentioned a few times throughout
this video, all of the resources for this will be
linked in the video description. So be sure to check that out. But I think with all of that said, I'm
going to wrap this up. I do wanna thank you guys for spending a
few minutes of your day with me today. And I'll talk to you in the next video.
