High Availability Pi-Hole? Yes please!

hey is the internet down what looks up to me hey is something up with the internet uh let me check [Music] nope what's up hey is the internet down huh oh let me check well technically the internet is up it's just that dns is down but you just can't resolve any domain names what what what what um what oh yeah i rebooted by hole hey welcome back so i'm techno tim and today we're going to talk about running two pi hole servers and keeping them in sync and real quick before we get started if you run into any problems along the way check out my live stream i stream on twitch every tuesday thursday and saturday i spend a lot of time answering your questions so if you need some help along the way come by and say hello and another thing before we get started thanks ahead of time for the likes and comments it lets me know if i'm on track so let's get into it so you know what pihole is it's a self-hosted network-wide ad blocking service it helps block ads and malware sites for your whole entire network and you might be running a bare metal say on a raspberry pi or virtualized in a virtual machine or containerized using docker or kubernetes and regardless of how you have it installed we all run into the same problem it's that when pi hole is down no devices on your network can resolve dns names and while you can run two instances of pi hole it's been a challenge to keep those in sync you see i use pi hole for more than just ad blocking i also use it for local dns and with the latest edition of cnames i'll start using that too so now the challenge for me is keeping all of that in sync i've been looking for a way to keep my local dns entries my block list and my allow list in sync across two pi hole instances and while pi hold doesn't officially support this yet there's an open source project that's made this possible that project is called gravity sync gravity sync will synchronize your pi hole databases across two pi hole instances and while there are some limitations it synchronizes the most critical things for me to keep my network running so the things that will sync are my block lists my domain allow and blocks any of the custom regex block and allow lists i have and my local dns settings and soon it'll synchronize cnames so there are a few things it will not sync so it won't synchronize local network configuration it won't change or use the admin password or account in pi hole it won't change any of the upstream dns servers it won't work with dhcp and it won't sync any of my stats but this works out fine for me because i don't use dhcp in pihole in the stats while it would be nice to synchronize these i'm okay with the stats of my primary biohole server and i think this is a fantastic gap solution until pie hole if they ever do allows us to create a highly available piehole cluster but till then this is as good as it gets so let's hop right in the first thing you're going to need is a pie hole server now you should already have one set up and configured so i won't go into how to set that up but after that you'll actually need to set up a second pi hole server now your second pi hole server can be installed on any of the ways i mentioned earlier it can be bare metal on a raspberry pi it can be virtualized in a full virtual machine or it can be containerized and choose your way of containerizing it all three of these will work but there's one caveat if you containerize it and we'll get into that in a little bit but you'll want to get pie hole installed on your second machine and after the install is complete we'll go out to the web ui and once you're here we'll log in and configure a few things now the way that i have mine configured is that i match my secondary server with my primary server so let's take care of that real quick first we go into settings and after going into settings i'll pull up my primary pi hole server on the left and have my secondary pi hole server on the right one thing i recommend doing is backing up your primary pi hole server that's as simple as going into teleporter and clicking backup but we'll configure this in a way that we don't affect the primary server at all but we'll get into that here in a few then i'll copy the settings from my first pi hole server over to my second pi hole server next in dhcp you'll want to make sure that this is disabled and then you'll want to check the rest of the settings to make sure that they match the next thing we'll want to do is verify that we have all of the required packages on both servers and most of these packages are required for piehole anyway but you'll want to be sure that you have these requirements fulfilled before we go any further and that's as simple as just installing these packages the next thing we'll need to do is make sure that we have a password list sudo on both servers and this is so rsync can work properly now at least in ubuntu it requires a password when you use sudo but we're going to remove that requirement so we can synchronize some files in rsync because when we run an rsync we don't have an easy way to provide a password so on my secondary pi hole server the one i just created i'm going to enter this command here it's sudo editor equals nano vi sudo and we'll go down to this line right here we'll comment that out and we'll add a new line now this is basically saying that any of our sudoers don't have to provide a sudo password when they run a sudo command and on my pi hole server i only have one user so i'm going to apply this setting but feel free to change this to suit your needs so we'll want to save that and i found that i needed to reboot my server after i apply that setting then you'll want to do that same thing to your primary pi hole server after applying that setting save and reboot okay so i promise that's the only thing we're going to change on our primary pi hole server after this we're going to leave it untouched then we'll want to go to our primary pi hole server and we'll want to run this command this command is going to do some checking to see if we can synchronize the two machines and it goes through and validates authorization then it makes sure we have the required components then it does a little bit of diagnostics runs a status report and then a few other things but it exits without making any changes and this is just making sure that our system is ready to be synchronized then we'll go to our new pi hole server and we'll run this command this command is going to do some of the same things but it's actually going to configure our machine to synchronize with our primary and so it has a wizard to walk through a few things and i'll go through the advanced options so that we can see them so first it asks whether or not this one's in a docker container and then whether or not the remote ones in a docker container the default path to pi hole the ssh port that rsync is going to use whether or not we want to ping and check our ssh key location whether or not we want to replicate local dns our backup retention and then we'll supply the primary ip address of our pi hole server and now it's going to test it's going to ask for a ssh user prompt whether or not we want to connect and then prompt for our password and then it's configured so now if we go into our new pi hole server our secondary one we can go and check for some of our block lists and you might notice that nothing's here well there's a few more things we need to configure the first thing that i do is just compare so before i write anything i just want to compare whether or not there are any differences between my piehole server and we do that by running the script with compare and you can see that there's a difference in my gravity database my custom lists and that replication is required so there's a couple things we can do here we can actually sync this both ways or we can pull from the primary to the secondary and on this initial sync you should probably just pull because our primary pi hole server has all of the changes that we need and we know that our new pi hole server doesn't have any so let's pull the changes from the first one to the second one and the way that we do that is run that same script but add poll and as you can see it pulled it down so let's refresh our new pi hole and here we go we can see some of our blocked items we can also see that our allow list synchronized and if we go into our dns records we can see our local dns synchronized as well so that worked out pretty good but let's test it again so let's add a new domain to our block list let's say example.com we don't like example.com they have bad examples and so let's add it to our block list so we see here on our primary pie hole it's on our block list let's run this pull again and if we go to our secondary dns and go into our block list we can see example.com is here with our comment and we can do the same with our allow list let's say example.org they have good examples we add it to our allow list go back do a poll again go into our allow list and filter on it and we can see we have example.org along with our comment so this is working really great but we want to automate this we want this poll to run on a schedule and so we'll run it on a task and so the way that we automate this is by running the same script with an automate flag when we run this script with automate it's going to ask us a few questions first our frequency in minutes now you can set this to whatever you like i wouldn't set it too low or anything too aggressive but at the same time i wouldn't set it too high or too lacks you do want to make sure that these are mostly in sync especially for your local dns records as you're setting them up but i think that 15 minutes is pretty reasonable next is the hour of day you want to back this database up and this really doesn't matter this job runs so fast that you could run it anytime but for most backup jobs i pick off peak hours so three and after that the cron job is scheduled and if you want to check the cron job we'll just run a cron tab e choose our editor and we can see the cron task here now if you pay attention to the first line it's actually running this with a smart flag or smart argument and the smart is a two-way sync and while i trust that this script will synchronize things back and forth efficiently and smartly i guess i'm only going to have my secondary pi hole server poll just like the command we ran and this is because i'm never going to use the secondary pi hole server ui i mean i get it if my primary pi hole server is down i'll have to use the secondary one but on a day to day i'm always going to use my primary piehole server so i'm actually going to change this to pull now this is totally up to you and i totally trust this smart feature but the way i look at it is i never want my secondary one to ever touch or affect my primary one so that this secondary one is kind of throw away and i can always depend on my primary for the right data so i'm going to say this and again you could totally keep it smart i'm sure the feature works great but i'm going to go with poll and so you may have noticed that we had two cron jobs and it even asked us if we wanted to back up our database and so worth mentioning here is that this is all it's running is this backup job and so this is really just a nice utility to backup your databases and restore them if you like and on a day-to-day basis you won't have to worry about this this is happening in the cron job but i figured i'd call it out just in case you need to restore the database at some point and you would just use this restore function and so that's all we need to do to have two pi hole servers and keep them in sync this is a really quick and easy way to ensure that when one pile-hole server goes down your house is still able to resolve dns and since pihol is so lightweight and so versatile it can go on almost any machine on your network and provide a little bit of redundancy now i know that this is very unofficial and i know we all wish that pie hole had high availability or some redundancy and some synchronization built in but they don't for now but if you're watching pyhole please add it please but this is as good as it gets until then so a huge thank you to the gravity sync repo because the next time i need to do some maintenance on one of my piehole servers i don't have to worry about anyone asking me if the internet is down because technically it's not down it's that dns is down and you know what i mean so what do you think about gravity sync what do you think about providing a little bit of redundancy to your pi-hole servers do you wish that pi-hole would add this as a core feature to their product if so let me know in the comments section below and while you're down there don't forget to give this video a like and consider subscribing if you haven't already and if you run into any other problems along the way hop in my twitch stream and let's figure it out so thanks so much for watching and until next time stream on my friends good to hear it's it's always dns yeah it actually kind of um it might be dns now before i was using an ip for my trunad server lesson learned lesson learned now that i got real dns running at home no more ip addresses for me i i get it that ip addresses are are fantastic um and they're kind of you know absolute but for me i'm going to stick to dns so that i don't run into problems like this anymore so i can move my trunad server anywhere and it doesn't matter because the ip address behind the dns entry can change and i don't have to touch any of my other infrastructure so that's another thing i've been going through a lot of painful dns stuff lately uh in the stuart home for sure well stuart yeah that's my last name anyways in the techno home i'll say that

Channel: Techno Tim

Views: 93,393

Keywords: pihole, pi-hole, ha pihole, h.a. pihole, gravity, sync, gravity sync, how to sync two pihole servers, high availability, run two pi-hole servers, load balancer, gravity-sync, open source, technotim, techno tim, docker, containerize, multiple pihole servers, pihole in ha, keep in sync, synchronize pihole, cname, dns, dns record, raspberry pi, pi zero, bare metal, virtual machine, FTLDNS, backup pihole, database, Blocklist, local dns, pi-hole high availability, highly availabile, ha, pair

Id: IFVYe3riDRA

Length: 14min 27sec (867 seconds)

Published: Sat Dec 12 2020