HAProxy-WI: Run Lots Of Public Services On Your Home Server

this video is sponsored by lenode thanks lenode use our affiliate link below and get sixty dollars uh for your initial credit you can play around with their kubernetes engine it's gonna be awesome you can also use lenode for this video for this setup in this video it's pretty awesome what we plan to do i've got wendell.tech like that's a domain name but you know i want to host stuff on the internet but i'm also kind of a data hoarder so and my internet connection is pretty fast so it doesn't make sense to me to pay for really expensive cloud hosting but i can still get a benefit from cloud hosting that is i can i can run a machine in the internet that sort of protects my home setup that's on my lan um from just the ravages of the internet at least provides a little bit of protection and a little bit of obfuscation so i can terminate window.tech on a lenode but it acts as a gateway for my home network a bunch of machines on my home network yeah it's a little dangerous and yeah you gotta dot the eyes and cross the t's and you know there's a little bit of risk here know what you're doing but i'm gonna show you how to set this up with h a proxy and there's actually a gui for h a proxy called h a proxy wi that i'm also going to show you so let's dive in and take a look [Music] if you're following on patreon this is sort of the you know second version of this video this is something that i've been working on for a long time i sort of did the first version of this or the search the first video on this on patreon to get some feedback from patrons and sort of run through some some setup and big thanks to snazzy labs because uh he let me use him as the guinea pig i guess to uh to do this setup and so we went through and set up his his stuff like this with his own domain and and sprinkled in some special sauce just just for snazzy so that's pretty awesome thanks snazzy there's a how-to on the forum and this is going to kind of run through that but here's the idea here's sort of the big picture idea we set up sub domains like plex dot wendell.tech and mb.window.tag which is another media server and nextcloud.window.tag and these subdomains run services now normally you could just set up you know multiple nodes or mult multiple virtual hosts on a single node and be able to run all those services out in the cloud all your files and everything are stored in the cloud well for me i actually want to run stuff locally and like an email server you know again with the data hoarding uh want to run an email server but i don't necessarily know that i want to have all of my emails forever stored in the cloud i mean a lot of bad things could happen with that setup that might not be the best setup you know somebody could break in and steal stuff and maybe that would be terrible i kind of want to have my email local so i can have my arms around it but i don't want to run an email server on the public internet well wouldn't it be awesome if you could have a server on the internet that acts as your gateway and it forwards traffic to your home connection and your home connection only accepts inbound traffic from that machine on the internet that's a little bit more secure it's also a little bit more obfuscated so that bots and i mean there are search engines for vulnerabilities now like showdown uh shohan just scans the internet and then later if it's like oh this version of this is vulnerable you can just go to that search engine and say show me who's running this version of this service and immediately find a bunch of servers that are vulnerable well if you run this setup and you do it right those kinds of automated scanners are going to have a real tough time figuring out what's going on with your lenode because it looks like it's just static web page hosting but if you massage it a certain way or you go to a special url or you do http authentication or http certificates authentication then it's going to give you a different answer now this setup is pretty basic we are going to set up let's encrypt i'll walk through that but i want to introduce you to ha proxies it stands for high availability proxy you know it's the reliable high performance tcp http load balancer yeah i mean it's gonna just load balance your single home internet connection but it's a really well supported robust open source solution it can proxy http connections as well as email yeah you can set up an email and mail exchanger set your mx to be the lenode machine but it's actually just going to proxy that connection back to your home machine and your home machine is actually going to run all the stuff that it needs to run to you know handle your email and your isp can't detect this because your home machine your home firewall is only going to accept a connection from the lenode and everything between the node and your home connection if you do it right is encrypted and it doesn't even run on normal ports anyway so like if your isp blocks inbound port 25 not a problem it's port 25 to your lenode machine and then like some random port like in the 50 000 which looks like just normal regular internet traffic from the lenode machine to your internal mail server yeah it's not not a big deal to do that you can have all those different services running on different ports but on the front end you don't have to specify the port number you just go to https you know nextcloud.window.tech and plex.window.attack and based on the name it figures it out that's the magic of h.a proxy all right let's dive into the how-to so if this sounds like it's a little too complicated and you need a gui that's h a proxy wi but i'm going to come back to that because in order to use h a proxy wi the gui you need a working h a proxy and ideally also a working engine x which is a web server it's a pretty awesome web server so let's start with setting up the basics and the first thing you need to set up is dns so i'm talking about names like wendell.tech and uh you know plex.window.tech and the easiest thing to do is to set up a wildcard a dns wildcard so you can set up if you have your own domain name most of your registrars you can just enter an asterisk for the a record or the quadruple a record a quadruple a record is for ipv6 a single a record that's for ipv4 you use a star for that and then you can specify the ip address of your lenode machine that doesn't have to be lenode it could be on amazon ec2 but you know amazon's not sponsoring this video the node is you could also run this on your home connection like if you're if you're cool with a dns entry pointing to your home internet connection and sort of amplifying the background chatter of things on the internet trying to break in sort of the zombie army i mean you can run it on your home connection you'll have to change the how-to a little bit there's sort of less extra steps your proxy basically just has a public ip address on one side and lan ip addresses on the other side so you don't have to forward as many ports through your firewall you really just need that that https port forwarded through your firewall and probably also port 80 unencrypted but the only thing port 80 is really going to do is just redirect traffic to port 443 for the ssl encryption and we'll get to ssl i've got a set up here with let's encrypt which is a little different than what h.a proxy wi the gui does out of the box but i've got a script for for helping you out with let's encrypt but we'll get to that in a how-to so setting up dns sort of a first step and i think you know i've explained it a sort of a high level what we're going to do so once you've got dns set up and and all of your various subdomains any subdomains that it could be pointing at your lenode ip address or your the machine where you're going to run ha proxy then you're in pretty good shape i would go ahead and configure nginx set up engine x and set up nginx you don't need ssl on nginx you want to set up engine x to run on port 81 on localhost it doesn't even have to listen on your public ip address you can just set it up to listen on localhost so you've got less of an attack surface area and that's just the best practice really so port 80 is going to actually be serviced by h a proxy as well but on port 80 on aj proxy we're just going to forward all of that traffic to local host port 81 which is engine x and then in our nginx configuration as you can see in the how to the engine x configuration just says oh you're on http now you need to redirect 302 redirect to https colon slash whatever host name you put in uh and that's it so and the url so if there's a url so basically that's just a catch-all it's like if you accidentally type http it's just going to try to force you to https which is also handled by h.a proxy for this how-to i'm using centos and it's just a you know yum install ha proxy um install nginx not really a big deal you do have to configure if you're using centos you do have to configure se linux to permit the http server engine x and h proxy in this case to make outbound uh proxy connections and you do need to permit um the http server or nginx to run on a non-standard port 81. there are some built-in ports i could have run it on another port other than 81 by just querying which ports were available but i forgot about that when i set this up so it's just as easy to add the port to the list of allowed ports now you might be thinking well i don't want to do all this command-line stuff i want to do it from the gui it's like well it's good for you to have some idea of what's going on at the command line level and then sort of manage it with the gui after the fact um once you get nginx set up and once you set up your uh front ends then you can sort of revisit it with the aj proxy wi gui so we'll come to that in in just a minute once you've got your ha proxy set up you can edit your slash etc hdaproxyhaproxy.cfg just as it is in the how-to on the forum and i've got an example front-end for you there the thing to understand with aj proxy is that it can proxy a couple of different ways one of those ways is it will actually sniff or look at the http header and then make some determination as to what it should do with that traffic based on what's in the http header ssl has this most modern browsers have this feature called sni which means that with the ssl request it also passes the host name so you can have one ip address in one port but you can actually run a whole bunch of websites on that one ip address that's we've been doing this for 10 years with unencrypted internet connections but with ssl because everything was encrypted it was previously hard to do that the ssl specification had to be modified and that's the s i specification so older browsers like ie internet explorer 6 for example struggle with s i but just about everything under the sun supports s and i at this point so when you do an https request it will pass the dns name along with it and he ha proxy knows how to read that dns request so we create a front end that has all the rules here these acl rules that um will be handled based on the host name now you don't have to do checks based on hostname you can actually do a lot of checks based on other things and in fact plex media server is a little persnickety like if you just do the hostname type proxying it didn't seem to be quite enough in my particular setup i actually had to add some special rules to handle plex to handle a proxy for a plex connection and with the plex connection the front end is ssl but also the back end is ssl so i'm sort of introducing more stuff that h.a proxy does so this this one front end that's running on port 443 is like a traffic cop and it decides which way it needs to route your request to which back end based on that host name that means that you also need multiple backend definitions in the h a proxy configuration so that it knows exactly where to route your stuff now the config that i have in the how-to you're not going to be able to copy-paste that and just use that because i've got a lot of back-ends in this example configuration to keep it you know not from being not super confusing doesn't have all of those back-ends but you can see the back end definitions are very simple we've got basically what you want to call it and the ip address that you want to send that traffic and the port and so this is on your local firewall this is your public internet connection on your home connection or wherever these servers are physically located and the port number on which this service is running i'm imagining most of you are just running a very simple nat setup at home network address translation it's like most of your internal ips are like [Music] 192.168.1.1.1.2.1.3 and your gateways maybe.1.254 something like that on your home firewall you'll forward from your one public internet ip on some random high number port i would suggest to port over 10 000 just because isps can do funny stuff with ports port numbers less than 10 000. for forward some port more than ten thousand only from your lenods public ip address you can set those security options so it doesn't just let anybody in on those port numbers it's only traffic from your lenods ip address or your cloud servers ip address on that same port number to a particular service so in the example here we're running you know plex media server on a non-standard port but it doesn't really matter because when you just go to https plex.window.tech h.a proxy takes care of that request that came in on port 443 the standard https port and then forwards that through you know that plumbing to the weird port on the public side and then the firewall itself will go ahead and forward that to an internal 192.168. whatever ip address that it happens to be running on and like magic the plex server is available this works pretty much the same for any http request the one exception to that is a mail server with the mail server i'm going to suggest that you use tcp style forwarding i didn't quite get to it in the how-to but if you want to run an mx you just you forward port 25 to some other port not 25 almost guarantee you your isp is going to block that you can run some some really high port from your node machine on your firewall and your internal side of it and uh you're going to want to use tcp forwarding and you're going to want to check the aha proxy documentation which is quite good it's the ha proxy service not the gui h.a proxy wi and you're going to check the documentation for doing tcp forwarding that doesn't work based on host name so any host name port 25 it will accept an inbound connection and it will forward to your home connection now your home connection doesn't actually have to be up all the time email is actually very resilient the email protocol means that when some server is trying to send your server email if your server is down it'll actually retry without any error for up to four hours that's the default so if your internet connection is intermittently up and down as long as it's not down for more than say four hours this actually works really well as a you know as an email proxy because you can run your email out on the internet and you can actually even add services to this you can chain services together so if you want to do gray listing you can forward port 25 to a local port on your lenode machine just like we're doing with nginx let the gray listing thing sort of say is this a good connection or not what ips is this coming from and then forward it on to your internal machine because your internal machine is not going to know exactly necessarily everything you need to know about the ip address that that came from and some other parameters there so you probably do want to do gray listing and that's it for a future how-to if you want to you know join the level 1 patreon if you want to fund this kind of thing because there's it took a little while as some of our patrons can tell you from the first version of this so once you've got this working at a very basic level at a command line level um you can use the h a proxy wi gui but we're still not i'm still not quite ready i'll mention one thing about nginx so in the how to engine x is configured on port 81 so you install nginx and by default it's on port 80. i switched that in the configuration to port 81 because i want h.a proxy to listen on port 80 and the reason for that is that i want to serve a static website with nginx and potential for other things later so an unencrypted connection comes in on port 80 h and that hits h a proxy and then h a proxy says oh this is unencrypted forward it to nginx and then nginx says well just forward this to port 443 which will then be handled by h a proxy the really interesting thing that i can do though is i can configure a default rule in h a proxy so that any domain name that is not recognized in the configuration anything i don't explicitly have an acl rule for will also get redirected to the local nginx server and i'll just serve a basic splash page so all the bots and crap like that scanning the internet if they encounter this they'll just get a splash page now i can also make it to where that like i can you know plex.window.tech requires http authorization or ssl certificate authorization or plex.window.text some domain name and it'll make it a lot harder to discover for a bot to discover what those urls are and sort of scan it based even based on a hostname so it is a little bit security by obscurity and security by security is not really security at all but every little bit helps don't do that as your only measure but be aware that it's an option and there's some fun stuff that you can do with that h.a proxy wi so my suggestion for installing h8 proxy wi is to do it through docker and it does require a little bit of a subscription it is it's from one guy so you know vetted security i don't know but this is the only gui that i could find for h.a proxy wi that is very very good now if you run pfsense on your router it actually has h.a proxy built in and there is a gui for it npf sense so if you're using something like pfsense for your home router and you're not going to do the lenode thing just out of the box it totally works fine and you have a pretty reasonable aha proxy gui it doesn't look anything like the ha proxy wi gui but it is a reasonable gui aj proxy wi gui actually does a lot of other stuff like monitoring and it can also manage nginx as well as h.a proxy so it'll help you write the acl rules and it has a gui for creating the rules initially then once the rules are created it just gives you a textual editor that you would use to edit the rules so what my suggestion for setting it up is to install docker on your lenode machine and then install the ha proxy wi docker container when you create the docker container if you're following the documentation on the ha proxy website which is fine i don't need to rewrite that uh one of the steps says to have it listen on port 443 that's the only step you need to change my suggestion would be to make it listen on port 8443 because you're your node with its single public ip address port 443 is meant for the traffic that is going to your next cloud and your plex and all that kind of stuff you could forward it through aha proxy itself but then if there's something wrong with your ha proxy config you can break it it's easier to just have it listed on port 8443 and then you can go to you know wendell.tech colon8443 and get the ha proxy login gui you did remember to set a good password by default three accounts are created you want to disable the ones that you don't use and set a secure password on the ones that you do and if you're gonna just use admin change the admin username like that's a security best practice don't have an account called admin you know pick like and i pick a like your name or something that is not easily guessable by a bot that's just going to scan the internet and look for this kind of thing now here's the here's the rub here's the only thing that is slightly unsettling about this software other than it's just from one guy which it is open source and you can compile it from github but here's a subscription service and we'll talk about that you need to upload an ssh key that has access to modify the configuration on your lenode so we've got this docker container running inside a larger lenode machine and we want to configure the docker container to be able to connect via ssh to the host machine so literally in the configuration for h a proxy it's like configure servers it means servers that are running already running h a proxy and nginx so we can create an ssh key on the lenode machine just ssh hyphen keygen t rsa if you want to create an rsa key and upload that public and private key to our lenode machine and then configure the public key and the authorized keys for the root user or another user that has permission to modify the aha proxy configuration using root is kind of unsettling to me but it would be a lot more steps to configure a user that is locked down to use that so depending on what your needs are for security know that that's like maybe a little asterisk and an exercise left to the reader and or viewer and maybe something for me to revisit in a future video but ha proxy will literally connect via ssh load the configuration give you stats and all this other kind of stuff it also gives you some gives you some neat graphs and monitors up time and shows you little little beads about status and updates things like that it's really meant for more of like a commercial use case i suppose but i emailed the author and talked to talk to him because initially he had some fairly expensive subscription options and he added a lesser expensive subscription option since i talked to him because i was like hey you know this would be a good thing a good gui for home users and it may be the case that the the work that i've done and the work that he's done maybe we can just wrap that all up inside one docker container so it'll run h a proxy wi and everything plus h a plus h proxy itself plus nginx inside one container for like the home user use case and then you can just handle everything at once the one thing that it does it will handle ssl you can paste your ssl private key and your certificates into the gui and then it will configure h.a proxy with that for you which is nice but you don't actually have to do that if you use the script that i wrote for let's encrypt and so that's the last part of the how to that's on the level one forum so let's encrypt is awesome it's a free ssl certificate service just know that when you request a let's encrypt ssl certificate the ssl certificate that you requested goes to a website somewhere so that is not secret information so if you if i requested superfoo bar secret something or other dot wendell.tech and it's like hahaha no one's ever going to be able to see that because it's hidden behind a wildcard and i'm going to do special things with that that doesn't actually work that way because it's going to show on the ssl certificate which is going to show on the let's encrypt side yes you can get a wild card uh ssl certificate from let's encrypt which you have to add a dns entry for that again an exercise left to the reader but you know there you go so with let's encrypt with the script here what it does is it stops nginx and ha proxy and then it runs the nginx the cert bot uh standalone and so certbot will then listen on port 80 and 443 for the challenges you specify the domain names that you're using that's what the dash d is in the in the example script here it'll get the certificate and then it will take the key and the certificate and concatenate them together which is the format that h a proxy needs and it'll stick it in it in the etc folder and then h a proxy uh can you can specify in your ha proxy configuration to use that file and then the script will restart h a proxy in engine x so you can cron this script to run let's say once a week or once a month or whatever and it will run and renew your ssl certificate and because it's stopping and starting h8 proxy it'll be fine this isn't a good way to do it for a production system because there's going to be that 30 second period where your stuff is completely unavailable while you get new ssl certificates you definitely should do the dns wild card thing because the challenges go through the dns side that way but when i was setting that up cert bot was actually crashing on centos uh trying to do that and so there's a ticket open for that on github and it's a known issue with the uh cert bot and wild card certificates and the plug-in for that and centos and something about a python version so you can also just specify all the different domains you're going to use and you'll get one certificate that has all those all those domains on it if you buy a wildcard certificate that's good for like three years you can literally just paste it in ha proxy gui and you're good to go so this is an adventure and that's kind of a long adventure and it's uh it's it's there's not like this really isn't complicated there's just a lot of steps but if you do this you'll have a lot more clear understanding of how internet plumbing works or doesn't work depending on what your needs are and there's some other little useful nuggets i didn't explicitly call out in a how-to about mechanically how all of this works but i fully expect a lot of comments on the level one forums uh again special thanks to lenode for helping sponsor this and big thanks to snazzylabs for being my guinea pig for testing this and also our patrons you know a lot of our patrons of floatplane subs asked for this kind of a thing because i mentioned it's like hey wouldn't it be neat if you had this kind of a setup and the reality is like the smallest least expensive lenode machine that you can get will run this just fine and it'll work great you don't need a lot of storage you don't need a lot of cpu horsepower and this will be a front end to whatever sort of beastly system you've got running internally i mean you could have a hundred terabyte plex server behind this and it'll just cost you you know a very very small amount of money a month to run i mean you can even run this on you know you can basically run this on a raspberry pi on the internet if you really wanted to i mean it doesn't take a lot of horsepower so i'm little this is level one this has been a look at h a proxy and by proxy h a proxy wi uh i'm signing out and i'll see you later

Channel: Level1Linux

Views: 41,166

Keywords: technology, science, design, ux, computers, linux, software, programming, level1, l1, level one, l1Linux, Level1Linux

Id: Kh21q7LY-F8

Length: 25min 24sec (1524 seconds)

Published: Fri Jul 24 2020

Reddit Comments

[u/GreeneSam]

I love HAProxy. I used to use it paired with letsencrypt to create my personal website and services.

Anyone here have any advice on running letsencrypt without port 80? Last I looked it was a pain to do.