Put Wildcard Certificates and SSL on EVERYTHING - Traefik Tutorial

today we're going to regain control of our certificates no more self-signed certificates no more odd ports and no more http we're going all in with ssl for our external services and for our internal services too we're going to use wildcard certificates from let's encrypt so we can have an unlimited amount of certificates we're also going to harness the power of dns internally so we can refer to our services however we like and then we're going to harness the power of external dns with a provider like cloudflare so we can get wildcard certificates and we're going to do all this with fortaner in traffic and then after we get that set up i'm going to share all of my configurations with you with everything i've learned with setting up a reverse proxy i've learned a lot while setting up my reverse proxy with plex trunanz pi-hole proxmox sync thing homebridge and more and so rather than you spend all of that time figuring that out i'll share all of my config with you and if that sounds good to you give this video a thumbs up and if you don't like it at the end you can take it back so how are we going to do this first you'll need a linux machine and then you'll need to make sure that docker is installed on that machine and that's as simple as running a few commands that you can find in the documentation to copy and paste and follow along with you should be able to run docker dash dash version and see a version output this means docker is installed and you're on track next we'll need to install docker compose and you should be able to run docker dash compose dash dash version and see a version output the next thing you'll want to verify is that you have a static ip now this could mean that you've hard coded one on the server or you have a dhcp lease and reservation for this machine but either way you'll want to make sure that your ip address doesn't change and it's also a good idea to create a local dns record for this machine too now it's not required just makes things a little bit easier and people ask me all the time how i set up my local dns and i use pihole and if you need help with that i just made a quick video on how to do it in a couple of minutes but that's as simple as going into pi hole and creating a record that points to this ip address now notice here that my portainer server is named portainer.local.technote now i own technotim.live but i've created a sub sub domain within my domain so that i can refer this machine internally without that request going external and so if i do an nslookup for that machine ortainer.local.technotim.life i should see a response with my local ip address so this means it's working and this is probably a good point to mention that you should own an external domain the certificates we're going to create belong to this domain and you'll need to make sure that you own it it's also a good idea to set this up in cloudflare now you don't need cloudflare to do this traffic supports lots of different dns providers but cloudflare makes it super easy and has a lot of other benefits too and now we're almost ready to set up fourteener but we wanna do one thing first first we wanna set up traffic now what is traffic traffic has many different products but we're gonna install the proxy today and traffic's proxy makes exposing services really easy especially in containers and it acts like an edge router for all of your containers and so we're going to set up traffic as a reverse proxy to connect to our other services on our network and traffic also has let's encrypt and let's encrypt will let us get our certificates and then we'll use our dns provider cloudflare in this case to get wildcard certificates for traffic so this will allow us to provision any number of sites we have internally using this wildcard certificate and so before we set up portainer we want to set up traffic first so that portainer can use some of the labels from traffic so that our portainer server also has ssl and again don't worry i'll share all of my configuration so you can copy and paste it so let's set up traffic first so back in our server the one running docker let's create a folder first for traffic and then in here let's create a data folder and so if we look at our server we should have a traffic folder and then inside of there have a data folder then we'll want to cd into our data directory and create a acme.json file this will hold our certificates and then we'll set some permissions on this file a 600 then we'll create a traffic.yaml file and this is our traffic configuration so now that we created that configuration we actually need to put some config inside of it and here's what our traffic configuration looks like it's pretty basic but it works like this first we have an api key and the api is the traffic api and we're telling it to allow the dashboard so we're going to host our traffic dashboard and then we're going to turn debug on so we get some additional logging next we have two entry points one for http and one for https and then we're allowing insecure certificates to be in between now i know this sounds scary but this is pretty common and so what this will allow us to do is put our reverse proxy and our ssl in front of another server that has a self-signed certificate which is the case with a lot of services that are out there like proxmox and so we're going to turn that on so we can still proxy through traffic to get to those services and next for providers we're using docker so we're binding to the docker socket and then we're providing our config file which we haven't created yet but we will here in a few and then we're supplying our certificate resolver so this is whoever owns your dns for me it's cloudflare but traffic supports a lot more providers than cloudfare and if you're using anything else it's as simple as looking at their documentation and providing some additional keys but for me i use cloudflare so then it uses the acme key with an email which would be your dns provider account email storage or a file where our certificates are going to be written and then who our dns provider is in my case again it's cloudflare and then we'll have to set up some resolvers and so these resolvers are pointed at clown flares resolvers now you might think to put your internal dns here and that might work in some cases but i found that i need to use the external resolvers so point directly at their dns for this to work the next thing we'll need to create is our docker compose file now our docker compose file is a way to instantiate or create a docker container and this is why we installed docker compose earlier now you can do this without docker compose you can use it with the docker cli but you'll end up creating a ton of labels if you can't tell by this yaml file and so i found that it's a lot easier to use docker compose because one you have a manifest that you can hold on to for later and two it's a lot easier to copy and paste this into an editor like vs code rather than a terminal session so i'll walk through this but i'll only call out the highlights so you don't gloss over but this is our docker compose for traffic we're going to use the traffic image of latest we're going to name it traffic we're going to restart it unless it stops typical docker stuff we're going to expose some ports on it so do some networking here we're also going to pass some environment variables to it now these are going to be your cloudflare account and your cloudflare api key but if you use a different dns provider it's going to be those but here you would pass in your account information then we're going to map some volumes and so these volumes map from the host to this container so mappings for time zones for docker so the path to our traffic yaml file we just created we're saying it has read-only access and then the path to our new certificate file this acme json file and we're giving it read write access since we don't specify and then a path to our config file which we haven't talked about yet and then we're giving it read only access and next are our labels and this is why i was saying it's a lot easier to do it in yaml i don't think i've ever said that before something's easier enamel but anyway this is a lot easier than copying and pasting the cli command with lots of additional labels you just do it here in an editor and so there's a lot going on here so i'll just call out the highlights but these are the routers and the rules that i found that i needed to work with all of my services so you don't have to figure this out later but the highlights are first of all we're exposing our dashboard on this dns name so mine's called traffic dash dashboard dash internal dot local.technotim.life this can be anything you want but just make sure that you have a dns sentry pointing back to this portainer server otherwise it's not going to work next is our password to see the dashboard and this is in basic auth and so we'll need to generate a password for this and paste it in here and rather than talk about that now i'll have those commands in the documentation just in case you don't turn on the dashboard next we have headers redirects and a lot of additional stuff for traffic then we have all of our redirect rules for traffic so that if something comes in on http it's going to redirect it to https automatically then once again we have our cert resolver and so mine is cloudflare so you'll need to make adjustments if you use a different provider and next is where the magic happens so this is where we get our certificates so here i'm requesting a certificate for local.technotim.live and then i'm also requesting a sans or a subject name alternative for a wild card of any sub-domain hanging off that originally going into this i thought i could get a wildcard certificate for all sub-sub-domains but it doesn't work that way i learned that you can only get a certificate for one level deep so this was the easiest way that i found to do it was just ask for a certificate for this local subdomain so that i can get wildcard certificates for any sub sub domain so any domain hanging off of local.technotim.life and just to reiterate these are for internal services only and we'll configure these services in our traffic config and then another call out really quick is this network down here and this networks is actually really important this network called proxy so we're creating a docker network called proxy and saying external is true so what this does is allow our docker containers to also use this network so that when we come in through the reverse proxy of traffic it can route it to these containers on that same network called proxy and that's how we get ssl but they don't only have to be services within docker and i'll show you that here in our config so let's spin up traffic first and make sure it's working first let's create that network the one i call called proxy so it's docker network create and then the network name is proxy and then let's go back into the root of that traffic folder and let's copy and paste that docker compose file here so that we can run docker compose and spin it up so you'll want to copy and paste your docker compose file to this folder or you'll want to rsync it or get it there somehow and save it then we'll want to go into the data folder and modify our traffic config and you'll want to do the same thing copy and paste the contents from traffic.yaml to that file or our sync it there or get it there somehow another thing we'll need to create is our config.yml and so this config is a config that will end up passing to traffic with all of our routes now we don't need anything in there now we just need to make sure it exists when the container spins up that's as simple as just doing a touch config.yml and now if we do an ls we should make sure that we see config.yaml a data folder which houses our traffic config as well as our certificates and then a dockercompose.yaml file and if you see all this we can spin this up so we can run a docker compose up dash d this will create our container and now it should be running you can check by running docker ps and we see our container here and to see our dashboard all we need to do is go to that domain name that we specified in the config so for me it was traffic dashboard dash internal dot local.technotim.live and now i'm prompted for our basic authentication password and if you provide that password you should now see your traffic dashboard first things first dark mode always dark mode there we go but what's more awesome than a dashboard and dark mode is that we actually have our wild card certificate now so let's check if we go into certificates and we check our certificate you can see i'm using a wildcard certificate from my domain technotem.live it's actually local.technotim.live and this is why we added a subdomain in front so these are all routed internally they don't resolve externally and so now that we have our traffic container running with our dashboard running and ssl certificates now let's get portena running and put a certificate in front of that super easy to do so back in our server let's create a new folder for portainer and this is going to be at the same level as traffic is so you can see the pattern here all of my containers have their own folder so we'll make a directory called portainer in here we'll create a docker compose and in here we'll create a folder called data and then for that docker compose file we'll need to put some contents and you guessed it oreo and so here's what our docker compose yaml looks like we're using the image portainer we're naming it portainer and then we're going to restart it unless we stop it and we're setting some security policies as well and then networks notice proxy network and so this is how when you come in the reverse proxy it can route to this docker container because it goes over the network called proxy and then to portainer and this is the really cool thing about traffic it's a container so then we're going to set some volume so time zone stuff docker stuff to map to docker and then a folder that we just created data and then again we have all of our labels that we talked about we're going to use these labels for all of our containers running on that portainer machine the only thing we're usually going to change is right here our host rule and then our secure redirect to that host and so for me i'm naming my server portainer.local.technotim.live and again you'll have to use dns to create that dns entry and point it back to this machine the sportainer machine so it can get routed appropriately so this says hey when you see portainer.local.technotim.live come to this machine route it to this container to the internal load balancer on port 9000 and port 9000 sounds familiar right it's pertainers port so it's routing it to this machine through traffic through this docker proxy network to port 9000 on this container and then serving it out securely so let's copy and paste this or transfer it to that machine and then we'll save it and then we'll do the same thing daca compose up dash d so it pulled down the image created the container and now we're done now normally we would go to this machine on port 9000 but since we created traffic it should have automatically created a route for us so that we can go to it through the reverse proxy so before we go there let's look at the dashboard first so if we go back to our dashboard and go under http because this is a reverse proxy so it's http traffic we now see that portainer has a route and it has tls so this is really awesome so let's go to portainer over https now and if we go to that dns entry over ssl we can now see pertainers running and if we look at our certificate it's the same certificate that's the cool thing about wildcard certificates now we can create an unlimited amount of sub domains under this domain and have ssl and so when it created this route it didn't even go out to let's encrypt and request another certificate it just applied it to this route so super awesome so let's create our user and let's manage the docker api so connect and here we have portainer running and if we go under this local machine and then go to containers we can see our traffic container here so really cool we can manage it from here now too okay so we got traffic up and going we got ssl certificates they are wild cards we know how to add additional containers and have those route through the reverse proxy and use our ssl certificate but now how do we use an external service how do we proxy through this machine and get to something like proxmox super easy and i'll show you now the reason i'm using proxmox as an example is because it's a little more challenging than the rest but we'll take care of all of the challenging things in this one with some headers and some additional middleware so that you don't have to deal with additional services that also require that middleware or those headers and again just a reminder in case you're skipping through the video all of this will be included in the documentation so that you can use this one config and it'll work with most of your home lab services and i highly encourage you if you find more to open poll requests on this repo to add additional services that are common across most people's home labs so back on our server in our traffic folder remember we created that empty config.yml it should be right here in the root of traffic and if we edit that we shouldn't see anything there this is where config is going to go so this is config for the actual reverse proxy in our routers and any additional networking that we need to apply to the traffic service so it's empty now so let's add some config to it so we can get to our proxmox server okay so what does that config look like i've reduced it down to the most simple version of this that i could instead of confusing you with all of my config and here's how it works first we have a config so this is a config for http so a config for a reverse proxy then we have some routers so so this is the incoming request how does it handle that incoming request well i'm saying hey the entry point is going to be https and the entry point is going to be pve1.local.technotim.life so this is the domain name that i want to refer to this machine as so my proxmox server i don't want to refer to it by ip address anymore or weirdport i want to refer to it by this new dns name again you'll have to create a dns entry for this so i created a dns entry for pve1.local.technotim.live but i didn't point it at my proxmox server this might sound confusing i pointed it at this portainer server because this is my reverse proxy so this dns entry right here should not point at the service you want to go to it should be pointed at this reverse proxy this portainer server or this traffic container did i drill that in yet but you want to make sure that points to this server and not your destination that's coming up and so we're going to add some middleware of default headers and so what's our middlewares our middlewares are right here so we're going to do an https redirect so anything coming in on https is automatically going to get redirected to https then we have some default headers we add to all of these requests and this took a lot of experimentation to figure out and that's why i started with proxmox because proxmox does have some additional custom headers that it needs to know about in order to work because if you think about proxmox it's not only going to the http s endpoint to get to the server and to view all your stats it's also remote controlling that in a web vnc session so i figured it out so you don't have to and then tls this is always empty our service is called pve one this can be called whatever you want it should match this pve one up here generally and then our service which we're referring to is pbe1 so this is the key between these two these two absolutely have to match otherwise it won't find the service we're creating a load balancer and the url that it's going to route to is actually now the ip address of your proxmox server and i purposely left this as an ip address you could use dns here as well uh but i left it as an ip address just so it's really obvious and so this url will be the ip address and port of the destination you're going to so in this case for us it's our proxmox server and then we'll pass some host headers equals true and then down here the default whitelist or allow list in our ipl list is is just a range and you can choose whether or not to adopt this but i'm basically saying hey all private networks can get to this now you can constrain this or remove it all together but for me this just ensures that only ip addresses for whatever reason that are internal are going to be able to get to this and not ip addresses from the outside and then here's how it's secured secure chain middlewares and then our default whitelist and our default headers so this is all we need to apply now to traffic to actually create this new route to get to our proxmox server so let's do that so we're going to edit that config yaml for traffic and we're going to paste that config in here or ursync it here however you want to do it then we'll save it so we can do this a couple ways what i usually do is just force the recreation of the container and why do i do that is because when the container gets created it's going to look for that config in the slash config folder and bring that container up with that new config file that we just created so the way i do it is just docker dash compose up dash d and then dash dash force recreate right or wrong is how we do it so we're recreating that traffic container now it's pulling in the new config and now we should have a new route if we go back into our dashboard and after going back to our dashboard if we go into http we should see our new route so here it is host techno pve1.local.technotim.live so this means we should now have ssl for our proxmox server so let's check that out and if we go there to pve1.local.technotim.live we should see our proxmox server and you'll notice we don't have any odd ports on the end and we don't have any certificate warnings we look at our certificate it's the same certificate we've been using the whole time our wildcard certificate and if we sign into our server and shoot well forget my password right and if we get our password right and sign into our server we should now see our dashboard and here's something that should also work that wouldn't work if you didn't use my guide is vnc in or remoting into this server something that took me a long time to figure out and so there are lots of nuggets like that in my configuration that are on my github site and in the documentation site so i highly encourage you to use those instead of figuring out yourself and if you figure out some more options feel free to open a pull request and with this now you can host all of your home lab services with ssl using portainer and traffic and you can also use this for external services as well it would be as simple as port forwarding 80 and 443 to this machine but no with that comes risks so are you running portainer or traffic are you running nginx proxy manager or just plain old nginx do you think this is a little bit easier to set up if so let me know in the comments section below and remember if you found anything in this video helpful don't forget to like and subscribe thanks for watching what services do you want to run and so when you start to build up your foundation you got to think think like it don't get me wrong like i love the journey i love building up you know hardware and then putting software on top of that building in redundancy and layering on like all of this cool tech uh but you always have to keep your mind you know you always have to think about the end product what is the end product or the end goal i'm thinking of this more like product management or software development um you know you're building this platform to eventually do something and what is that something you want to do maybe it's host game servers maybe it's host web services maybe it's hosting a bot or hosting some code you wrote or a website or a blog so remember to always keep that in mind is

Channel: Techno Tim

Views: 85,950

Keywords: traefik docker, traefik, docker, docker compose, portainer, portainer with ssl, ssl, wildcard certificates, certificate, secure portainer, traefik tutorial, traefik reverse proxy, cloudflare, proxy, reverse proxy, expose service, self-hosted, selfhosted, letsencrypt, homelab, home lab, portainer ssl setup, proxmox certificates, portainer tutorial, how to get wildcard certificate letsencrypt, tutorial, guide, selfhosting, techno tim, dns, pihole, sans, subdomain, nginx proxy manager alternative

Id: liV3c9m_OX8

Length: 25min 1sec (1501 seconds)

Published: Sat Apr 24 2021