IP Address Allow/Deny with Cloudflare Workers and Upstash Redis

hey how's it going everyone it's lee halliday and today we are learning about cloudflare workers and specifically we're going to build an allow denied list based on your ip address so we're going to allow certain ips access to our website and we're going to block everyone else and we're going to be doing this using redis so we're going to be storing the ip addresses that are allowed in redis and we'll be sort of intercepting requests and denying ones that aren't in the list so if you haven't worked with cloudflare workers before it's obviously a product by cloudflare which i use for my dns hosting and it's this code that sort of sits in the middle of a request from a user to your server so if we head over to this image that they have this is the example where visitor to example.com makes a request to your website from their browser or wherever the cloudflare worker sits in the middle so very much like service workers within the browser they can intercept requests and they can modify them both the requests going out and the response coming back they can sort of stop the request from going out to your server or they can just let it pass on through so we're going to be building this code that sits in the middle here intercepts requests and then based on the ip address decides whether to allow it on or whether to just respond that it's blocked there's a great article by lisa on the upstash blog on how to implement this ally allow deny list i sort of took this as the starting point but we're gonna work through the whole from from start to finish of how to start from zero and how to get it deployed and working on a domain that you own it took me about eight hours yesterday to figure out how the heck all of this entire ecosystem works but it all starts with this thing called wrangler wrangler is a cli tool that you can use locally to help with your development and you can sort of run these service these uh cloudflare workers locally just like you're doing localhost development so i'll link to this instructions on how to get started but we're going to start with is this typescript template it seems like the easiest way to get up and running building out a cloudflare worker so i'll share this and we're going to start from the same point and we basically just have to copy this line of code here and we'll go down to our terminal and i will get over and we'll change the name of this project to the allowed we'll just call it allowed so we'll run this it's downloading the template and then we can cd into the allowed folder and we'll just npm install everything and then we'll go take a look at the code so this won't take too long to get set up and then we will cool open our code and there's not too much going on in this example but there are two files i want to touch on i'm just going to remove the test folder because we're actually not going to be doing any tests today but they do have a setup that you can use to sort of like mock out the service worker and test that it's working okay maybe i'll save that for a follow-up video if you're interested so we're going to delete this and there's two things so this is wrangler.tomml file and this is where we configure what our cloudflare worker is called what what account id it lives under what type of code it is because they support rust and c plus plus or c or something like that we're doing typescript that will compile the javascript and why don't we just fill in these two things right now so i have my cloudflare dashboard open free accounts are fine for all of this and i'm going to go and copy my zone id and then we'll copy my account id as well and we can just leave the rest blank i don't think we're going to touch it anymore so close this now head over and i'll show you sort of an intro of what these things actually look like so they start listening for an event and the event we're listening for is a fetch event so remember if you go back to this image the user makes a fetch request to your server and we're sort of sitting in here in the middle intercepting this request so we've intercepted it here's the event with all of the information about this incoming request and then it's up to us to basically decide how are we going to respond what are we going to make the user's request do will we allow it to continue on will we respond with some text it's up to it to us and we're going to pass the event request to a function called handle request and right now all it's doing is basically every single request that goes through this cloudflare worker is going to respond with some text that says request method and then like get request or post request or whatever so why don't we run it so we can see what this looks like the first thing you're going to want to do is click a type wrangler login and that will make sure that you have your write cloudflare api key and all of that stuff i've already done it so i'm going to skip that step and just say wrangler dev so wrangler dev will build this and then it will give you a local host url that you can visit and then we see this text request method get so why don't we come in here and tweak it a little bit so what we can actually do is we can grab the ip address that my request is coming from so we'll say const ip is equal to request headers and we are going to get a header specifically that is cf connecting ip so this is a special header that cloudflare gives you about the user's ip and just because it could be null we're going to default with nullish coalescing to an empty string just to keep typescript happy so now we can put in here from and we can embed the ip address i'm saving it it automatically detects changes rebuilds your script and then we can refresh and now we get that this request is coming from 70.29 so this is my ip address great so let's continue with this example where we're trying the end goal is to basically block this request unless this ip address is in a redis database that we're hosting on upstash so we're going to work our way towards that example and the next thing we're going to do is basically check to make sure that this ip address is in the allowed list so we're going to say we'll just put a variable here allowed and it is going to call a function that won't exist yet but we're going to await for this function and then we'll call it is ip allowed and we'll pass in the ip address so we can just declare this right here below so it's an async function is ip allowed and the ip is going to be a string and it is going to resolve to a boolean just like that so if we just return false right here so it doesn't like that right because it's an async function we have to say that it's going to be a promise that will resolve to a boolean so now it's happy and we'll check here if allowed and then we can if um if it's allowed we could do something we can allow it to continue on to where it was trying to go to why don't we handle the blocking right now so if not allowed and then we'll return a new response that says something like um ip is not allowed and will be nice and we'll set the status of the response to 403 which i think is forbidden cool so that means if we get down to here that means this ip address is allowed so why don't we just change it and we say ip whatever is allowed like that we'll let it rebuild which just takes a second we can refresh and then we get this ip address blah is not allowed okay so we obviously need to go fill in this is ip allowed function here and for that we're going to go set up a redis database on upstash so i'm going to go to my up stash console that i've already logged in so if you haven't logged in you'll click here click on console i do github connect because it's pretty easy and i keep my sort of development logins together and we'll create a database we'll just give it the same name as our app allowed and i'm going to put this in we'll say north virginia yes we want tls and yes strong consistency and this just takes a few seconds to get set up and then we'll be good to sort of continue on with the rest of this example awesome so the one cool thing that upstash gives you that most or all other redis providers don't is that they allow you to connect to your redis database both through the typical redis connection tools so if you click on redis connect you can use node redis and node you can use i o redis you can connect through python so you can connect the typical way but we can't do that in this scenario because you can't install these and use these inside of the cloudflare workers so instead what they give you is a restful api for you to interact with your redis database so we're going to copy this url and later we'll come back for this header here but we'll just do one thing at a time so what we're going to do is we're going to say the url is going to be this so this is how we connect to our database and then these are sort of the functions that we're performing on redis so the function that we're performing here is set so a set is a unique array where we're going to store all of the ip addresses and we're just going to be checking if something is a member of this set so it's a s is member so what set are we looking in we will call this the allowed set and then we need to pass in the ip address that we want it to check for so we can just embed the ip address in the end like that so then we want to make the fetch request to this restful redis api so we'll say response is equal to a weight fetch to this url and we need to pass some headers so we need to pass in the authorization header so we can say authorization is equal to bearer and then we need to put in the long basically the access key the token or the password that will allow us to connect to our redis database so i'm going to copy this and yes i could just embed it in like this call it a day but it's not a good idea to commit this and and leave it sitting in your code and there's actually a better way with cloud filler workers and the wrangler cli tool to put this in as a secret so that's what we're going to be doing we're going to be cutting this out and replacing it with we'll just call it our redis password just like this so it's typescripts freaking out right now cannot find this don't worry we'll take care of that in a second but instead we're going to get out of this tool um running the wrangler dev and we're going to instead say wrangler secret and then that will tell us what commands we can run and we're going to put a new secret and we will call this our redis password the same variable i just used in the code run this it's going to ask us to paste in that secret and it's creating the secret for the script that was named aloud from our wrangler.toml file and it's up there on cloudflare now so now we can use this redis underscore password so this script would actually run but right now typescript is blocking us from being able to run it because it still doesn't know where it's getting this value from so what we're going to do is just create a new file dot d dot ts where you can declare sort of global typescript variables and we're going to declare const redis pw which is a string so this would make our script happy but we need to tell our typescript config to include that new file that we created because right now it's only looking in the src folder so we'll just put dot d dot ts like that and it might take a second but that should eventually perfect make typescript happy so it will read this global definition file know that redis pw is available to us and now we can actually use this so when you run wrangler dev next and it spins up it will actually inject this redis pw password variable into our script so it pulls it down from cloudflare the the secret we just up uploaded okay next we need to basically get the result back from the our redis database so what we're going to do is we're going to say data is equal to a weight and we just need to convert the response into json and for now i'm just going to say console.log so we're still going to return false we're not using it yet but let's just see what redis is giving back to us so if we go back here we refresh the page and then we should be showing us sometimes it takes a second there we go so this is what redis is giving us back it's giving us back result that's either going to be a zero if it is not in the set or a one if it is the ip address so what we're going to do instead is we're going to say result like that and we're going to give it a type hint and we're going to say result is going to either be a zero or a one so what we can do now is we can just say return result is equal to one just like that so basically if it's equal to one you're allowed otherwise you're denied so at this point we should still be denied so it still is not allowed and maybe we should go at ourselves so that we can see what it looks like when you are allowed so to do that i'm going to go back to the up stash console close this and i'm going to go back into redisconnect and the redis cli is right here and this is basically how to connect to this redis dashboard from the redis cli tool that i've installed locally using homebrew when you just do brew install redis you also get the redis cli along with it so we're going to copy this why don't we just go to a new tab so we don't have to stop wrangler over here and i'm going to connect here so now i can interact with this same redis database but from my console and i'm going to call the command s add for set add so that the key that i'm doing it to is the allowed set and then the member so this would be the actual ip address that i want to add to the set so i haven't memorized my ip address so let's go copy and paste it and boom it's in there so it should be if we go back and refresh now it's allowed so i've just added my ap address to the allow list so now i can visit correctly this website so remember this beautiful diagram visitor visits website worker checks to see if my ip address is in the correct set and if it is it should allow the request to go to where it was heading in the first place otherwise it will respond with denied um okay so we've sort of we're at like an incomplete example right now i said we're going to allow the request to continue on to the end point but like where is this request going i'm just going to like 127.0.0.1 etc so what i need to do is instead of returning responses allowed i need to basically allow that fetch request to continue on and i'm going to put in one little thing to basically allow for this to work well locally or when i've deployed this up to cloudflare and what we're going to be doing is attaching it as a route to a subdomain that i have workers.lee holiday.com so we're going to check as route and the way we're going to basically do this is first look at the url where the request is going to so the way we get that is we can just say new url is request.url and then what we can do is we can say if it does not url.host.end with workers.dev so why this when you deploy your cloudflare worker up there they'll sort of give you a url workers like something aloud lee holiday.workers.dev so if we're just accessing this website um via this url what we're going to do as route is just respond with ip address is allowed otherwise actually i want it the opposite [Music] if not as route sorry about that so if it is a route meaning we've attached it to a to an actual subdomain and if this is confusing hopefully we'll make more sense in a second when i actually attach it to a subdomain so if it's a route we are going to return uh just fetch allow the request to continue on this request otherwise we'll return just ip is allowed and this is basically for testing whether it's working locally so if to illustrate this point a little bit better why don't i console.log these things so the url and the as route in there we'll just make sure it builds correctly and then i'll show you what it's doing hopefully i didn't mess up um wrangler watch change message failed to send i don't know why don't we just restart it okay i think it's doing okay so we're going to come back here refresh it so we're still allowed right let's go check what it was logging out so url as wrote false so i actually just want um why don't we do the url.host like that let it build upload okay so refresh cool so even though i'm running it on localhost.whatever 8787 port what it's actually doing is it's proxying the request to this allow.leighhalliday.workers.dev which is where my cloudflare worker is running on so it's sort of weird and um what we actually want to do is basically have this cloudflare worker listening to requests going into workers.legality.com so this subdomain i've already set up in cloudflare so if i go to dns i am pointing workers as a cname over to versailles so if i were to go to versailles i've added workers.legality.com as one of the sub domains that's attached to this specific app that's the landing page for my next level next gs course so this sub domain is already set up and it's good to go and what i want to basically do is tell cloudflare to basically intercept requests to this subdomain and pass it through the cloudflare worker that we got set up so to do that we first need to publish our worker so we can do that with wrangler publish and what that's going to do is basically upload this worker to cloudflare so it will just take a second and now it's available at this domain that we were looking at before so you can see it's now available live and it's it's allowed and that's why i had this code here to basically check if i'm just running off of this workers.dev don't allow the request to keep going because it's not really trying to go anywhere it's just trying to check if i'm allowed or denied so return that it's allowed but if it's running elsewhere so if it's not on this workers.dev domain allow it to continue on so we're going to attach it now to my workers.legality.com subdomain so we're going to go into workers um you can see here if i go into manage workers i have my one worker here it's sort of cool it gives you some stats about like how many requests is it receiving and how long does each request take so it takes like under a millisecond which is crazy that's how fast your call to redis is which is insane um but we want to attach this to a route so we're going to add a route and we basically tell cloudflare like what what sub-domain or what types of requests should pass through this cloudflare worker so we're gonna allow it to listen to anything on dot workers.leealiday.com any path we could limit it to just the api path if we want but we're going to allow it on anything and we're going to attach our allowed worker here okay so now when i visit workers.legality.com it should still be the same because my ip address is allowed it so it's allowing that fetch request to continue on to the endpoint but what if we were to go back into the redis console and say remove from set the allowed set my ip address so 29.72.160 okay so my ip address is blocked now so now if we come back here and refresh it just took a second but i p address is not allowed so anyone that tries to go to workers.lehalid.com that request is being routed through the cloudflare worker so if we come back here we are checking to see if the ip address is allowed if it's not allowed it responds with this which is what we're seeing here if it is allowed i'm just i've got this if statement so if it's attached to a route in this case it is because it doesn't end with workers.dev we're going to allow the request to go through otherwise we're just going to say it's allowed so i could come back here and i could re-add the ip address to the allowed set come back here refresh oh on the actual one and it allows the request to go through so we just built an ip address allow deny list with cloudflare workers storing the ip addresses in redis one question that may come up is why store them in redis when cloudflare has its own key value pair database distributed database you can use and the reason is redis you can connect to through a lot of different ways and a lot of different tools such as this redis cli here that i'm using in the console such as a rails application a php app a node app next.js wherever so it gives you a lot more flexibility to connect any way you want to and manage your allow and deny list whereas if it's using the the database that uh the key value database that cloudflare provides you don't have that level of flexibility so that's why i chose to use redis through the up stash rest api that they have now you can even see the the document the documentation the throughput the graph that that my redis database is receiving and how many connections it has and number of keys and all that all that stuff so i hope you enjoyed this if you wanted to take this further on the cloudflare documentation they have a ton of fantastic examples of like how you could just return like html how you can redirect the user to a different request how you can do a b testing so depending maybe what cookie is coming in with the request you could serve up one website or another how you can alter the incoming headers or the response headers and there's a ton of different things you can use when you're sort of intercepting that request and sitting in the middle such as what we covered today building an allowed deny list deploying it publishing it up to cloudflare and then attaching it to a route so that it can block or allow requests going to my app hosted on versailles so you can see how fast it is just as an example if i go into the network tab um sometimes it's a bit slower than others there we go there's a fast one so 183 milliseconds it went from my browser through cloudflare up to redis and back to check if i'm allowed and then to versel and back all in 183 milliseconds which is pretty crazy but as we saw earlier the the worker itself isn't actually taking very long to process the request one millisecond of uh sort of cpu processing time um now keep in mind cpu processing time is different than actual time because while it's waiting for the request to return from redis or wherever that doesn't count as cpu processing time that just counts as like waiting time in the uh node event loop but anyways these are details that don't matter hope you enjoyed this video uh i'll publish the code so you can follow along yourself take care bye

Channel: Leigh Halliday

Views: 997

Rating: 5 out of 5

Keywords: cloudflare workers, redis, allow/deny ip

Id: g6hGJcuscoM

Length: 30min 4sec (1804 seconds)

Published: Mon Jul 19 2021