Building serverless applications with Cloudflare Workers' Cron Job Scheduler and Key Value Store

hey everybody my name is christian freeman i'm the developer advocate for cloudflare workers and today on the channel we're going to be talking about scheduled workers or cron triggers as you've probably seen them referred to in our docs so real quick i'm going to show you how to use schedule workers so writing your own and then deploying it using wrangler and then after that i'm going to talk to adam who's built a status page project using kv ron triggers and cloudflare workers and kind of show you what it looks like in practice to use crown triggers and deploy something to production so let's hop into the video and excited for you to learn more about scheduled workers okay so to start i'm just going to make a new project using wrangler so i'll just say wrangler generate my project if we look at index.js you can see that it's this is like kind of the standard default wrangler template here so i have addeventlistener which has this fetch event that it's looking for and when one of those requests comes in it just calls this handle request function so this is all fine maybe i'll delete this comment just to give myself a bit of room but what i'm going to do here down at the bottom is also listen for a new event called scheduled and so that's kind of the string value here i pass in so now worker supports two different event listeners fetch and scheduled and to uh respond to that scheduled event i'm just going to call a function here passing an event and i'll just say event wait until do scheduled thing and i'll pass in event.request and so just like with my other function here i can write async function do scheduled thing which takes in a request and in this case i don't really have anything that i want to do here because this is just going to be like an example but you know i can just say returning response okay maybe here i will actually uh you know go do some tasks maybe go update an api or get something like weather data so with my schedule worker actually written now i need to publish it and the easiest way to do that is to come to wrangler.tamul okay so i filled in my account id and now what i need to do is actually add the cron trigger syntax into my configuration so to do that i'm going to make a new section called triggers and underneath that i'm going to say cron's equals and then i'm going to give it an array so this is going to be an array of cron strings if you aren't familiar with that you should check out crontab.guru which i'll put a link to in the description it's just a simple editor with a bunch of uh interactive ui for writing uh crontab expressions and in this case i'm gonna use this one i already filled in which is basically hourly so run every hour which is what the second star means at minute zero run it basically you know at the beginning of the hour and so i'll just say zero star star star star so now i'm ready to publish my project i'll run wrangler publish and you can see that it will get it ready to build there's actually no build here to do because it's just a plain javascript project but it's now published it with this schedule zero star star star star so it's published my function and it's also going to run this function specifically this section down here which is my scheduled code and it's going to run it at the beginning of every hour if you're interested i have an example project that does something a little bit more complex that you can go and check out it's on github at signal nerve workers weather cron and basically what it does is it uses a weather api to send the current temperature using telegram uh and a bot account so this is a pretty common you know whether examples are pretty really really common so if you're interested in learning how to set this up but understand how a schedule worker can run something like this every day or every hour or something like that i'll put a link to this in the description you can go check it out so like i said in order to kind of showcase how you might use something like this uh in production with a much bigger and more complicated application i wanted to invite adam on the show adam is one of our community champions in the workers discord and he's built a really awesome status page project that uses workers and kv and cron triggers to check your site's uptime uh it's a really cool project and you can deploy it for free now with our workers kv free tier and stuff like that so it's really worth checking out and without further ado i'm going to invite adam on the show and we'll learn more about his project hey everybody i'm here with adam who's going to be walking us through his status page project that he built with workers adam you want to introduce yourself real quick hey everybody uh thank you for the invitation uh my name is adam and i'm currently working as infrastructure engineer and i just have passion for automating stuff and uh lately building open source projects including the status page yeah uh and i'm super excited that a lot of people internally are really into adam's status page project it kind of combines a lot of the things in the workers ecosystem really well um the thing that we're going to try and talk a bit about today that we haven't done a lot of kind of content on in the past is the cron triggers functionality which powers a lot of the kind of um well i mean i guess maybe to start if you just want to share your screen and we can kind of walk through it as i kind of intro it but but the idea you know traditionally behind a status page or something like that is i have some website or something and i want to make sure that it's up you know not just like in the middle of the day when i'm working on it but also like at weird times in the day right the middle of the night and stuff like that and so cron triggers is uh you know what allows you to go and you know run some code every half an hour every five minutes or something like that and and make sure that things are still looking um like you'd expect so i guess to start you want to kind of just introduce the project and talk about sort of how it works at a high level yeah sure so uh the status page is simple page written in uh react actually using uh the flare react framework that is really cool to dynamically render react applications yeah on the edge and uh internally it's using the chrome triggers uh which you can run every minute or based on the on the current schedule the thing it does it just checks for a set of uh url addresses and saving the the state to the kv value storage right so and for people who aren't familiar with workers kv that's our key value store um it basically allows you to uh the very simple version is it it just lets you store data at the edge so you know in this case if i want to have some sort of kv data set up for like workers.cloudflare.com i'm assuming that the kind of structure of it is like you know workers.cloudflare.com and then like an associated time stamp or something like that and and i don't know we can get in the details of how it's actually stored if you want to but the idea is like instead of you know sending that out to postgres or to like firebase or something like that you have a pretty simple api for just storing data um and retrieving it from right inside your workers um project how how long have you been working with kv before this project were you familiar with it already or was it pretty new at this point it was pretty familiar uh i used to do some uh smaller workers that were using the kiwi as awesome and how about um flare react so that um again if people know about that project it's uh it's kind of like a next js sort of api but it's not uh it isn't next but it's like based on it kind of conceptually um but it's like a react framework for for workers were you pretty new to that as well i've actually never used flare react i know of it and i'm pretty familiar with it from like a high level but haven't actually built anything with it so yeah the flare react was uh totally new to me as i was kind of looking for a new ways to uh explore new new uh things on the edge so one of it was to build some kind of front end uh with some parts rendered on on the edge and that's how the status page idea uh come up basically yeah yeah that that totally makes sense i i've heard a lot of good things about it and uh and yeah i can imagine there's already a lot of really cool stuff you could probably do with um like it's we we've done projects with like html rewriter which is another platform tool we have and like getting data from that and in-lining into react applications and stuff but at the end of the day that all happens on the client but having this stuff be rendered on the edge is like that's very very cool and exciting exciting tech so yeah in case of the status page i'm basically reading the the state from key value uh key value storage and passing it as props if you are familiar with next.js uh to render the page yeah so in this case then is there any sort of client rendering that's going on or it's literally a hundred percent like it just comes to you as like a static there is client-side rendering for uh the days basically guys uh this is something i don't want to render on the edge it was working it was consuming uh quite a lot of uh cpus especially if i had uh like 50 monitors or something like that so you just get like a big kind of payload of all the like days or something like that and then render that on the client yeah yeah yeah exactly cool um so how does it work uh like under the hood if we can look at the code i guess to start all right yeah we can yeah maybe let's start with the the aspect of like going and checking for if something's um you know if something's up or down basically like the crown trigger part of it how how does let me yeah let me first show the uh the part of the the the event uh scheduled event i guess yeah so uh everyone is familiar with with a fetch listener but listen on the http requests yeah in this project i'm rendering the page and then you can set up the the scheduled event and do stuff on the current schedule so yeah let's jump into the uh configure function can you see it properly uh no i can't i can only see your browser really ah okay let me let me yeah i was i was showing uh the listener and then the the scheduled one so you can have uh you can have them separated right so i guess i should kind of explain for people who aren't super familiar with workers so you have basically your entry point to a worker is your event listener like adam was saying and traditionally this has been the fetch event which is your incoming http requests so anytime a request comes in from a client uh there's going to be this fetch event that's going to be responded to and that's where you're going to send back like html or json or whatever you're doing with your serverless function basically right and then the scheduled event listener is specifically for cron triggers and so um you can say you know when a schedule event is fired and we'll look at like setting that um you know setting that like schedule event how often it should fire and stuff like that but basically um you'll call some sort of function so adam has process cron trigger here and you'll just respond with that um well i guess you don't even respond with you you just you basically say wait for this uh asynchronous function to finish so go do some task and then at the end of that you know your your cron trigger schedule task will be complete uh exactly yeah yeah uh just to add some some notes in my case i'm actually uh i think returning yeah response okay because uh for easy testing uh i did set up a simple uh fetch event uh so locally i'm just uncommenting and the this line and calling the process configure another http request so you can like manually make sure that it goes and does the thing you need it to do and then probably get the error back if something goes wrong yeah yeah cool yeah yeah cool makes sense uh all right so uh this is the function uh what it does it's just reading the uh state of all the monitors let me show the config first so you can actually fork this repository and use it by uh yourself and all you need to do is just change the uh configuration cool yeah and so a monitor would be like uh an endpoint basically or like a url that you want to test right exactly yeah so you can set up up to 50 monitors uh that's because the limitations of the workers internal and yeah in the script in the processed chrome trigger i'm just going through all the uh items in the in the monitors and doing basically let me find it so yeah loading the uh configuration and then looping through the configuration and doing the the fetch request on on the url cool yeah so and and then you're setting up a bunch of like custom uh you can sell like a user agent and stuff like that cool yeah and and that um i imagine like with fetch yeah like on line 54 you basically just like say fetch the url and then like the fetch api is so easy to use that like you just pass on that and then object and it's really like no different than if you were just gonna go get this like in a regular workers function like a status page check isn't super it could i mean maybe if you want to do something like super complicated with it you could but like generally it just means like is am i able to get 200 back from this basically right which is which is exactly easy to reason about yeah the default uh expected status is 200 and you can also specify your your own in the in the configuration so if you want to test the uh status that it returns 502 for example you can it doesn't have to be 200. okay cool yeah go ahead say i'm just again like constructing the full state of all the monitors and writing it back to the same uh same key in the key key value storage and in the meantime if i recognize that the the monitor is not operational that basically means if the expected status does not equal to the uh redonate status from the fetch fetch uh method you can specify the slack webhook or even uh telegram it will send you the notification that's super cool and all of that is just configured it looks like using uh secrets right so you just pass like a wrangler secret which is kind of like our kind of our version of environment variables for your worker i guess is the best way to think about that i'm assuming that's what this must be right yeah yeah cool that's awesome are you gonna add uh because i know we talk on the workers discord all the time are you gonna add a discord integration to it or i don't really know i feel like that may be more complicated than i'm already looking into that yeah yeah so uh i definitely wanna try to add a discord and i think uh recently someone shared some some kind of library you can you can use just import it into your workers and you can just like use it yeah or easy as running the fetch method you get like a url that's like an authenticated url that you could probably set as a secret and then you just like give it a json payload and that's like the simple version of it it's not too bad but if someone has a library out there i would definitely i've been using some discord stuff myself and could definitely improve the code that i wrote so cool so so basically so far what happened is you know you have all these monitors that get set in or you have like the monitors that are in your config and then you have an associated thing in kv which says like you know we checked this last at i don't know 11 30 or something like that and it was up now we're gonna go fetch it again or uh you know make a request to that url and then if it's changed we're gonna say oh something is you know the website is down or whatever all right i suppose it must also be if it was down before and now it's up then you probably send a notification for that as well right so it's just like if it changed basically yeah yeah yeah uh i think the logic is somewhere here yeah if the monitor status changed and that's that means if the last status equals not does not equal the the current status yeah okay cool and then so after that then you send notifications if you need to and then after that i'm guessing you probably just go and put it in kv right or what happens then after that yeah putting that to 2kv uh and that's basically it i'm doing some some uh things uh with a response time it's near uh near real i would say let me check uh oh yeah i'm basically uh getting the the uh time before and after uh got it yeah so it it just gives you a sense of like how quickly your page is loading or something like that right yes and then you can see that on the on the status page if you will go to the current day uh you can also see and that's another cool thing about the chrome triggers it runs from multiple uh cultural data centers uh it gets scheduled during the day to four of them as far as i could uh see yeah and it runs in the least uh uh utilized data center during the the time i would say right yeah we're doing like some sort of resource conservation by saying like what are the ones that are not being used as much our data centers are running there but it also uh it makes the project look very uh makes it look very like expensive you know or it's like wow cool it ran in warsaw it ran in singapore that's so cool yeah this guy has these servers everywhere that's cool um yeah awesome so um great and and i think like that so the general idea of like oh so let's say you wanted to do like a one-off status check right like i wrote a serverless function where i had my website and i wanted to go check all of this information like that is pretty easy to reason about you basically do it like you've done here and then when it comes to actually like persisting it into kv basically all you're doing is you're like getting a json string out of kb and you're parsing it into an object you change whatever you need to change to indicate like whatever the most recent uh you know like read is and then you just save it back to kb right like the data aspect of it isn't super hard to reason about um but then the cron trigger aspect of it is like okay now i want to do this you know every what is it every five minutes or how often do you run it usually i run it every one minute every one minute cool yeah yeah so and i think the freak here is actually enough to run it every two minutes okay cool so yeah i mean it's it basically is like if you have this task that you're already doing now you want to do it with some sort of um recurring schedule um you just add a crown trigger to it and then now you're doing it uh it's going and running it for you on its own all the time so how so you know someone sets up a config for the project the sas page right they define all these endpoints you have all this functionality defined for them um to go and do it how do you actually tell um workers like okay i'm ready deploy this and set up a cron trigger so can you walk us through how you set that part of it up yeah sure so i actually used the wrangler uh action for for github so all i needed to do is uh prepare some kind of uh default configuration where i'm saying uh or specifying how often the current trigger should be uh run or scheduled so that's simple as adding the crons you can specify more than more of them so maybe you wanna uh check every five minutes and then i don't know if that's a good example and then every every one hour or or something like that you could say like well i think a good example this would be like let's say you have a site where you know it's really busy at one time during the day like say it's a journalism like a news session right i want to check at like 9 00 a.m est and then i also want to check like every six hours or something in addition to that but you know like our peak traffic is at this time like we really got to make sure that it's up at the specific time um yeah so and and this is um this is what i just lost you completely on my screen because i somehow just hid everything on my computer windows is awesome uh so this is like a cron uh what do they call it it's cron syntax so if you've ever worked with the cron tool in unix and linux it basically i believe it's like uh each little column here is like minutes is it minutes seconds hours yeah something like that it's we can put a link in the description for people to find it but um yeah this is exactly what i'm using yeah yeah yeah so you'll get uh you basically can you can go to krontab.guru they have a bunch of examples i think on the homepage too or you can basically say like what you want it to be like maybe you want it to be every day or you want it to be every hour yeah it's this examples one and then you get the specific string you put in there so so yeah and and in terms of the wrangler integrations that's our command line tool um so basically you just put in that string that you want and then when you wrangler publish your function or in your case if you're using the github action setup whenever you push something new to the project it will just attach this trigger to it so you didn't have to do anything in the ui or anything like that to get the setup uh just i'm assuming right hopefully didn't didn't happen it was just all kind of configured here from our command line tool uh like our config file yeah awesome is there anything else uh like kind of interesting or notable that you wanted to to kind of call out from the code base of the project uh i mean i don't think so like i had to do some some uh custom uh git up action on top of the wrangler action uh because by default the wrong collection does not support uh some kind of default uh secrets or default variables i i need to create a dummy secret in order to the the uh longer publish uh succeeds yes that is a known issue that i have been trying to get us to fix for a while yeah you can't you can't publish without having those secrets set right so yeah you have to get it yeah that totally makes sense yeah and yeah i did simple work around when i'm uh specific uh yeah the the secrets to some default and then basically checking uh not here yeah if it's specified and also does not equal the default then i'm processing the the select notification or the telegram notification got it yeah yeah it would be cool if we had some sort of like wrangler dot and i mean like dot m f like kind of that project or like the open source thing that a lot of languages have where you like have a dot end and if we just like automatically set it based on that that would be cool i'm gonna go think on that um so if people are interested in the status page um it looks like i mean clearly you spent time on like getting in a place where people can deploy their on their own right so what's the best way for someone to you don't have to actually like walk us through deploying it but if someone is interested in that where do they go and and like start that process i think the best way how to like discover new things is uh the build with uh page on workers yeah where our multiple projects actually listed so uh mine is actually the first place and you can just just yeah uh choose it visit the website no no uh go to the code base on github and from there uh there is the deploy with workers button that will actually walk you through the uh setup yeah cool uh i'm glad that that's in there i spent many months building that at that page so i'm glad that everyone's using it um yeah cool and then what what that will do is it will basically fork this onto your github repo and it will set up all the configuration details that you need to just have the auto deployment with github actions um so yeah that's awesome well thank you so much adam um i guess one last thing where if people want to like follow you and the stuff that you're working on sound you said you're working on a bunch of open source stuff where should they go find you online uh yeah sure uh i think it's listed in here as well uh so i started to be pretty active on my twitter so it should be not here twitter com or uh on the i'm pretty active on the discord workers community so feel free to find me there and then i jump yeah uh we'll put a link to that in the description as well um yeah and and i'll just plug it real quick so we recently launched our workers discord uh where people are sharing if you're interested in stuff like this like cool open source stuff you can build with workers adam and many other people are building super super cool stuff and sharing it in that community there's like i think close to 900 people in there now which is crazy to me that's growing uh really really quickly so uh cool man well thank you so much i and i i think this is like a perfect introduction to the kind of thing that you can do uh with cron triggers i mean people have raised like 15 million dollars for this exact page as a startup right and you built this you built this with like with kv and react and crown triggers and it works uh i'm i should go make one for all of my projects my my ghost blog was down yesterday after i sent my newsletter to like 4 000 people and i didn't notice it until i went on the website and i was like i was like oh uh so this would have been perfect for me especially if i can run it every minute so uh cool man well thank you so much and uh we'll put a link to all of this stuff and and give people a place to go if they want to deploy their own uh version of this but i appreciate you taking the time thank you for having me okay so that was hopefully a informative kind of look at how scheduled workers can be used in production i'll link to adam's twitter and you can check out his status page project and deploy your own if you'd like you'll find all the links to that in the description if you enjoyed the video come and say hi and uh you know show us what you're working on in the cloudflare workers discord which i'll put a link to in the description as well um it's a really cool community that's growing really quickly there's a lot of people in there talking about workers and what they're building uh so thanks so much hope you enjoyed the video and see you in the next one

Channel: Cloudflare Workers

Views: 1,709

Rating: 5 out of 5

Keywords:

Id: m3uhFpGOTB8

Length: 29min 31sec (1771 seconds)

Published: Mon Jan 25 2021