Install Apache Airflow for Windows PC

hello and welcome back to the channel I hope you all are doing well so this video is all about setting up and installing Apache airflow in your local machine so without wasting any time let's get into it so if you don't know Apache airflow is pretty popular open source platform for scheduling and orchestrating the workflows it's designed to help you create schedule as well as monitor complex workflows which makes it a very vital tool in the field of data engineering so without wasting any time let's jump on to the installation part okay so there are some prerequisites you need to cover first for installing Apache airflow the first one is you have to install Docker desktop for Windows as well as get the vs code so I'm not going to explain how to install vs code in your PC it's a fairly simple process because it's just a code editor you can download it through their website and you will not have an issue but if you have it you can let me know in the comments and we can solve it together but for the docker installation let's get started with the docker if you don't have otherwise you can skip this step as well so this is their official website for downloading the docker desktop for Windows I'll be giving this to you in the description below so that you don't go into any hassle because that is my task to solve your problems so you have this friendly button for Docker desktop for Windows to able to download it so just click on it it's a fairly small file so as you can see it's only like 573 mbase so let it get completed and we'll start the installation right away okay so as you can see the download is completed so you have to just go to the download folder and here you can see you've got the docker desktop installer.exe file so just double click on it and it's a fairly simple procedure for setting up the docker desktop okay so it's initializing the docker desktop now now so here you have to just add shortcut to the desktop you have to click on OK and there you go it is unpacking all the files and setting up the docker desktop on our Windows machine so let it get completed just grab a coffee and come back once it is done then we'll continue the further installation okay so as you can see the installation has been succeeded so you just have to click on close and go to the desktop where the shortcut is created and open the docker so that it can get started and Boot It Up the docker engine and here just launch the docker desktop and you have to just click on accept and you can just continue without signing in because we don't need it you can click anything here so I'll just click that and there you go it is starting the docker engine right now so that we can instantiate our airflow on the vs code so we just need a simple yaml file which can kick it off our airflow on the Windows PC and we can reach it out using the browser to its open port it is fairly simple process we just need to just create a one file where we put all the configuration it's just two lines to only two configurations so that is also a simple step so let this engine keep running and there you go the engine has already started it totally depends upon your system performance mine is not that much good because it took like one two minutes to kick it off but whatever it is it got started finally so the next step would be you need to go into this link so I'll be giving this link to this yaml file file in the description below so just click on this and as you can see this is the dot yaml file we need to save in the separate folder but for now let's download it so just save it click save and there you go you can just click on to the dry you will not save but here is the tricky part you don't need this whole name you just need this Docker Dash compose dot yaml but as you can see it is a text document so you don't have to save it as a text because once you save it as a text it will shows like Docker Dash compose dot ml dot txt you don't need that if you do so just remove that DOT test extension by renaming that file but for me it looks good just save it up there you go you downloaded Docker Dash composer.tml that is the only file we need to kick off our Apache airflow okay so once you have this Docker Dash compose file you just have to copy that file and save it somewhere else so I'll just go to see the user the username and here let's create one folder so the folder name I think should be materials so here in the materials let's press this file that's that's all you have to do but we also have to create one dot EnV file where we will Define two configuration variables so that is our step so to do that we will go ahead and open the vs code and let's get started with that so I hope you already installed vs code in your system if not just do so it's a pretty simple process I'll open a vs code right now and here in here we need to go to file and open a folder and we need to open that materials folders that you have just created so just navigate quickly to that path and here we got the materials folder so just select it and there you go we got our yaml file present here so here we need to again create a new file so I'll just right click and create on new file the file name would be dot EnV you have to follow this naming convention because it may create an issue to during the setup once we do that it's empty file so just I will give this two lines which is airflow image name and the airflow user ID in the description below so that you can directly paste it up and just go ahead and save that file okay so once you paste it just save this file and that's it that's all you have to do now the next step would be you need to kick off your Docker Dash compose file to do that just go into the terminal and create a new terminal here and in here this is very simple command you have to just go like Docker Dash compose up and you have to pass the argument as Dash D that's it once you execute this as you can see it is pulling all the files it is pulling airflow in it the web server the scheduler worker as well as the triggers so this is pulling all these services and once it is done then you can just go ahead and listen to the port 8080 and I hope that should be open on your PC and you can just log into the airflow right away so just let it get completed and will continue once it is done and there you have it all your services are running fine without any issues since we got all the green lights here right so which means that we have successfully started our Apache airflow service so what are you waiting for just head back to the browser and listen to the localhost 8080 where you can reach Apache airflow service that's fairly simple so you just open a new tab and click on to the localhost 8080 so I'll just give it that and there you have it this is the login page for the airflow so we are almost there but there is one thing which comes in our way which is username and password so we don't have that yet created so we can first create like a admin privilege username and password and I have some work around I have a command which you can put to create a admin username as well as you can have any password and also it will have all the accesses so this is the command so here we are just creating a admin user as well as the admin password with a role of admin so just copy this and just paste it in the vs code and it will create the user for us so as you can see it is like running and creating our user right away so it will take one or two minutes so as you can see there you have it your admin is created with the role of admin so it will have all the Privileges for your Apache airflow service so once you have it just go ahead and just give like admin password so give admin and admin that's it just enter it and as you can see you will be able to log into the airflow so that's it that's all about it you have successfully set up the Apache airflow in your Windows PC so this was really simple and you can definitely do it on your own so if you face any difficulties during the installation or if you need some additional resources I'll be happy to help you so just let me know in the comments and we can discuss about it more so as you can see you've got all the Dax which is director cyclic grab the data set security browse so this is like the discussion for another lecture where we will cover all about Apache airflow and we can have like a series of 5 to 10 lectures so get you started airflow but for this lecture that's about it so I hope it was clear and you are successfully able to install Apache airflow in your Windows system if you enjoyed it just subscribe to the channel and also ring the notification Bell to get the latest updates thanks for watching

Channel: AmpCode

Views: 20,525

Keywords: install apache airflow for windows pc, apache airflow, airflow, apache airflow tutorial for beginners, install apache airflow, install airflow, apache airflow installation, apache airflow tutorial, apache airflow full course, airflow installation, big data, big data analytics, data analytics, data analyst, data engineer, data engineering, data engineer skills, data science, what is data science, ampcode, airflow tutorial, airflow tutorial for beginners, big data tutorial

Id: Fl64Y0p7rls

Length: 9min 34sec (574 seconds)

Published: Mon Oct 02 2023