How to download install and setup/configure Apache Spark on Windows 10/11 | Updated 2024

hello guys in this video I will show you how to download install and configure or set up Apache Spock on Windows 11 operating system the process is same for all Windows operating systems steps involved in Apachi Spar installation on windows are first we have to download or install Windra or 7 G or any other software to unip the jip files okay second we have to install Java 8 later version then we have to download and install python then we have to download Apache SP then we have to extract then we have to download Hardo in itals after that we have to create folders and last step we have to set the environment variables then we will see how to connect to Apachi Spar okay so first step is download the windr go to wiy rare.com this is the official website to download the windr click on this download then click on windr English 64bit see download has been started go to downloads see Winer has been downloaded successfully now install it by clicking on this exe file click on install click on Okay click on done okay close this one now next step is is we have to download and install Java 8 or later versions okay now go to google.com then type Java jdk click on Java downloads Oracle the first link whatever the links are the commments I'm going to use in this video are available in the video description see here the latest I'm going to download Java jdk 17 click on it then click on Windows then see here x64 installer click on this bin EXE the size is 154 megabytes wait until this download completes see download is completed again click on this jdk windows 64 bin. exe file click on next see it is going to install in this path if you want to change you have to click on change and you have to select the path click on next see Java has been installed successfully click on close okay next step is python okay go to python.org click on downloads see here the latest stable version is python 3.12 okay if you want to download old versions you have to click on downloads then go to see here we have the wer versions but the latest python version is fine for Apachi Spar click on this download python 3.12 again go to downloads exercises only wait until download completes see python download completed click on python exe file choose the checkbox that is ADD python exe to path then click on install now we have successfully installed python also click on close now next step is we have to download Apachi Spar okay so go to go to Apachi spark website that is spark. Apachi .or click on download see here the version at the time of making this video is the 3.50 is the latest one select this one or from the drop down if you want specific of any of these versions you have to select then choose a package type I'm going with pre-build for Apachi hard 3.3 or later okay now click on after selecting these types then you have to click on spark 3.5 Gip file click on it click on any one of the download links I'm going with this link or you can click on this both are same so I'm going with this one go to downloads see download has been started go to downloads see here it's a is 382 MB wait until this download completes now go to downloads directory that is sh folder right click on it wi rare then extract to this path so it is going to download here only again it contains one more directory with the same name see extraction is completed see here we have these files next step is we have to download how do when it exe file we can get that from this link see here we have the latest 3.30 okay how downloaded hard 3.3 are later so so I'm going to download this one but if you click on it then click on download it is not downloading so it is always better to download this entire Gip file click on select it I will keep this link also in the video description then click on code then click on download jip the size is only 27 mbes see here extract this one also right click and Windra extract to this one okay see we have extracted spark as well as vils now the next step is we have to create folders for spark and how bin okay after installation and extraction of the files next step is we have to create folders in C drive or any Drive okay any Drive is fine now go to C drive I'm creating folder and giving the name as spark hit enter Then double click on it or open it then go to downloads directory copy all these files from bin to this release release file copy all these files right click and click on copy or contr C then go to the directory which we have created that is spark and paste them next go to C drive again create new directory name it as Hardo go to this directory again create one more directory I'm giving name as give the name bin hit enter again go to this bin directory now go to downloads directory go to Wi NS again wi n Go to 3.3 go to bin and copy only this win.exe file simply select it and control C then go to this howo bin directory and paste here click on this paste button on Windows 11 or control V okay so we have created these folders and copied the extracted files to these locations now next we have to set up the environment variables for Java spark and hard now open environment variables start then type our environment see we have the edit the system environment variables click on it this is the one way other way to open environment variables is right click on the PC then click on properties then click on Advanced system settings then click on environment variables in the user variables section click on new the variable name give the Java uncore home in the variable value we have to give the downloaded directory go to C drive program files Java copy this path then paste here click on okay the same way create two more directories for spark and hard okay variable name is Hardcore Home pa is C drive Hardo copy it paste here click on okay create new variable for spark spark uncore home then variable value go to C drive spark copy the path paste here click on Okay now click on path click on edit click on new we have to give this one by one I will give I will keep all this in the video description click on new simply type or paste them Java _ home C then forward slash bin then click on new copy this one new paste here click on New Again copy this one paste here click on okay I hav't done nothing just I have given this home then slash Java home how home then spark home then I have given at the end slash and B Okay click on Okay click on Okay click on okay so we have done or we have set everything now let's open command prom CMD verify now give the command Java see we have installed Java verify the Java version see I have installed Java 17 okay CLS to clear the screen now run the spark hyphen shell if it connects to Apachi Spar which means we have successfully configured the Apachi Spar okay hit enter see we are successfully connecting to the apis Spock we got the Scala shell and we have installed Apachi Spock 3.5.0 okay see here Java version is 17 you want to verify just give any command print welcome to AR schools YouTube channel if this is helpful subscribe my channel also share see we got the output you want to quit from this console you colon quit see we left from the spark shell this spark is for Scala but if you want to go for Apachi is p spark just give P spark I'm going to clear the screen P spark now we will get the python shell see here we got the python 3.12 we have also installed Python 3 see we have installed python 3.12 here also we got the same python 3.12 shell okay so in this video we have seen how to download extract and install or setup or configure the Apache Spock on Windows 11 operating system the process is same for all the op all the Windows operating systems then we have seen how to connect to spar shell for Scala or connected to P spark for more Apachi or Apache spark for scalar python videos for more please subscribe my channel thank you

Channel: r2schools

Views: 5,560

Keywords: Apache Spark setup on Windows 11, Apache Spark setup on Windows, Apache Spark for Scala setup on Windows, Apache Spark for Scala setup on Windows 11, Apache Pyspark setup on Windows 11, Apache Pyspark setup on Windows, Apache Spark Installation on Windows, Apache Spark Installation on Windows 11, Apache Spark 3.5 Installation on Windows, Apache Spark 3.4 Installation on Windows, Apache Spark 3.3 Installation on Windows, Apache Spark 3 Installation on Windows, Apache Spark Tutorials

Id: IOckqej11HM

Length: 15min 2sec (902 seconds)

Published: Sun Dec 10 2023