Installation of Apache Spark on Windows 11 (in 5 minutes)

hello everyone this is Amrit pal Singh welcome to the next video in this video I'll be teaching you how we can install Apaches spark on Windows machine Apaches Spar is a big data framework which can handle both streaming and Bash data and it can be downloaded very easily on the Windows machine but there are some requirements so what are the requirements for that I have uh listed it on my uh notepad file we need to have these three things okay with with us first Java should be available Java is a prequest for all the big data tools okay even for Hadoop and other tools it is required followed by python is important because Apaches spark can easily work in python as well as in a Scala so python is important we need to download python then we need to download the setup of a purchase Spark from internet followed by we require also two files from the internet which is Hado wi utils and had. dll which I'll be giving you the link then afterward we need to set the environmental variables followed by we need to test the py spark and Spark shell okay So currently I have opened up the simple uh my search engine and in which I have just written here spark download Windows okay this is the first link which I have to click and this is the link uh which we'll be uh going on okay this is the link on which you can find the recent version uh is a 3.5.0 so this is the one which I'm be downloading and it's written here it's pre-built for Hadoop 3.3 and later okay this is important so when we click on this link it'll be start downloading okay this I'm already having in my machine you can see this is available as my zip file this is the one which I got from internet afterwards what we require is we require the two files which I'll be getting from this link one is had. dll and win.exe which I'll be getting from this particular link I'll be providing you this link in my description box don't worry okay I need to keep these two files in my folder when utils I made one folder in my C drive under which I made a subfolder bin under which I have replaced these two files okay had. dll and Vin util so we need to place these two files in exactly in the same same way when utils you can just rename it with any other name as well hardly matters but you should be having bin uh in the sub folder under bin we have a two files available so we have done this step afterwards what we require is we need to have the python in my system the python is 3.2.1 the current version you can download this one and followed by we have to have we need to have a Java in our system so these are the prequest for this particular in installation so I'm having everything with me so let's now give the environmental variables I've already have opened up the environmental variables uh open up this page go to this dialogue this one environmental variables button and click on new and simply give here spark uncore home okay and because it's available in my C drive so let me uh go to my C drive and get the name copied here okay uh properties copy and give [Music] a path here we are done okay it's already been extracted from a zip file okay followed by you can see when utils I've already have given the path in my environmental variables in the C Drive WI utils okay afterwards click on the path and give a new entry click on new and percentage spark uncore home percentage slash bin done and you can see there's one more entry available when utils bin please remember we need to make two entries in in environmental variables one is utils one is spark undor home click on Okay click on okay we are done with the installation okay now we can check with the uh this Powershell opening with the uh administrator mode and go with the CD go to your location where the spark is available go to its bin okay go to its bin and simply type in spark hyphen Shell by default the spark open up in a Scala prompt okay par spark has been written in Scala language so if you type spark hyphen shell it will be opening up in the in the Scala prompt right this is the first way we can check you can see in no time you'll be getting up the spark logo it means it's it's perfectly opening up right now we can also check out the Sparks opening up with the python because a PES spark support Python language so again go to with the with the CD uh command go to the bin and simply type in py spark so py spark will open up in a python prompt so it's up to you if you're comfortable in a python you can code in a python if you're comfortable in the Scala you can code in a Scala you can see in no time you'll be getting up the uh Apache spark opening up in a python mode as well and currently using three 3.2.1 which is my uh current version which I've already have shown you it may take take some time here let's wait for a minute maybe so it'll be opening up the py spark for us you can see it's now opening up and let's we got it right so this marks the end of this video I hope you must have uh understood how we can install Apache spark on Windows machine for rest of the content I'll be covering up in the next video thanks for watching guys see you next video

Channel: Unboxing Big Data

Views: 10,950

Keywords: spark installation on windows 11, apache spark installation on windows 10, apache spark, spark installation on windows 10, how to install apache spark on windows 10, apache spark installation, spark installation windows, apache spark installation windows, pyspark installation on windows, apache spark installation on windows 7, hadoop installation on windows, how to install apache spark in windows 10

Id: wEEpULTvkZM

Length: 5min 31sec (331 seconds)

Published: Sun Dec 24 2023