Apache Spark - Install Apache Spark 3.x On Windows 10 |Spark Tutorial

hi friends welcome to another video of big Tech talk in today's video we will learn how to set up Spar 3.x on Windows operating system so let's start the video for the installation we need Java spark package and V utils I assume that Java is already installed on your system as it's very easy to install and most of the people have already PR installed it just to verify if Java is installed on your system go to command prompt and type Java hyphen version ideally you should see the output with Java version if no then go and install Java 11 on your operating system now let's download Spar so open your favorite browser and search for spark download at the time of recording this video we have spark 3.5.0 as the latest version so click on spark 3.5.0 bin Hadoop 3. dgz and then click on the very first link it will start the download let's wait for the download to finish okay my download is completed let's go to download folder and unzip it right click on spark 3.5.0 and 7zip and extract to spar 3.5.0 bhadu now open the extracted folder and you will see a tar file so right click 7zip and extract to spark 3.5.0 bin hadu these are the actual spark files let me go back and rename this folder to spar 3.5.0 now I will copy this folder under D Big Data local setup SL spark local setup this is the place where I usually set up my spark you can choose your own folder now we will set the system environment variable by typing environment in Windows search click on system environment variable and then go to environment variable I will create a spark chome variable over here so click on new and give the variable name as spark unor home and the path is D Big Data local setup SL spark local setup SL spark 3.5.0 and then click on okay now open path variable and add percentage spark uncore home percentage /bin and then click on okay to save the changes so we have set up spark home in environment variable now it's time to download wi utils which is Windows boundary for Hado for download go to this path and click on Hadoop 3.0.0 bin scroll down and you will see vu. exe click on it and then download this file by clicking on download icon over here now let me go to download folder and copy the V utils and I will place it under D Big Data local setup slh Hadoop /bin now we have to add Hadoop home in environment variable click on new and enter the variable name as Hadoop unor home and the value is d/ Big Data local setup SLO now let's add Hadoop home in the path variable so open path variable percentage Hado uncore home percentage /bin click on okay to save the changes and our spark setup is completed just to verify if everything is working fine open command prompt and type spark shell you should see a spark console we are running spark 3.5.0 and in master as local now let's try to read a CSV file and see so I will write Val DF equals Spar do read do format which is CSV and then in option header is true do load I will mention the path which is D sample data input and country. CSV and to show the DF I will write DF do show and here is the output so our spark setup works as expected so we have successfully installed spark 3.5.0 on Windows system so friends we have reached to end of our today's video if you have any question let me know in the comment section do hit the like button and subscribe for more such video thank you

Channel: Big Tech Talk

Views: 14,883

Keywords: Apache Spark, Spark, Windows10, install, Setup, Install on windows, Install Apache Spark On Windows, How To Install Apache Spark On Windows 10, SparkInstallationWindows, HowToInstallSpark, ApacheSparkTutorial, big data tutorial, big data, apache hadoop, spark tutorial, spark installation windows, spark3, spark3 setup, spark on windows, how to install spark on windows, how to install spark 3 on windows, wintils, hadoop liberary for windows, parallel processing

Id: FIXanNPvBXM

Length: 6min 28sec (388 seconds)

Published: Mon Nov 06 2023