Apache Spark - Install Apache Spark 3.x On Ubuntu |Spark Tutorial

hi friends welcome to another video of big Tech talk in today's video I will try to set up spark 3.5.0 on umuntu operating system so let's start the video I will set up spark in four simple Steps step one is to install Java on Ubuntu operating system step two is to download the spark package from its official website step three is setting up the environment variable and step four is verification of a spark installation so let's install Java on our system let's check if Java is installed on our system so type Java hyphen version okay looks like Java is not installed on my system so to install Java I will write Pudo AP install default jdk I will provide sudo password and then why to install the packages it will install Java looks like Java is installed let's check the Java version installed so Java hyphen version and we have installed open jdk version 11 let's go to step two that is download the spark package open your favorite browser and search for spark download click the URL over here and it will take you to the official website of spark now select your spark version so for this video I will select spark 3.5.0 and in the package type select pre-build for Apachi Hadoop 3.3 and later and click on spark 3.5.0 bin Hadoop 3. dgz it will take you to a different page where you can download so let me copy the URL and in the terminal I will create a directory where I will download the spark package so I will switch to root user by sud sudo Su so to create a directory I will type mkdir SL opt SLS spark now in this directory I will download my spark package so I will write w get and the URL once the download is completed untar the file using tar hyphen xvf and the package name once it is done we will move to step step three that is setting up the spark home so to add spark home in the path variable we need to make changes in The Bash RC file so go to home folder and type vi. bashrc and scroll till the end of the file and hit I for insert and then type Spar uncore home equals SL opt SL spark SL spark 3.5.0 bin Hardo 3 and in the next line just type export path equals to Dollar path dollar spark home/ bin and again the spark home slash bin so by doing this I have set The Spar CH in the path variable now save the file and Source it so that variables can be used in this environment so Source do bashrc once it's done let's move to step four that is verification of the installation so in the terminal type Spar hyphen shell and if everything is okay we will see a spark console okay so spark console is up and running and we have Spar 3.5.0 now let's try to execute some Spar command so I will try to read a CSV file which is in my downloads folder that is mock data.csv so in the terminal I will type Val DF equals Spar do read. format and the format is CSV do option where header is true do load and the path of the CFC file that is home work PC downloads SL Mock data.csv and and hit enter and then DF do show to show the output so this is the expected output and looks like our spark installation is working fine so friends we are done with setting up spark 3.5.0 on Ubuntu operating system if you have any question let me know in the comment section do hit the like button and subscribe for more such video thank you

Channel: Big Tech Talk

Views: 3,699

Keywords: Apache Spark, Spark, install, Setup, Install on Ubuntu, Install Apache Spark On Ubuntu, How To Install Apache Spark On Ubuntu, SparkInstallationUbuntu, HowToInstallSpark, ApacheSparkTutorial, Spark install linux, ubuntu, install spark on ubuntu, spark3.x

Id: ei_d4v9c2iA

Length: 6min 34sec (394 seconds)

Published: Mon Mar 04 2024