How to install and configure Apache PySpark 3.5.1 on Ubuntu 24.04 |Apache PySpark Install Linux 2024

hello guys in this video we will see how to install and configure PES ppar 3.5.1 on2 24.0 for LS version The Pri requisites are Java and python python version should be the greater than three so by default python comes with all operating systems but anyway verify again open Terminal type python hit enter okay it is Python 3 you the Python 3 and hit enter see we have the Python 3 it is the latest version okay so and next verify the Java version see I have already installed Java version but if you want to install Java version you can go to the link provided in the video description and install it crl C to quit verify the Java version see I have installed open jdk 17 see I will keep the link how to download and install Java jdk on open2 even if you don't know just open Firefox or any browser then just search for Java download click on this Java downloads I will keep this link in the video description then you need to download this debn package see right click on this debn package copy the link address then go to the terminal then paste it and hit enter see this is the way to download after that you have to give sudo dpkg hph I then copy this Dean package if I copy this deian package and paste it hit enter that's all with these three lines we will install the Java JD okay control to clear screen next step is we have to go to the spark official website that is apach spark. Apachi doc then click on download see here we have the latest version 3.5.1 and the older one is 3.4.3 okay both are the stable version so latest so select the latest one and choose a package type prebuild for apach 3.3 or later okay and later so click on this spark 3.5 1 bin Hard 3 g f click on it to download now click on this link or any of the links to download so click on it see download started and it size is 382 MB wait until download completes see download is completed now go to the terminal terminal go to downloads LS see here we have the spark G file okay so we have to un this file T hyphen XV ZF then give the G file give the spa and give the tab to auto fill and hit enter see extraction is going on LS see you have the spark extracted folder so re name it by running MV spark to just give the spark I don't want this much bigger name so it will rename to spark LS see name has been changed to spark okay now we have to set the spark variable so give the control n PWD CD spark PWD we need this path to set the spark uncore home my so VI then go to the bash profile bash RC you this sudo and it enter I will keep these commments in the video description it enter go to the bottom shift a export type these commands I will keep this also in the video description spark uncore home equals to home R2 schools let me check the path again PWD see it is just download directory you can place this Spar somewhere else but for the demo purpose I'm going to use the same path copy it and here we have to give this entire part not remove this one and give the path so paste that path hit enter now export path equals to Dollar path colon dollar spark uncore home then you the bin this small case you the bin and Escape colon WQ exclamation which means it is going to save the path and hit enter then run the source source The Bash profile bash RC and hit enter see we have successfully set it I just added last two lines that are shown that are given in the video description then we have Source The Bash profile so we have successfully set the or set or configured the F spark or P spark on 1 12404 now verify the installation and configuration success or Not by connecting to spark shell spark hyphen shell you will hit enter see we have successfully connected and this is my local IP see you have successfully connected to Scala see this is the Scala shell and this is the version of the spot if you go to this URL that is copy it go to the new tab paste it hit enter see this is spark but if you want P Spar control Z to quit now give the p contrl l to clear screen p SP and hit see see this is a web UI for P Spot copy it new tab paste it hit enter see by spark shell but here it is spark shell which means scal shell okay so also see here this is the version of the by spark or spark okay so in this video we have seen how to install and configure the P spark or spark 3.5.1 on W 24.0 for L version for more P spark or Scala or buntu or Linux videos please subscribe my channel thank you

Channel: r2schools

Views: 145

Keywords: Apache PySpark setup on Ubuntu 24, Apache Spark setup on Ubuntu, Apache Spark for Scala setup on Ubuntu, Apache Spark for Scala setup on Ubuntu 24, Apache Pyspark setup on Ubuntu 24, Apache Pyspark setup on Ubuntu, Apache Spark Installation on Ubuntu, Apache Spark Installation on Ubuntu 24, Apache PySpark 3.5.1 Installation on Ubuntu, Apache Spark 3.4 Installation on Ubuntu, Apache Spark 3.3 Installation on Ubuntu, Apache Spark 3 Installation on Ubuntu, Apache Spark Tutorials, scala

Id: dGkhnTZcjLo

Length: 8min 13sec (493 seconds)

Published: Thu May 02 2024