Download Apache Spark™ Choose a Spark release: Choose a package type: Download Spark: Verify this release using the and project release KEYS. Note that, Spark 2.x is pre-built with Scala 2.11 except version 2.4.2, which is pre-built with Scala 2.12. Spark 3.0+ is pre-built with Scala 2.12. C: Spark bin spark-submit -class org.apache.spark.examples.SparkPi -master local C: Spark lib spark-examples.jar 10 If the installation was successful, you should see something similar to the following result shown in Figure 3.3.
- Apache Spark Tutorial
- Apache Spark Useful Resources
- Selected Reading
Spark is Hadoop’s sub-project. Therefore, it is better to install Spark into a Linux based system. The following steps show how to install Apache Spark.
Step 1: Verifying Java Installation
Java installation is one of the mandatory things in installing Spark. Try the following command to verify the JAVA version.
If Java is already, installed on your system, you get to see the following response −
In case you do not have Java installed on your system, then Install Java before proceeding to next step.
Step 2: Verifying Scala installation
You should Scala language to implement Spark. So let us verify Scala installation using following command.
If Scala is already installed on your system, you get to see the following response −
How To Install Apache Spark
In case you don’t have Scala installed on your system, then proceed to next step for Scala installation.
Step 3: Downloading Scala
Download the latest version of Scala by visit the following link Download Scala. For this tutorial, we are using scala-2.11.6 version. After downloading, you will find the Scala tar file in the download folder.
Step 4: Installing Scala
Follow the below given steps for installing Scala.
Extract the Scala tar file
Type the following command for extracting the Scala tar file.
Move Scala software files
Use the following commands for moving the Scala software files, to respective directory (/usr/local/scala).
Set PATH for Scala
Use the following command for setting PATH for Scala.
Verifying Scala Installation
After installation, it is better to verify it. Use the following command for verifying Scala installation.
If Scala is already installed on your system, you get to see the following response −
Step 5: Downloading Apache Spark
Download the latest version of Spark by visiting the following link Download Spark. For this tutorial, we are using spark-1.3.1-bin-hadoop2.6 version. After downloading it, you will find the Spark tar file in the download folder.
Step 6: Installing Spark
Follow the steps given below for installing Spark.
Download Apache Spark For Windows
Extracting Spark tar
The following command for extracting the spark tar file.
Download and Install Malwarebytes If you are on macOS 10.10 Yosemite, download version 3 of Malwarebytes for Mac. If you have a newer version of macOS, instead see Download and install Malwarebytes for Mac. Locate the downloaded file Malwarebytes-Mac-3.x.y.zzz.pkg. Download the latest version of Malwarebytes for Mac. Locate the downloaded file Malwarebytes-Mac-4.x.y.zzz.pkg. 4.x.y.zzz represents the version number. In most cases, downloaded files are saved in the Downloads folder. Download malware mac. Download Malwarebytes for Mac (the free version) and you get a 14-day trial of the premium version with automatic (real-time) virus and malware protection. After 14 days, your trial reverts to a limited disinfection scanner. Buy the premium version now to prevent infection in the first place.
Moving Spark software files
The following commands for moving the Spark software files to respective directory (/usr/local/spark).
Setting up the environment for Spark
Add the following line to ~/.bashrc file. It means adding the location, where the spark software file are located to the PATH variable.
Use the following command for sourcing the ~/.bashrc file.
Step 7: Verifying the Spark Installation
Write the following command for opening Spark shell.
If spark is installed successfully then you will find the following output.