PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchTo get a working local Apache Spark 4.2.0 on Ubuntu, install a Java runtime that Spark supports, download a pre-built Spark archive from Apache, extract it, and start a bundled shell in local mode. That setup runs on one machine and needs no cluster. If you mean a multi-machine deployment, the cluster section below covers Spark Standalone and marks where YARN and Kubernetes begin.
Before you start: versions and requirements
Spark 4.2.0 is the release this guide uses, and Apache lists it as released July 14, 2026. Check Apache’s downloads page for the current release before you pin version numbers, and substitute the exact filename it shows for your package.
As an Amazon Associate I earn from qualifying purchases.
- Java: Spark 4.2.0 runs on Java 17, 21, or 25. Java 25 builds before 25.0.3 are deprecated, so use 25.0.3 or later if you choose Java 25, or pick 17 or 21.
- Python: Python 3.10 or later, if you plan to use PySpark.
- Scala: Spark 4 is built with Scala 2.13, and Scala 2.12 support has been dropped. Match your Scala application to 2.13.
- Ubuntu release and architecture: Java package names differ between Ubuntu releases. Run
lsb_release -aanddpkg --print-architecturebefore installing anything.
Step 1: Install and point JAVA_HOME at a supported JDK
- Check whether Java is already present with
java -version. If the version is outside the list above, install a supported one alongside it. - Refresh package lists with
sudo apt update, then search for the OpenJDK packages your release offers:apt-cache search openjdk. - Install the JDK 17 or 21 package your release lists. On releases that package it, the name is typically
openjdk-21-jdk-headless. Use the name the search returned, not a guessed one. - Set
JAVA_HOMEto the installed JDK so Spark does not depend on whicheverjavacomes first on PATH:export JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))). Add that line to~/.bashrc, open a new terminal, and runjava -versionagain.
If your release does not package a JDK in the supported range, use an official OpenJDK build, unpack it under your home directory, and point JAVA_HOME at it.
Step 2: Download Spark and verify the archive
On Apache’s downloads page, choose release 4.2.0 and a package type. A pre-built package for a Hadoop version is the simpler choice for a local install. The Hadoop-free build suits environments that supply their own Hadoop jars. Download the archive together with its signature and checksum files from the same page.
#1 Best Overall
Apache recommends verifying the release against its published KEYS file before you use it. Import the keys and check the signature, using the filename you downloaded:
- Import the KEYS file:
gpg --import KEYS - Verify the signature:
gpg --verify spark-4.2.0-bin-hadoop3.tgz.asc spark-4.2.0-bin-hadoop3.tgz. Replace the example name with the package you chose.
Follow the checksum step described on the same page if it is provided. Do not extract an archive that fails verification.
Step 3: Extract Spark and set SPARK_HOME
Extract the archive into a directory you manage, then point SPARK_HOME at it and add its bin directory to PATH:
Recommended Free Tools
mkdir -p ~/opt && tar -xzf spark-4.2.0-bin-hadoop3.tgz -C ~/opt- Add these lines to
~/.bashrc, adjusting the directory to match what was extracted:export SPARK_HOME=$HOME/opt/spark-4.2.0-bin-hadoop3export PATH=$SPARK_HOME/bin:$PATH - Reload the file with
source ~/.bashrc.
Step 4: Run Spark locally
Local mode needs Java on PATH or JAVA_HOME set, and no Hadoop cluster. Start from the extracted directory and run the bundled shells with --master "local[2]". The value local uses one worker thread, and local[N] uses N threads:
cd $SPARK_HOME- Scala shell:
./bin/spark-shell --master "local[2]". Ascala>prompt means the shell started and created its Spark session. - Python shell:
./bin/pyspark --master "local[2]", which opens a PySpark session. - Bundled example:
./bin/spark-submit examples/src/main/python/pi.py 10. The job prints an estimate of pi and exits.
Apache’s documentation presents these commands as examples. If one fails, the Java check in Step 1 and the PATH setting in Step 3 are the first things to recheck.
Alternative install paths
- PyPI:
pip install pysparkinstalls PySpark from the Python package index. Java is still required at runtime. - Docker: Apache publishes Docker images. Use them when your project already runs in containers, and check your project’s needs before choosing this route.
Choose a deployment: local, Standalone, YARN, or Kubernetes
Local mode is the path above. “Install Spark on a cluster” means choosing one of the cluster managers Spark supports. The table compares the four routes.
Rank #4
| Route | Use it when | Trade-off |
|---|---|---|
Local mode (local or local[N]) |
Learning, development, and testing on one machine | Work runs on a single host; no distributed workers |
| Spark Standalone | You want a Spark-managed cluster on your own machines without another resource manager | You manage start scripts, ports, SSH access, and security yourself |
| YARN | You already run a Hadoop YARN cluster | Setup depends on your Hadoop environment; follow Apache’s YARN deployment guide |
| Kubernetes | You already run workloads on a Kubernetes platform | Requires cluster and container configuration; follow Apache’s Kubernetes deployment guide |
Spark Standalone: set up a multi-node cluster
Standalone is Apache’s simple deployment mode. It can run on one machine for testing or across several nodes. Every node needs the same Spark distribution and a compatible Java runtime, so extract the archive to the same path on each host.
Start the master
- On the master host, run
./sbin/start-master.shfrom$SPARK_HOME. - Note the
spark://HOST:PORTURL the master prints. The default service port is 7077. - Open the master web UI, which listens on port 8080 by default, and confirm it loads.
Start the workers
- On each worker host, run
./sbin/start-worker.sh spark://HOST:7077, replacing the host with the master address you noted. - Return to the master web UI and confirm each worker appears. If it does not, recheck the master URL and port.
Launch workers across hosts with conf/workers
To start all workers from the master, list worker hostnames in conf/workers, one per line. The launch scripts reach each worker over SSH, which needs key-based, passwordless access from the master. Then run ./sbin/start-all.sh on the master.
Best Value
Submit a job to the cluster
Point spark-submit at the master URL, for example ./bin/spark-submit --master spark://HOST:7077 followed by your application file. Use a Spark and Java version that matches the cluster.
Security before you expose a cluster
Do not treat an open Spark master or web UI as a safe default. Apache’s standalone guide states the point directly:
“Security features like authentication are not enabled by default.”
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.Apache Software Foundation, Spark Standalone Mode, Spark 4.2.0 documentation.
Quick Recap
Bestseller No. 1SaleBestseller No. 3
- Limit access to the master and worker ports to hosts that need them, and keep the cluster on a trusted private network.
- Enable RPC authentication with
spark.authenticateand follow the deployment security guide before you use an untrusted network. - For encryption, Apache’s security guide prefers TLS-based RPC encryption. It requires keys and certificates to be configured first.
Common problems and fixes
- Shell fails to start or reports the wrong Java: run
echo $JAVA_HOMEandjava -version. Confirm the JDK is 17, 21, or 25.0.3 or later, and open a new terminal after editing~/.bashrc. spark-shellnot found: confirmSPARK_HOMEpoints to the extracted directory and that$SPARK_HOME/binis on PATH.- Worker missing from the master UI: check that the master URL matches the host and port the master printed, that port 7077 is reachable from the worker, and that the worker can resolve the master hostname.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
