How to Retain the spark2-submit Syntax for Spark 2.4 used in BDA 4.x Post Upgrade to BDA 5.2 (Doc ID 2895742.1)

Last updated on JULY 20, 2024

Applies to:

Big Data Appliance Integrated Software - Version 5.1.0 and later
Linux x86-64

Provide options for retaining the spark2-submit syntax for Spark 2.4 used in BDA 4.x after upgrade to BDA 5.x.

For example when upgrading from BDA 4.13/CDH 5.15 to BDA 5.2/CDH 6.3.4 both with Spark 2.4:

In BDA 4.13/CDH 5.15 Spark 2.4 uses a spark2-submit syntax for spark job submission.
In BDA 5.2/CDH 6.3.4 Spark 2.4 uses a spark-submit syntax for spark job submission.

The steps in this note provide a way for allowing Spark 2.4 jobs to retain their spark submit commands without change post upgrade to BDA 5.x.

	To view full details, sign in with your My Oracle Support account.
	Don't have a My Oracle Support account? Click to get started!

In this Document

Goal

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.