How to Retain the spark2-submit Syntax for Spark 2.4 used in BDA 4.x Post Upgrade to BDA 5.2
(Doc ID 2895742.1)
Last updated on APRIL 17, 2023
Applies to:Big Data Appliance Integrated Software - Version 5.1.0 and later
Provide options for retaining the spark2-submit syntax for Spark 2.4 used in BDA 4.x after upgrade to BDA 5.x.
For example when upgrading from BDA 4.13/CDH 5.15 to BDA 5.2/CDH 6.3.4 both with Spark 2.4:
- In BDA 4.13/CDH 5.15 Spark 2.4 uses a spark2-submit syntax for spark job submission.
- In BDA 5.2/CDH 6.3.4 Spark 2.4 uses a spark-submit syntax for spark job submission.
The steps in this note provide a way for allowing Spark 2.4 jobs to retain their spark submit commands without change post upgrade to BDA 5.x.
To view full details, sign in with your My Oracle Support account.
Don't have a My Oracle Support account? Click to get started!
In this Document
|Options to retain the spark2-submit Spark 2.x submit syntax post upgrade to BDA 5.x|