Spark Job on BDA Edge Node Fails to Run and Gives Message "Only local additional python files are supported"
(Doc ID 2535178.1)
Last updated on JULY 20, 2024
Applies to:
Big Data Appliance Integrated Software - Version 4.11.0 and laterInformation in this document applies to any platform.
Goal
A spark version 2.2 job on an edge node of a BDA 4.11 / CDH 5.13.1 cluster is being run with the job command line below:
When trying to run with this syntax, an error is raised in the spark shell and the job does not run. The error is:
"Only local additional python files are supported"
It seems that the job fails due to the reference "hdfs:/<FQ_HOSTNAME>/wbdata/artif/lib/python/<FILE>.egg."
If the egg file is place in the local (edge node) HDFS filesystem, it runs fine.
The goal is to determine if there is any other workaround to this issue.
Solution
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Goal |
| Solution |