BDD: After Patching To BDA 4.4, BDD 1.0 Data Processing Command Fails With Spark On Yarn Error' Could not find or load main class org.apache.spark.deploy.yarn.ApplicationMaster" (Doc ID 2136359.1)

Last updated on JUNE 05, 2016

Applies to:

Oracle Big Data Discovery - Version 1.1.0.0.0 and later
Big Data Appliance Integrated Software - Version 4.4.0 and later
Information in this document applies to any platform.

Symptoms

Running a BDD command-line data pro

For example:

data_processing_CLI -d dwh -t [[hivetable]



Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/bdd/edp-1.1.0.13.49/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/bdd/BDD-1.1.0.13.49/common/hadoop/lib/bddHadoopFatJar.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Unknown/unsupported param List(--num-executors, 1)....

 

yarn logs

Error: Could not find or load main class org.apache.spark.deploy.yarn.ApplicationMaster

 

 

Changes

Updated the Big Data Appliance (BDA) to BDA 4.4 which includes Cloudera (CDH 5.5.1)

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms