My Oracle Support Banner

Launching a spark2-submit Job from an OdiOSCommand Behaves Differently when Executed Using Local / No Agent or a Standalone Agent (Doc ID 2542116.1)

Last updated on MAY 24, 2019

Applies to:

Oracle Data Integrator - Version 12.2.1.3.0 and later
Information in this document applies to any platform.

Symptoms

An Oracle Data Integrator (ODI) 12.2.1.3 Package executes the OdiOSCommand to launch a spark2-submit job which is executing a custom jar and some operations in the Hive databases:

spark2-submit --master local[*] --conf spark.debug.maxToStringFields=50 --driver-memory 3g --executor-memory 10g --num-executors 10 --executor-cores 6 --principal <PRINCIPAL_NAME> --keytab <KEYTAB_NAME> --class <JAVA_CLASS_NAME> <CUSTOM_JAR_NAME> <JAR_PARAMETERS> > <JAR_OUTPUT>.log

When running the Package with ODI Local / No Agent,  the jar output log contains the expected information, and all the Hive databases can be accessed.

When running the Package with a Standalone Agent (located on the same server with ODI Studio, and started with the same OS user), the jar output log shows that the Hive information is not loaded correctly and does not connect to the proper Hive metastore. It is only showing a default database and the error message below:

Exception in thread "main" org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database '<HIVE DB NAME>' not found;
   at org.apache.spark.sql.catalyst.catalog.SessionCatalog.org$apache$spark$sql$catalyst$catalog$SessionCatalog$$requireDbExists(SessionCatalog.scala:174)
   at org.apache.spark.sql.catalyst.catalog.SessionCatalog.listTables(SessionCatalog.scala:727)
   at org.apache.spark.sql.catalyst.catalog.SessionCatalog.listTables(SessionCatalog.scala:712)
   at org.apache.spark.sql.execution.command.ShowTablesCommand$$anonfun$17.apply(tables.scala:760)
   at org.apache.spark.sql.execution.command.ShowTablesCommand$$anonfun$17.apply(tables.scala:760)
   at scala.Option.getOrElse(Option.scala:121)
   at org.apache.spark.sql.execution.command.ShowTablesCommand.run(tables.scala:760)
   at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
   at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
   at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
   at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
   at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:190)
   at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3253)
   at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
   at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3252)
   at org.apache.spark.sql.Dataset.<init>(Dataset.scala:190)
   at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:75)
   at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:638)
   at sparkconsolidation.consolidation$.main(consolidacion.scala:146)
   at sparkconsolidation.consolidation.main(consolidacion.scala)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
   at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:892)
   at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

The security settings and Kerberos ticket are the same in both cases.

Changes

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.