My Oracle Support Banner

On BDA 4.9 Spark Applications Fail with "org.codehaus.janino.JaninoRuntimeException: Constant pool for class ... has grown past JVM limit of 0xFFFF" (Doc ID 2542832.1)

Last updated on NOVEMBER 08, 2019

Applies to:

Big Data Appliance Integrated Software - Version 4.9.0 and later
Linux x86-64

Symptoms

On BDA 4.9 with CDH 5.11.1/CM 5.11.1 Spark applications fail with:

<TIMESTAMP> INFO scheduler.DAGScheduler: ResultStage 1 (checkpoint at CheckpointUtil.scala:48) failed in ##.### s due to Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task ## in stage 1.0 (TID <TID>, <HOSTNAME>.<DOMAINNAME>, executor 1): java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.codehaus.janino.JaninoRuntimeException: Constant pool for class org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection has grown past JVM limit of 0xFFFF

The error stack causing the task failure is:

Caused by: org.codehaus.janino.JaninoRuntimeException: Constant pool for class
org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection has grown past JVM limit of 0xFFFF
   at org.codehaus.janino.util.ClassFile.addToConstantPool(ClassFile.java:499)
   at org.codehaus.janino.util.ClassFile.addConstantNameAndTypeInfo(ClassFile.java:439)
   at org.codehaus.janino.util.ClassFile.addConstantFieldrefInfo(ClassFile.java:344)
   at org.codehaus.janino.UnitCompiler.writeConstantFieldrefInfo(UnitCompiler.java:11109)
   at org.codehaus.janino.UnitCompiler.getfield(UnitCompiler.java:10778)
   at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3829)
   at org.codehaus.janino.UnitCompiler.access$8800(UnitCompiler.java:206)
   at org.codehaus.janino.UnitCompiler$12.visitFieldAccess(UnitCompiler.java:3793)
   at org.codehaus.janino.UnitCompiler$12.visitFieldAccess(UnitCompiler.java:3762)

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.