My Oracle Support Banner

Oracle Loader For Hadoop Job Fails With Java.lang.OutOfMemoryError: GC Overhead Limit Exceeded (Doc ID 1966732.1)

Last updated on NOVEMBER 26, 2019

Applies to:

Oracle Loader for Hadoop - Version 3.2.0 and later
Information in this document applies to any platform.

Symptoms

While using OLH connector to create datapump files inside HDFS for Hive tables. the OLH job fails with following error:

java.lang.OutOfMemoryError: GC overhead limit exceeded
  at java.util.Arrays.copyOf(Arrays.java:2367)
  at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
  at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
  at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
  at java.lang.StringBuilder.append(StringBuilder.java:132)
  at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDDLFromFieldSchema(MetaStoreUtils.java:494)
  at org.apache.hadoop.hive.metastore.MetaStoreUtils.getSchema(MetaStoreUtils.java:711)
  at oracle.hadoop.loader.lib.input.HiveInputSplit.(HiveInputSplit.java:166)
  at oracle.hadoop.loader.lib.input.HiveInputSplit.(HiveInputSplit.java:232)
  at oracle.hadoop.loader.lib.input.HiveInputSplit.(HiveInputSplit.java:253)
  at oracle.hadoop.loader.lib.input.HiveToAvroInputFormat.addSplits(HiveToAvroInputFormat.java:630)
  at oracle.hadoop.loader.lib.input.HiveToAvroInputFormat.getSplits(HiveToAvroInputFormat.java:427)
  at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1079)
  at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1096)
  at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:177)
  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:995)
  at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:948)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
  at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:948)

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.