My Oracle Support Banner

Hive Jobs Fail with "[uber-SubtaskRunner] org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator: Native-Task doesn't support secure shuffle" (Doc ID 2459600.1)

Last updated on OCTOBER 12, 2018

Applies to:

Big Data Appliance Integrated Software - Version 4.10.0 and later
Linux x86-64

Symptoms

Hive jobs start failing with:

java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.
  at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
  at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:134)
...


The failing Hive jobs have errors in the YARN Application logs which look like:

ERROR [uber-SubtaskRunner]
org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator: Native-Task doesn't support secure shuffle
WARN [uber-SubtaskRunner] org.apache.hadoop.mapred.MapTask: Unable to initialize
MapOutputCollector org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator (1 more collector(s) to try)
org.apache.hadoop.mapred.InvalidJobConfException: Native-Task doesn't support secure shuffle

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.