Teragen-Terasort-Teravalidate Fails After 2.5 Upgrade Running "mammoth -c" on Oracle Big Data Appliance with LZO compression RPMS Installed

(Doc ID 1676583.1)

Last updated on MAY 30, 2014

Applies to:

Big Data Appliance Integrated Software - Version 2.5.0 and later
Linux x86-64

Symptoms

After upgrading to BDA 2.5.0, with  LZO compression RPMS for 1.2.3 installed, the Teragen-Terasort-Teravalidate tests fail while running 'mammoth -c' with the following:

Running 100 GB teragen-terasort-teravalidate Hadoop Validation Test
 teragen         : 35 s
 terasort        : 3 s
 teravalidate    : 2 s
 -----------------------------
 Total time       : 40 s

 Status : failed

 

The HDFS *-tera-out file shows the following:

14/05/23 13:52:26 INFO terasort.TeraSort: starting
14/05/23 13:52:27 INFO mapred.FileInputFormat: Total input paths to process : 320
java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzopCodec not found.
    at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:134)
    at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:174)
    at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:96)
    at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:86)
    at org.apache.hadoop.examples.terasort.TeraInputFormat$TeraRecordReader.<init>(TeraInputFormat.java:155)
    at org.apache.hadoop.examples.terasort.TeraInputFormat.getRecordReader(TeraInputFormat.java:200)
    at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:123)
    at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:243)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzopCodec not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1680)
    at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:127)
    ... 21 more

 

Note that this happens after the LZO compression RPMS are installed / updated for the new version.

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms