Running a Sqoop Command Brings Down the HBase RegionServer on One Node of the BDA Cluster

(Doc ID 2096063.1)

Last updated on JANUARY 19, 2016

Applies to:

Big Data Appliance Integrated Software - Version 4.3.0 and later
Linux x86-64

Symptoms

Running a Sqoop from an edge node brings down the HBase RegionServer (RS) on one node of the BDA Cluster.  The HBase RegionServer goes down on the same node every time.

1. Checking in Cloudera Manager shows the HBase RegionServer reporting various errors like:

The health test result for REGION_SERVER_GC_DURATION has become bad
The health test result for REGION_SERVER_MEMSTORE_SIZE has become bad
The health test result for REGION_SERVER_MASTER_CONNECTIVITY has become bad
The health test result for REGION_SERVER_SCM_HEALTH has become bad

2. The log file: /var/run/cloudera-scm-agent/process/<most recent>-hbase-REGIONSERVER/logs/stdout.log shows an OutOfMemoryError killing the RS:

Mon Dec 28 13:24:54 EST 2015 Starting znode cleanup thread with HBASE_ZNODE_FILE=/var/run/cloudera-scm-agent/process/<most-recent>-hbase-REGIONSERVER/znode66803 for regionserver
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p
/usr/lib64/cmf/service/common/killparent.sh"
# Executing /bin/sh -c "kill -9 66803
/usr/lib64/cmf/service/common/killparent.sh"...

3. The log file: /var/run/cloudera-scm-agent/process/<most recent>-hbase-REGIONSERVER/logs/stderr.log shows that the RS only has 500MG of heap:

-Xms52428800 -Xmx52428800

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms