My Oracle Support Banner

Node 3 Migration Fails During StartHadopServices When "Starting Hadoop Services" Due to a Missing JournalNode Directory on the Migrated Host (Doc ID 2524852.1)

Last updated on NOVEMBER 08, 2019

Applies to:

Big Data Appliance Integrated Software - Version 4.11.0 and later
Linux x86-64

Symptoms

Node 3 migration fails during StartHadopServices, at the point of "Starting Hadoop Services" step with:

************************************
Error [24192]: (//<HOSTNAME5>.<DOMAIN>//Stage[main]/Hadoop::Startsvc2/Exec[setup_scm]/returns) change from notrun to 0 failed: /opt/oracle/BDAMammoth/bdaconfig/tmp/setupscm.sh &> /opt/oracle/BDAMammoth/bdaconfig/tmp/setupscm_<##>.out returned 1 instead of one of [0]
************************************

In the example here Node 3 is being migrated to Node 5.

1. Cloudera Manager (CM) shows that the hdfs service is in bad health because the JournalNode is in bad health on the server being migrated to.  The active NameNode is in bad health as well.

a) The JournalNode logs show errors like:

IPC Server handler 3 on <PORT>, call org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocol.heartbeat from <PRIVATE_IP_HOST>:<PORT> Call#<##> Retry#<##> java.io.FileNotFoundException: /opt/hadoop/dfs/jn/<CLUSTERNAME>-ns/current/last-promised-epoch.tmp (No such file or directory)

b) The NameNode reports that it is out of sync with the JournalNode.

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.