On Oracle Big Data Appliance , CDH cluster Node01 Migration Repeatedly Fails at Step10 and both Namenodes are Standby (Doc ID 1994118.1)

Last updated on OCTOBER 11, 2016

Applies to:

Big Data Appliance Integrated Software - Version 4.1.0 and later
Linux x86-64

Symptoms

Node01 migration repeatedly fails at step10 (tried 3 times) while migrating to a non critical node. In all cases the issue is that both NN's are in Standby mode.

Errors from setupscm_1427309198.out

Command 4393 finished after 5 seconds
Operation failed
Result Message is: "Command Initialize is not currently available for execution.",
API Version used is v9
Succeeded. Output in : /opt/oracle/BDAMammoth/bdaconfig/tmp/clusters_*******-cluster_services_zookeeper_commands_restart.out

Command 4453 finished after 5 seconds
Operation failed
Result Message is: "Failed to restart service.",
API Version used is v9
Succeeded. Output in : /opt/oracle/BDAMammoth/bdaconfig/tmp/clusters_*******-cluster_services_yarn_commands_start.out

Command 4585 finished after 340 seconds
Operation failed
Result Message is: "Failed to create HDFS directory.",

When the failure occurs followed Doc ID 1957275.1 to format failover controller and restart the CDH services fine. But again migration command fails at the same point.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms