On Oracle Big Data Appliance, Node Migration Fails while BootStrapping NameNode
(Doc ID 2048973.1)
Last updated on NOVEMBER 08, 2022
Applies to:
Big Data Appliance Integrated Software - Version 4.1.0 and laterLinux x86-64
Symptoms
NOTE: In the examples that follow, user details, cluster names, hostnames, directory paths, filenames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner.
On Oracle Big Data Appliance, Step11 of critical node migration fails while starting CDH services.
Error [19123]: (/<HOSTNAME3>//Stage[main]/Hadoop::Startsvc2/Exec[setup_scm]/returns) change from notrun to 0 failed: /opt/oracle/BDAMammoth/bdaconfig/tmp/setupscm.sh &> /opt/oracle/BDAMammoth/bdaconfig/tmp/setupscm_<TIMESTAMP>.out returned 1 instead of one of [0]
setupscm_<TIMESTAMP>.out file on BDA Node03 contains below errors.
"Failed to bootstrap Standby NameNode namenode (HOSTNAME1):
++ find /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode -maxdepth 1 -name '*.py'\n+
OUTPUT='/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/topology.py\n/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/cloudera_manager_agent_fencer.py'\n
+ '[' '/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/topology.py\n/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/cloudera_manager_agent_fencer.py' '!=' '' ']'\n+
chmod +x /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/cloudera_manager_agent_fencer.py /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/topology.py\n+
'[' DATANODE_MAX_LOCKED_MEMORY '!=' '' ']'\n+ ulimit -l\n+
export HADOOP_IDENT_STRING=hdfs\n+
HADOOP_IDENT_STRING=hdfs\n+ '[' -n '' ']'\n+
acquire_kerberos_tgt hdfs.keytab\n+ '[' -z hdfs.keytab ']'\n+ '[' -n hdfs/@ ']'\n+ '[' -d /usr/kerberos/bin ']'\n+
export PATH=/usr/kerberos/bin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/kerberos/bin\n+
PATH=/usr/kerberos/bin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/kerberos/bin\n+
which kinit\n+ '[' 0 -ne 0 ']'\n++ id -u\n+
export KRB5CCNAME=/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201\n+
KRB5CCNAME=/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201\n+
echo 'using hdfs/@ as Kerberos principal'\n+ echo 'using /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201 as Kerberos ticket cache'\n+
kinit -c /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201 -kt /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/hdfs.keytab hdfs/@\n
kinit: Password incorrect while getting initial credentials\n+ '[' 1 -ne 0 ']'\n+ echo 'kinit was not successful.'\n+ exit 1\n.",
++ find /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode -maxdepth 1 -name '*.py'\n+
OUTPUT='/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/topology.py\n/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/cloudera_manager_agent_fencer.py'\n
+ '[' '/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/topology.py\n/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/cloudera_manager_agent_fencer.py' '!=' '' ']'\n+
chmod +x /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/cloudera_manager_agent_fencer.py /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/topology.py\n+
'[' DATANODE_MAX_LOCKED_MEMORY '!=' '' ']'\n+ ulimit -l\n+
export HADOOP_IDENT_STRING=hdfs\n+
HADOOP_IDENT_STRING=hdfs\n+ '[' -n '' ']'\n+
acquire_kerberos_tgt hdfs.keytab\n+ '[' -z hdfs.keytab ']'\n+ '[' -n hdfs/@ ']'\n+ '[' -d /usr/kerberos/bin ']'\n+
export PATH=/usr/kerberos/bin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/kerberos/bin\n+
PATH=/usr/kerberos/bin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/kerberos/bin\n+
which kinit\n+ '[' 0 -ne 0 ']'\n++ id -u\n+
export KRB5CCNAME=/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201\n+
KRB5CCNAME=/var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201\n+
echo 'using hdfs/@ as Kerberos principal'\n+ echo 'using /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201 as Kerberos ticket cache'\n+
kinit -c /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/krb5cc_201 -kt /var/run/cloudera-scm-agent/process/1415-bootstrap-standby-namenode/hdfs.keytab hdfs/@\n
kinit: Password incorrect while getting initial credentials\n+ '[' 1 -ne 0 ']'\n+ echo 'kinit was not successful.'\n+ exit 1\n.",
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |
| References |