My Oracle Support Banner

Oracle Big Data Appliance Node Migration Frequently Asked Questions (FAQ) (Doc ID 1946287.1)

Last updated on JULY 17, 2023

Applies to:

Big Data Appliance Integrated Software - Version 4.0 and later
Linux x86-64

Purpose

This document provides answers to frequently asked node migration questions on Oracle Big Data Appliance (BDA). 

Questions and Answers

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Purpose
Questions and Answers
 What is the expectation on a Hadoop cluster with regards to Hardware/Software resiliency?
 What is the minimum BDA cluster size for node migration or reprovision?
 Is the process for migrating services from a critical BDA server to another node automatic?
 Where are critical services migrated to?
 Is it possible to select the noncritical node the services from a failing/failed node are moved to?
 Is it mandatory to remove/decommission any non BDA nodes from the Cluster prior to migration/reprovision?
 What should be done if the steps to reprovision a non-critical BDA node are done out of order and decommission was skipped and reimage done first?
 After decommissioning a node it shows up on the QUARANTINED lists in the /opt/oracle/bda/install/state/config.json file.  What should those look like post-decommission?
 When performing node migration on Node 1 can you select any noncritical node to install Mammoth on and take over the Node 1 role?
 Could all 4 critical nodes be migrated to the same noncritical node?
 When you migrate a critical node to a noncritical node is it possible to get overcommit violations? 
 What about cases where Nodes 1-4 are configured with additional CPU and memory but now the services are migrated to another node?
 Why does a noncritical node need to be decommissioned with: bdacli admin_cluster decommission node-name?
 When a node is migrated does its name or IP address change?
 When a critical node has its services migrated to another node will that new address need to be broadcast to users?
 Can Cloudera Manager be moved directly from Node 3 to another node?
 Is there any impact to client or edge servers after critical services are migrated?
 Based on the above is it correct that Mammoth automatically upgrades client configurations?
 What about services not managed by Mammoth?  When a critical node has its services migrated will those services be migrated too?
 In the case of a failing node 3 and subsquent migration of the critical services to a new node, what if the Hive metastore is pointing to the MySQL database instance on the original node 3?  Does the bdacli  point the Hive metastore to the new/backup MySQL DB?  Or, does the administrator need to go into CM and point the hive metastore to the backup mysql?
 If the Sentry Service was enabled with Mammoth, is there anything special to do during node migration?
 What happens if the Sentry Service was not enabled with Mammoth?  What needs to be done on node migration?
 If the keytrustee service was added with Mammoth are there any special steps to take during migration?
 If the Key Trustee Servers are off the BDA but CM managed is there any impact on Node migration?
 Will node migration result in any changes in private keys or trust stores or impact renewed certificates?
 When migrating Node 3 will the migration replicate the MySQL datbase from Node 2 and all configuration files?
 In the case of BDD being located on Node 5, will migrating Node 4 migrate the Node 4 roles to Node 5? Or will they be migrated to Node 6?
 Before migration is it required to remove the HBase role from the cluster? 
 If you made custom changes to /etc/my.cnf file in both node02 and node03 by hand for example according to Doc ID 1529258.1, does "node03 migration" copy /etc/my.cnf file from node02 to new-critical node?
 What about critical services configured on a non-critical node?
 After migrating a critical node is it mandatory to migrate it back to it's original configuration?
 After migrating a critical node can the cluster be restored to the original configuration where critical services reside on nodes 1-4 in the case of a single rack?
 Does node migration  handle the case of all 4 critical nodes going down at once?
 What about cases of nodes 1/2, 2/3, 3/4 failing together? Do the scripts handle this case?
 What about an Oracle NoSQL DB BDA cluster.  Can those nodes be migrated as well?
 What about Kerberos?  Can nodes be migrated in secure clusters?
 What about downtime?  What is expected during node migration, reprovision, etc.?
 Is it ok to remove the /opt/oracle/bda/network.json or rack-network.json/cluster-network.json from a reimaged node which will be reprovisioned?
 When a node is reprovisioned, is there any password requirement?
 If the Mammoth script fails during migration, can you back out the changes?
 Instead of performing "node migration" using bdacli to migrate one or some roles/services to another node is it possible to do the same in Cloudera Manager using: "Migrate Roles for hdfs", "Add Role Instances" or "Add a Service"?
 Node 1 Migration/Reprovision with CDH 5.5.2 Q/A
 In the case of migrating Node 1 on BDA V4.4 cluster with CDH 5.5.2 are there any special considerations?
 In the case of subsequent Node 1 reprovision on a BDA 4.4 cluster with CDH 5.5.2 are there any special considerations?
 Can you get a new Server, off BDA, move the Mammoth 4.4 bundle there and migrate the Node 1 functionality there instead of to Node 5 ?
 During migration what services are migrated and what services are not migrated ?
 On BDA V4.5 what are the general steps to follow if a non-critical server needs to be reprovisioned, due to requiring a reimage.
 On BDA V4.5 do you need to apply patch 23763068 before node migration/reprovision/etc?
 Is it possible to move some critical roles from the critical nodes 1-4 to another node?
 In the case of migrating the roles for multiple critical hosts is there any recommended order of migration?
 In the case of migrating the roles for multiple critical hosts as described above what is the recommended method for performing multiple migrations?
 How many steps are there migrating the host with CM role, Node 3 by default?
 Does node recommission have any downtime?
 Can the NameNode(NN) ID and ResourceManager(RM) ID used by Cloudera Manager(CM) be retained in the case of a node migration?
 Can the RM ID be restored if the RM role is migrated back to the original host? Or will any action be able to restore the original Resource Manager ID?
References

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.