My Oracle Support Banner

Upgrade Of BDA Cluster to 4.* Fails at Step 7 Running "mammoth -p" Command with "Hadoop::Upgradeactions/Exec[set_cluster_conf]" (Doc ID 2044936.1)

Last updated on AUGUST 03, 2021

Applies to:

Big Data Appliance Integrated Software - Version 4.1.0 to 4.2.0 [Release 4.1 to 4.2]
Linux x86-64

Symptoms

NOTE: In the examples that follow, user details, cluster names, hostnames, directory paths, filenames, etc. represent a fictitious sample (and are used to provide an illustrative example only). Any similarity to actual persons, or entities, living or dead, is purely coincidental and not intended in any manner.

Running "mammoth -p" command on upgrade to 4.* , step 7 failed with:

ERROR: Puppet agent run on node <HOSTNAME1> had errors. List of errors follows
************************************
Error [12527]: (//<HOSTNAME1>.<DOMAIN>//Stage[main]/Hadoop::Upgradeactions/Exec[set_cluster_conf]/returns) change from notrun to 0 failed: /opt/oracle/BDAMammoth/bdaconfig/tmp/setclusterconf.sh &>
/opt/oracle/BDAMammoth/bdaconfig/tmp/setclusterconf.out returned 1 instead of one of [0]
************************************


Checking in CM and all recent commands found host monitor failed to start with this message:

Service did not start successfully; not all of the required roles started: Service has only 0 Host Monitor roles running instead of minimum required 1.

 

Checking in CM > mgmt > Host Monitor is showing down.

From /opt/oracle/BDAMammoth/bdaconfig/tmp/cm_service_commands_restart.out:

Command ID is 38555
...........
Command 38555 finished after 60 seconds
Operation failed
Result Message is:   "Failed to restart service.",

/opt/oracle/BDAMammoth/bdaconfig/tmp/cm_service_commands_restart.out

shows:
"id" : 38555,
 "name" : "Restart",
 "startTime" : "2015-08-14T20:17:39.501Z",
 "active" : true,
 "serviceRef" : {
   "serviceName" : "mgmt"
 }


Host Monitor logs showed errors about missing partitions in the Host Monitor storage directory about LDBTimeSeries:

Host Monitor logs showed errors about missing partitions in the Host Monitor storage directory...

2015-08-14 11:34:12,879 ERROR com.cloudera.enterprise.dbpartition.PartitionManager: Error expiring partitions: /var/lib/cloudera-host-monitor/ts/type/partitions/type_2015-06-30T23:05:26.840Z does not exist
java.lang.IllegalArgumentException: /var/lib/cloudera-host-monitor/ts/type/partitions/type_2015-06-30T23:05:26.840Z does not exist
at com.cloudera.cmf.FileUtils2.checkDirectory(FileUtils2.java:75)
at com.cloudera.cmf.FileUtils2.sizeOfDirectory(FileUtils2.java:41)
at com.cloudera.cmon.tstore.leveldb.LDBPartitionManager.getPartitionSizeInBytes(LDBPartitionManager.java:670)
at com.cloudera.cmon.tstore.leveldb.LDBSizeBasedPartitionPolicy.getPartitionsToExpire(LDBSizeBasedPartitionPolicy.java:157)
at com.cloudera.cmon.tstore.leveldb.LDBPartitionManager.expirePartitions(LDBPartitionManager.java:974)
at com.cloudera.enterprise.dbpartition.PartitionManager.runPartitionManagement(PartitionManager.java:142)
at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesTable.forcePartitionManagement(LDBTimeSeriesTable.java:131)
at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesStore.forcePartitionManagement(LDBTimeSeriesStore.java:610)
at com.cloudera.cmon.firehose.Firehose.<init>(Firehose.java:105)
at com.cloudera.cmon.firehose.Main.main(Main.java:527)

 

 

Changes

 Upgrade to 4.2.0.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.