My Oracle Support Banner

Oracle Big Data Appliance V2.3.1 Mammoth Install Step 14 to Configure Enterprise Manager system monitoring plug-in Fails with ERROR : An unexpected exception (Doc ID 1600815.1)

Last updated on MARCH 23, 2014

Applies to:

Big Data Appliance Integrated Software - Version 2.3.1 to 2.3.1 [Release 2.3]
Linux x86-64

Symptoms


The Mammoth V2.3.1 install fails at Step 14 when configuring the Oracle Enterprise Manager system monitoring plug-in for Oracle Big Data Appliance with errors like:

Running: /opt/oracle/emcli_home/emcli discover_bda_cluster -hostname=bdanode01.example.com
-cloudera_credential=BDA_CLUST_CM_CRED -host_credential=BDA_CLUST_HOSTS_CRED -ilom_credential=BDA_CLUST_ILOM_CRED
-infiniband_credential=BDA_CLUST_SWITCH_CRED -snmp_string=public
ERROR: ERROR : An unexpected exception has been thrown. Please check the logs

 

More complete output looks like below.  For example where the rack name is "bdarack".

# ./mammoth -s 14 bdarack
  

INFO: Logging all actions in /opt/oracle/BDAMammoth/bdaconfig/tmp/STEP-14-20131106120608.log and traces in /opt/oracle/BDAMammoth/bdaconfig/tmp/STEP-14-20131106120608.trc
INFO: Checking configuration file /opt/oracle/BDAMammoth/bdaconfig/../mammoth-bdarack.params...
INFO: This is the install of the primary rack
INFO: Loading configuration file /opt/oracle/BDAMammoth/bdaconfig/../mammoth-bdarack.params...
INFO: Reading component versions from /opt/oracle/BDAMammoth/bdaconfig/COMPONENTS
==================================================================================
INFO: Wed Nov  6 12:06:08 PST 2013

INFO: Step 14  SetupEMAgent
INFO: Acquiring run lock ...
INFO: Creating nodelist files...
INFO: Creating environment.pp file ...
INFO: Making sure all puppet agents can be accessed.
INFO: Pinging puppet agents
INFO: Creating directories for em download This will take some time ...
INFO: Puppet script successfully sent to node bdanode01
INFO: Puppet script successfully sent to node bdanode02
...
INFO: Pinging puppet agents
INFO: Puppet agent run on node bdanode01 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode01-20131106120643.log
INFO: Puppet agent run on node bdanode02 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode02-20131106120643.log
...
SUCCESS: Successfully created em download directories.
INFO: Making sure all puppet agents can be accessed.
INFO: Pinging puppet agents
INFO: Downloading EM agent on cluster. This will take some time ...
INFO: Puppet script successfully sent to node bdanode01
INFO: Pinging puppet agents
INFO: Puppet agent run on node bdanode01 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode01-20131106120802.log
SUCCESS: Successfully downloaded EM Agent.
SUCCESS: Copying EM Agent on all nodes
SUCCESS: Ran /usr/bin/scp /opt/oracle/EMAgent/agent.zip root@bdanode02:/opt/oracle/EMAgent/agent.zip and it returned: RC=0
...
INFO: Making sure all puppet agents can be accessed.
INFO: Pinging puppet agents
INFO: Deploying EM agent on cluster. This will take some time ...
INFO: Puppet script successfully sent to node bdanode01
INFO: Puppet script successfully sent to node bdanode02
...
INFO: Pinging puppet agents
INFO: Puppet agent run on node bdanode01 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode01-20131106121105.log
INFO: Puppet agent run on node bdanode02 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode02-20131106121105.log
...
SUCCESS: Successfully deployed EM Agent.
SUCCESS: Ran cp -r /opt/oracle/bda/install/state /opt/oracle/bda/install/state-backup and it returned: RC=0

INFO: No BDA plugin installed
INFO: Deploying latest plugins on agent
Running: /opt/oracle/emcli_home/emcli deploy_plugin_on_agent -plugin=oracle.sysman.bda -agent_names="bdanode01.example.com:3872;bdanode02.example.com:3872;..."
SUCCESS: Deploying oracle.sysman.bda..
INFO: Still deploying: oracle.sysman.bda
INFO: Still deploying: oracle.sysman.bda
SUCCESS: Successfully deployed the plugin
Running: /opt/oracle/emcli_home/emcli deploy_plugin_on_agent -plugin=oracle.sysman.xa -agent_names="bdanode01.example.com:3872;bdanode02.example.com:3872;..."
SUCCESS: Deploying oracle.sysman.xa..
INFO: Still deploying: oracle.sysman.xa
INFO: Still deploying: oracle.sysman.xa
SUCCESS: Successfully deployed the plugin
Running: /opt/oracle/emcli_home/emcli create_named_credential -cred_name='BDA_CLUST_SWITCH_CRED' -auth_target_type=oracle_ibswitch -cred_type=IBSwitchCreds -attributes="IBSwitchUserName:nm2user;IBSwitchPassword:changeme"
SUCCESS: Credential BDA_CLUST_SWITCH_CRED created.
Running: /opt/oracle/emcli_home/emcli create_named_credential -cred_name='BDA_CLUST_ILOM_CRED' -auth_target_type=oracle_exa_ilom -cred_type=IlomCreds -attributes="UserName:root;Password:xxxxx"
SUCCESS: Credential BDA_CLUST_ILOM_CRED created.
Running: /opt/oracle/emcli_home/emcli create_named_credential -cred_name='BDA_CLUST_CM_CRED' -auth_target_type=oracle_hadoop_cluster -cred_type=AliasCredential -attributes="Alias:admin;Password:xxxxx"
SUCCESS: Credential BDA_CLUST_CM_CRED created.
Running: /opt/oracle/emcli_home/emcli create_named_credential -cred_name='BDA_CLUST_HOSTS_CRED' -auth_target_type=host -cred_type=HostCreds -attributes="HostUserName:root;HostPassword:xxxxx"
SUCCESS: Credential BDA_CLUST_HOSTS_CRED created.
SUCCESS: Ran /bin/cp -f /opt/oracle/bda/install/state/mammoth-saved.params /opt/oracle/BDAMammoth/bdaconfig/../puppet/modules/miscsetup/files and it returned: RC=0

SUCCESS: Ran /bin/cp -f /opt/oracle/bda/install/state/mammoth-bdarack-saved.params /opt/oracle/BDAMammoth/bdaconfig/../puppet/modules/miscsetup/files and it returned: RC=0

INFO: Making sure all puppet agents can be accessed.
INFO: Pinging puppet agents
INFO: Copying saved params file to all nodes. This will take some time ...
INFO: Puppet script successfully sent to node bdanode01
INFO: Puppet script successfully sent to node bdanode02
...
INFO: Pinging puppet agents
INFO: Puppet agent run on node bdanode01 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode01-20131106121855.log
INFO: Puppet agent run on node bdanode02 completed successfully.
INFO: Check the log file in /opt/oracle/BDAMammoth/bdaconfig/tmp/pagent-bdanode02-20131106121855.log
...
SUCCESS: Successfully copied saved params file to all nodes
Running: /opt/oracle/emcli_home/emcli discover_bda_cluster -hostname=bdanode01.example.com -cloudera_credential=BDA_CLUST_CM_CRED -host_credential=BDA_CLUST_HOSTS_CRED -ilom_credential=BDA_CLUST_ILOM_CRED -infiniband_credential=BDA_CLUST_SWITCH_CRED -snmp_string=public
ERROR: ERROR : An unexpected exception has been thrown. Please check the logs.
SUCCESS: Ran rm -rf /opt/oracle/bda/install/state && cp -r /opt/oracle/bda/install/state-backup /opt/oracle/bda/install/state && rm -rf /opt/oracle/bda/install/state-backup and it returned: RC=0

Are you sure you want to completely cleanup em and lose all related state ?

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.