My Oracle Support Banner

Exalogic Virtual: How To Debug And Fix Authentication Failure, Assets Blacklisting And Incorrect Credential Issues In EMOC? (Doc ID 2431602.1)

Last updated on AUGUST 06, 2018

Applies to:

Oracle Exalogic Elastic Cloud Software - Version 2.0.6.0.0 and later
Linux x86-64
Oracle Virtual Server x86-64

Goal

This MOS Note provides detailed steps as follows:

In Exalogic Virtual racks, often we see authentication failures and incorrect credential related issues in EMOC. Authentication failures happen due to having incorrect credentials in EMOC for following Exalogic Infrastructure assets.

Having authentication failures on above assets in EMOC leads to EMOC blacklisting above assets in EMOC Domain model. This leads to other issues when doing vDC tasks in EMOC few of which are listed below.

The cause for the authentication related issues in most of the cases has been due to known issues with the Exapasswd tool.

Solution

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Goal
Solution
 STEP 1: Gather IP's of Compute Nodes & Compute Nodes ILOMs, Storage Heads & Storage Heads ILOMs, IB Switches, OVM Manager
 STEP 2: Check which Assets have Authentication Failures/Incorrect Credential issues
 2.1. Check which Compute Nodes and Compute Node ILOMs Assets have Authentication Failures
 2.1.1 Check from Proxy Controller cacao logs which Compute Node OS & ILOM IPs are having authentication failures and are blacklisted.
 2.1.2 Check from EMOC BUI Console If the Compute Nodes which have authentication errors are blacklisted from EMOC BUI.
 2.1.3 Compile list of Affected Compute Node OS & Compute Node Asset IP's which have authentication issues
 2.2. Check which Storage Nodes and Storage Nodes ILOMs assets have authentication failures
 2.2.1 Check from Proxy Controller cacao logs if there are authentication failures on Storage Node OS & ILOM IPs
 2.2.2 Check from EMOC BUI Console if Storage Nodes are blacklisted
 2.2.3 Compile list of Affected Storage Head OS & Storage Head ILOM IP's which have authentication issues
 2.3. Check which IB Switches assets have authentication failures
 2.3.1 Check from Proxy Controller cacao logs if there are authentication failures on IB Switches IPs
 2.3.2 Compile list of Affected IB Switches IP's which have authentication issues
 2.4. Check if OVM Manager Asset has Authentication failures/Incorrect credential issues
 2.4.1. Check from EMOC BUI Console if OVM Manager is reachable
 2.4.2. Check from Proxy Controller cacao logs & OVMM Admin Server Log if there are authentication failures on OVM Manager
 2.5 Compile list of all the assets which have Credential Issues
 STEP 3: Fixing Incorrect Crendetials for assets which have Authentication issues
 3.1 Fixing Affected Compute Node OS & Compute Node ILOM authentication/incorrect credential issues
 3.1.1 Fixing Compute Node ILOM SSH & ILOM IPMI Credential issues
 3.1.2 Fixing Compute Node OS SSH Credential issues
 3.2 Fixing Storage Heads OS & Storage Heads ILOM authentication/incorrect credential issues
 3.2.1 Fixing Storage Head ILOM SSH & Storage Head ILOM IPMI Credential issues
 3.2.2 Fixing Storage Head OS SSH Credential issues
 3.3 Fixing IB Switches SSH & IPMI Credentials
 3.3.1 Fixing IB Switch SSH Credentials
 3.3.2 Fixing IB Switch IPMI Credentials
 3.4 Fixing OVM Manager Credential issues in EMOC
 3.5 After fixing the credential issues listed in this STEP 3, proceed with next section STEP 4 by stopping, starting EC & PC services and verify if the authentication/incorrect credential issues are fixed.
 STEP 4: Stop & Start EC & PC Services & Verify if the Authentication/Incorrect Crendential Issues are Fixed
 4.0 Stop & Start EC & PC Services in EC1 and PC1, PC2 Control vServers and clean up all old cacao logs on PC VM's
 4.1 Validate if Authentication Issues are fixed for Compute Node OS & Compute Node ILOMs
 4.1.1 Validate if Blacklisting is removed for Compute Node assets in EMOC.
 4.1.2 Validate that there are no authentication errors or blacklisting errors on Compute Node assets in PC cacoa logs.
 4.2 Validate if Authentication Issues are fixed for Storage Head OS & Storage Head ILOMs
 4.2.1 Validate if Blacklisting is removed for Storage Head assets in EMOC.
 4.2.2 Validate that there are no authentication errors or blacklisting errors on Storage Head assets in PC cacoa logs.
 4.3 Validate if Authentication Issues are fixed for IB Switches which had issues before
 4.4 Validate if Authentication Issues are fixed for IB Switches which had issues before
 4.4.1 Validate if OVM Manager is Reachable from EMOC BUI
 4.4.1 Validate OVM Manager is NOT marked as unreachable from Proxy Controller cacao logs
 Collect Full Exalogs Contact Support if still Authentication failure/Blacklisting of the assets is observed after following procedure in this Note
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.