My Oracle Support Banner

How to Prepare an Exalogic Infiniband (IB) Fabric for Planned Outage of an IB Switch (Doc ID 2211261.1)

Last updated on SEPTEMBER 25, 2023

Applies to:

Oracle Exalogic Elastic Cloud Software - Version 2.0.0.0.0 and later
Linux x86-64
Oracle Solaris on x86-64 (64-bit)
Oracle Virtual Server x86-64


Purpose

Prepare an Exalogic Infiniband (IB) Fabric, for any planned outage of an Infiniband Switch within that IB Fabric during Switch replacement.

The checklist in this Note will assist Customer-admin to determine if a full Fabric outage will be required, based on the results of checks done.

Scope

The scope of this document is EXALOGIC platform ONLY. All other platforms refer to following MOS Note:

<Note 2140928.1>: How to Prepare an Infiniband (IB) Fabric for Planned Outage of an IB Switch 

Planned Outage could include a Reboot (or boot after previous shut-down), Patching (firmware-upgrade), or Replacement of an IB Switch in the IB Fabric.

The checks and actions in this document are critical to ensuring that production traffic in the Exalogic’s Infiniband (IB) Fabric may be resilient to the necessary restart of the IB Switch involved in any of the above operations.

Based on the result of the aforementioned checks, guidance is provided - via a checklist - as to whether a full downtime of the IB Fabric will be required (full outage of all switches and nodes actively participating in the fabric). Customers should only take the IB Switch outage within a production IB Fabric, when all checks are cleared in the affirmative.

This document is referenced by several other Oracle Support knowledge articles, including:

<Note 1636229.1>: How to Prepare an Infiniband Switch for Replacement

The document distribution is EXTERNAL since it needs to be shared with and used by the Customer-admin, as well as referenced by Partners, Field Engineers, and Oracle Support. 

Details

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Purpose
Scope
Details
 1. Checks for IB fabric with multiple IB Switches
 1.1. Confirm Hosts bonding/IPMP/IO-path redundancy
 1.2. Check the opensm status and smpriorities on all switches
 1.3. Check IB Fabric using “ibswitches” and “getmaster” commands
 1.4. Check that all IB Switches can ping each other through management interfaces (via the Management Ethernet switch)
 1.5. Check IB partitions
 Validating Using Exabr
 Manual Procedure 
 2. Confirm type/extent of downtime required
 3. Complete the check-list template – IB Fabric preparation for IB Switch planned outage.
 4. Data Collection
 5. Proceed to next steps
References

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.