NIC Failover in IPMP Setup Causes the VIP to go and Remain Offline - CRS-5008 (Doc ID 1121816.1)

Last updated on MARCH 12, 2013

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.1 to 11.2.0.2 [Release 11.2]
Oracle Solaris on SPARC (64-bit)
Oracle Solaris 10
Oracle Enterprise Edition 11.2.x or higher
Oracle Clusterware 11.1.x or higher
Oracle Solaris IPMP

Note: Recommended Patches 9260196 and 9393817 should be applied.


Symptoms

Testing IPMP NIC failover in Oracle Solaris 10 with Oracle Enterprise Edition 11.2.x

Pulling the primary public NIC from one of the nodes you encounter the following error in the cluster Alert.log


CRS-5818:Aborted command 'check for resource: ora.net1.network <node1> for resource 'ora.net1.network'



Looking further in the $GRID_HOME at the orarootagent_root.log
the following is present

[ora.net1.network[[16] [check] CRS-5008: Invalid attribute value: nxge0 for the network interface




In the Operating system's messages file /var/adm/messages, all indications from an IPMP perspective point to a successful failover:



Messages File:


May  6 17:02:56 node1 nxge: [ID 339653 kern.notice] NOTICE: nxge0: xcvr
addr:0x1d - link is down
May  6 17:02:56 node1 in.mpathd[185]: [ID 215189 daemon.error] The link has
gone down on nxge0
May  6 17:02:56 node1 in.mpathd[185]: [ID 594170 daemon.error] NIC failure
detected on nxge0 of group live
May  6 17:02:56 node1 in.mpathd[185]: [ID 832587 daemon.error] Successfully
failed over from NIC nxge0 to NIC nxge2



The output of ifconfig -a also looks correct:


root@node1 #  /usr/sbin/ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g1: flags=1069040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE,FIXEDMTU> mtu 9000 index 2
        inet 173.15.0.6 netmask ffffff00 broadcast 173.15.0.255
        groupname priv
        ether 0:15:17:79:29:11
nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 90.224.207.115 netmask ffffffe0 broadcast 90.224.207.149
        groupname publ
        ether 0:21:28:34:c6:7c
nxge0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
        inet 90.224.207.138 netmask ffffffe0 broadcast 90.224.207.149
nxge0:2: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 3
        inet 90.224.207.144 netmask ffffffe0 broadcast 90.224.207.149
nxge0:3: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 3
        inet 90.224.207.147 netmask ffffffe0 broadcast 90.224.207.149
nxge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
        inet 88.89.93.195 netmask ffffff00 broadcast 88.89.93.255
        ether 0:21:28:34:c6:7d
nxge2: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE> mtu 1500 index 5
        inet 90.224.207.116 netmask ffffffe0 broadcast 90.224.207.149
        groupname publ
        ether 0:21:28:34:c6:7e
nxge3: flags=1001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,FIXEDMTU> mtu 9000 index 6
        inet 173.15.0.4 netmask ffffff00 broadcast 173.15.0.255
        groupname priv
        ether 0:21:28:34:c6:7f
nxge3:1: flags=1009040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FIXEDMTU> mtu 9000 index 6




Nodeapps output:


root@node1 # srvctl config nodeapps -a
VIP exists.:node2
VIP exists.: /node2-vip/90.224.207.150/255.255.255.224/nxge0
VIP exists.:node1
VIP exists.: /node1-vip/90.224.207.151/255.255.255.224/nxge0
        inet 173.15.0.5 netmask ffffff00 broadcast 173.15.0.255


Recommended Patches 9260196 and 9393817 should be applied

Changes

none.  New Setup

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms