Not able to Start "nodeapps" due to inconsistency between oifcfg iflist and getif

(Doc ID 2210762.1)

Last updated on DECEMBER 25, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.4 and later
Information in this document applies to any platform.

Symptoms

Nodeapps fails to start.

# ./srvctl start nodeapps
PRCR-1079 : Failed to start resource ora.gsd
CRS-5016: Process "/opt/oracle/grid4/bin/gsdctl" spawned by agent "/opt/oracle/grid4/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/opt/oracle/grid4/log/Node2/agent/crsd/oraagent_oracle//oraagent_oracle.log"
CRS-5016: Process "/opt/oracle/grid4/bin/gsdctl" spawned by agent "/opt/oracle/grid4/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/opt/oracle/grid4/log/Node1/agent/crsd/oraagent_oracle//oraagent_oracle.log"
CRS-2674: Start of 'ora.gsd' on 'Node1' failed
CRS-2674: Start of 'ora.gsd' on 'Node2' failed
PRCR-1079 : Failed to start resource ora.net1.network
CRS-2674: Start of 'ora.net1.network' on 'Node1' failed
CRS-2674: Start of 'ora.net1.network' on 'Node2' failed
PRCR-1079 : Failed to start resource ora.Node1.vip
CRS-2674: Start of 'ora.net1.network' on 'Node1' failed
CRS-2674: Start of 'ora.net1.network' on 'Node2' failed
CRS-2632: There are no more servers to try to place resource 'ora.Node1.vip' on that would satisfy its placement policy
PRCR-1079 : Failed to start resource ora.Node2.vip
CRS-2674: Start of 'ora.net1.network' on 'Node2' failed
CRS-2674: Start of 'ora.net1.network' on 'Node1' failed
CRS-2632: There are no more servers to try to place resource 'ora.Node2.vip' on that would satisfy its placement policy
PRCR-1079 : Failed to start resource ora.ons
CRS-2674: Start of 'ora.net1.network' on 'Node1' failed
CRS-2674: Start of 'ora.net1.network' on 'Node2' failed

 IFLIST presents a different IP subnet than getif causing nodeapps not to start

Node1[+ASM1]: oifcfg iflist -p -n
bond0 10.25.16.0 PRIVATE 255.255.252.0     <===== 
bond1 169.254.0.0 UNKNOWN 255.255.0.0

Node1[+ASM1]: oifcfg getif
bond1 169.254.0.0 global cluster_interconnect
bond0 10.25.18.0 global public      <=====

 

The VIPs shows also correct settings:-

Network exists: 1/10.25.18.0/255.255.255.0/bond0, type static
VIP exists: /xxxxx1h1v/10.25.18.89/10.25.18.0/255.255.255.0/bond0, hosting node xxxxx1h1
VIP exists: /xxxxx1h2v/10.25.18.95/10.25.18.0/255.255.255.0/bond0, hosting node xxxxx1h2

The resource ora.net1.network is also showing the correct settings:-

$./crsctl status res ora.net1.network -p | grep USR_ORA
USR_ORA_AUTO=static
USR_ORA_ENV=
USR_ORA_IF=bond0
USR_ORA_NETMASK=255.255.255.0
USR_ORA_SUBNET=10.25.18.0

 Whereas the "oraagent_oracle.log" file shows below while starting the nodeapps:-

2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] Checking if bond0 Interface is fine
2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] ifname=bond0
2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] subnetmask=255.255.252.0
2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] subnetnumber=10.25.16.0
2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] ifname=bond0
2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] subnetmask=255.255.252.0
2016-12-01 09:45:33.604: [ora.net1.network][974747968]{1:41130:1873} [check] subnetnumber=10.25.16.0
2016-12-01 09:45:33.605: [ora.net1.network][974747968]{1:41130:1873} [check] CRS-5008: Invalid attribute value: bond0 for the network interface 

Changes

 None

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms