Clusterware Fails to Start Due to PROC-44: Error in network address and interface operations (Doc ID 1515237.1)

Last updated on OCTOBER 04, 2017

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.2 and later
Information in this document applies to any platform.

Symptoms

After reboot, the clusterware is not starting on any node.

Here is an example from a 2 node cluster.

Node1:

> crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager

> crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
     1        ONLINE  ONLINE       Node1         Started
ora.cluster_interconnect.haip
     1        ONLINE  OFFLINE
ora.crf
     1        ONLINE  ONLINE       Node1
ora.crsd
     1        ONLINE  OFFLINE
ora.cssd
     1        ONLINE  ONLINE       Node1
ora.cssdmonitor
     1        ONLINE  ONLINE       Node1
ora.ctssd
     1        ONLINE  ONLINE       Node1         OBSERVER
ora.diskmon
     1        ONLINE  ONLINE       Node1
ora.drivers.acfs
     1        ONLINE  ONLINE       Node1
ora.evmd
     1        ONLINE  INTERMEDIATE Node1
ora.gipcd
     1        ONLINE  ONLINE       Node1
ora.gpnpd
     1        ONLINE  ONLINE       Node1
ora.mdnsd
     1        ONLINE  ONLINE       Node1

   

Node2:
     
> crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager

> crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
     1        ONLINE  OFFLINE
ora.cluster_interconnect.haip
     1        ONLINE  OFFLINE
ora.crf
     1        ONLINE  ONLINE       Node2
ora.crsd
     1        ONLINE  OFFLINE
ora.cssd
     1        ONLINE  OFFLINE                               STARTING
ora.cssdmonitor
     1        ONLINE  ONLINE       Node2
ora.ctssd
     1        ONLINE  OFFLINE
ora.diskmon
     1        ONLINE  ONLINE       Node2
ora.drivers.acfs
     1        ONLINE  OFFLINE
ora.evmd
     1        ONLINE  OFFLINE
ora.gipcd
     1        ONLINE  ONLINE       Node2
ora.gpnpd
     1        ONLINE  ONLINE       Node2
ora.mdnsd
     1        ONLINE  ONLINE       Node2

 
 
Try to start HAIP resource on Node1 using below command.

> crsctl start res ora.cluster_interconnect.haip -init  
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'Node1'
CRS-5017: The resource action "ora.cluster_interconnect.haip start" encountered the following error:
Start action for HAIP aborted
CRS-2674: Start of 'ora.cluster_interconnect.haip' on 'Node1' failed
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'Node1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'Node1' succeeded
CRS-4000: Command Start failed, or completed with errors.

 
Check the Cluster alert log and crsd.log from Node1.

alertnode1.log:
====================

2012-10-23 17:26:33.861
[ohasd(10269)]CRS-2765:Resource 'ora.crsd' has failed on server 'Node1'.
2012-10-23 17:26:35.136
[crsd(9258)]CRS-0804:Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-44: Error in network address and interface operations 
Network address and interface operations error [7]]. Details at (:CRSD00111:) in /opt/apps/grid/product/11g2/grid_home1/log/Node1/crsd/crsd.log.
2012-10-23 17:26:35.888
[ohasd(10269)]CRS-2765:Resource 'ora.crsd' has failed on server 'Node1'.
2012-10-23 17:26:37.165
[crsd(9271)]CRS-0804:Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-44: Error in network address and interface operations 
Network address and interface operations error [7]]. Details at (:CRSD00111:) in /opt/apps/grid/product/11g2/grid_home1/log/Node1/crsd/crsd.log.
2012-10-23 17:26:37.915
[ohasd(10269)]CRS-2765:Resource 'ora.crsd' has failed on server 'Node1'.

 

crsd.log (Node1):
====================

2012-10-23 17:25:33.592: [ default][613898416] CRS Daemon Starting
2012-10-23 17:25:33.596: [ CRSMAIN][613898416] Initializing OCR
[   CLWAL][613898416]clsw_Initialize: OLR initlevel [70000]
2012-10-23 17:25:33.664: [  OCRAPI][613898416]clsu_get_private_ip_addresses: no ip addresses found.
[  OCRAPI][613898416]a_init_clsss: failed to call clsu_get_private_ip_addr (7)
2012-10-23 17:25:33.870: [  OCRAPI][613898416]a_init:13!: Clusterware init unsuccessful : [44]
2012-10-23 17:25:33.871: [  CRSOCR][613898416] OCR context init failure.  Error: PROC-44: Error in network address and interface operations Network 
address and interface operations error [7]
2012-10-23 17:25:33.871: [ CRSMAIN][613898416] Created alert : (:CRSD00111:) :  Could not init OCR, error: PROC-44: Error in network address and 
interface operations Network address and interface operations error [7]
2012-10-23 17:25:33.871: [    CRSD][613898416][PANIC] CRSD exiting: Could not init OCR, code: 44
2012-10-23 17:25:33.871: [    CRSD][613898416] Done.

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms