After a node crash, reboot or crs is restarted, crs on a node cannot restart or causes node reboot (Doc ID 845889.1)

Last updated on NOVEMBER 13, 2012

Applies to:

Oracle Server - Enterprise Edition - Version 10.2.0.5.0 to 11.1.0.7 [Release 10.2 to 11.1]
Information in this document applies to any platform.
This problem can occur on any platform.

Symptoms

After a node crash, reboot or CRS is restarted unsuccessfully, CRS on a node cannot rejoin the cluster.
 

 
ocssd.log.node1:
[ CSSD]2009-03-06 12:03:29.992 [14] >TRACE: clssnmConnComplete:
connected to node 2, node2, con (100793510), ninfcon (100793510), state
0, flags 0x40d, ICIN 114611864, properties 1,2,3,4,5,6,7,8,9
[ CSSD]2009-03-06 12:07:12.775 >USER: Copyright 2009, Oracle version
11.1.0.7.0
[ CSSD]2009-03-06 12:07:12.775 >USER: CSS daemon log for node node1,
number 1, in cluster node1_cluster
.
occsd.log.node2:
[ CSSD]2009-03-06 12:02:40.084 [14] >TRACE: clssnmConnComplete:
connected to node 1, node1, con (100dc52d0), ninfcon (100dc52d0), state 0,
flags 0x40d, ICIN 114611864, properties 1,2,3,4,5,6,7,8,9
[ CSSD]2009-03-06 12:02:40.093 [16] >TRACE: clssgmClientConnectMsg:
properties of cmProc 100a9f7d0 - 1,2,3
[ CSSD]2009-03-06 12:02:40.093 [16] >TRACE: clssgmClientConnectMsg:
Connect from con(100ad7910) proc(100a9f7d0) pid(25340/25340) version 11:1:1:4
[ CSSD]2009-03-06 12:02:40.168 [14] >WARNING: clssnmeventhndlr: Receive
failure with node 1 (node1), state 0, con(100dc52d0), probe(0), rc=11
[ CSSD]2009-03-06 12:02:40.168 [14] >TRACE: clssnmDiscHelper: node1,
node(1) connection failed, con (100dc52d0), probe(0)
...
[ CSSD]2009-03-06 12:06:24.294 [14] >TRACE: clssnmConnComplete: node 1
bounced, closing con (0), flags 0x0001, node unique 0, prev unique
1236359009, msg unique 1236359232
[ CSSD]2009-03-06 12:06:24.294 [14] >TRACE: clssnmConnComplete: probe
(100dc52d0) from node 1, node1
[ CSSD]2009-03-06 12:06:24.295 [14] >TRACE: clssnmConnComplete:
connected to node 1, node1, con (100dc52d0), ninfcon (100dc52d0), state 0,
flags 0x40d, ICIN 114611864, properties 1,2,3,4,5,6,7,8,9
[ CSSD]2009-03-06 12:06:24.740 [16] >TRACE: clssgmExecuteClientRequest:
Received data update request from client (100aeef50), type 1
[ CSSD]2009-03-06 12:06:24.791 [14] >TRACE: clssnmHandleJoin: node 1
JOINING, state 0->1 ninfcon 100dc52d0
...
[ CSSD]2009-03-06 12:06:25.543 [14] >TRACE: clssnmUpdateNodeState: node
1, state (2/3) unique (1236359232/1236359009) prevConuni(1236359009) birth
(114611874/114611874) (old/new) /* the preConuni 'old' and 'new' is the same for a running node
[ CSSD]2009-03-06 12:06:25.543 [14] >TRACE: clssnmUpdateNodeState: node
1, node1, state not changed, eviction pending
[ CSSD]2009-03-06 12:06:25.543 [14] >TRACE: clssnmUpdateNodeState: node
2, state (3/3) unique (1235236386/1235236386) prevConuni(0) birth
(114611867/114611867) (old/new)  /* the preConuni 'old' and 'new' should not be the same for a joining node
...
[ CSSD]2009-03-06 12:06:25.549 [27] >TRACE: clssgmReconfigThread:
started for reconfig (114611874)
[ CSSD]2009-03-06 12:06:25.549 [27] >USER: NMEVENT_RECONFIG

another possible set of traces

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms