After Private Network is Restored, CSS Started But "CRS-2878:Failed to restart resource 'ora.cssd'" Causing CRSD to go Offline (Doc ID 1921160.1)

2014-07-25 11:12:26.510:
[cssd(12284)]CRS-1713:CSSD daemon is started in clustered mode
...
2014-07-25 11:15:44.348:
[cssd(12284)]CRS-1601:CSSD Reconfiguration complete. Active nodes are <nodename1> <nodename2> .
2014-07-25 11:15:45.432:
[ohasd(22341)]CRS-2878:Failed to restart resource 'ora.cssd'  ######## CSSD started and reconfig complete, but reported CSSD failed to start immediately.
2014-07-25 11:15:47.523:
[ctssd(13009)]CRS-2403:The Cluster Time Synchronization Service on host wvfyapp2 is in observer mode.
2014-07-25 11:15:47.927:
[ctssd(13009)]CRS-2407:The new Cluster Time Synchronization Service reference node is host wvfyapp1.
2014-07-25 11:15:47.929:
[ctssd(13009)]CRS-2401:The Cluster Time Synchronization Service started on host wvfyapp2.
2014-07-25 11:15:48.596:
[/oracle/app/11.2.0/grid/bin/oraagent.bin(22560)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "$GRID_HOME/log/wvfyapp2/agent/ohasd/oraagent_grid/oraagent_grid.log"
2014-07-25 11:15:49.512:
[ohasd(22341)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2014-07-25 11:15:49.512:
[ohasd(22341)]CRS-2769:Unable to failover resource 'ora.diskmon'.
2014-07-25 11:15:54.417:
[/oracle/app/11.2.0/grid/bin/oraagent.bin(22560)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "$GRID_HOME/log/<nodename2>/agent/ohasd/oraagent_grid/oraagent_grid.log"
2014-07-25 11:15:59.046:
[ctssd(13009)]CRS-2409:The clock on host <nodename2> is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode.
2014-07-25 11:16:10.714:
[ohasd(22341)]CRS-2878:Failed to restart resource 'ora.crsd'

2014-07-25 11:16:09.940: [ CRSMAIN][1] First attempt: init CSS context succeeded.
[ clsdmt][3]Listening to (ADDRESS=(PROTOCOL=ipc)(KEY=wvfyapp2DBG_CRSD))
2014-07-25 11:16:09.949: [ clsdmt][3]PID for the Process [13147], connkey 1
2014-07-25 11:16:09.950: [ clsdmt][3]Creating PID [13147] file for home $GRID_HOME host <nodename2> bin crs to /oracle/app/11.2.0/grid/crs/init/
2014-07-25 11:16:09.950: [ clsdmt][3]Writing PID [13147] to the file [$GRID_HOME/crs/init/<nodename2>.pid]
2014-07-25 11:16:10.681: [ CRSMAIN][3] Policy Engine is not initialized yet!
2014-07-25 11:16:10.681: [ CRSMAIN][1] CRS Daemon Starting
2014-07-25 11:16:10.682: [ CRSD][1] Logging level for Module: allcomp 0
2014-07-25 11:16:10.682: [ CRSD][1] Logging level for Module: default 0

...

2014-07-25 11:16:10.685: [ CRSD][1] Logging level for Module: OCRASM 1
2014-07-25 11:16:10.685: [ CRSMAIN][1] Checking the OCR device
2014-07-25 11:16:10.686: [ CRSMAIN][1] Sync-up with OCR
2014-07-25 11:16:10.686: [ CRSMAIN][1] Connecting to the CSS Daemon
2014-07-25 11:16:10.686: [ CRSMAIN][1] Getting local node number
2014-07-25 11:16:10.686: [ CRSMAIN][3] Policy Engine is not initialized yet!
2014-07-25 11:16:10.687: [ CRSMAIN][1] Initializing OCR
[ CLWAL][1]clsw_Initialize: OLR initlevel [70000]
2014-07-25 11:16:10.765: [ CRSD][3] CRSD exiting on stop request from default

######## CRSD fails to start.

2014-07-25 11:16:10.765: [ CRSD][3] Done.

2014-07-25 11:15:45.364: [ CRSPE][28]{0:9:16} CRS-2676: Start of 'ora.cssd' on '<nodename2>' succeeded

2014-07-25 11:15:45.432: [ CRSPE][28]{0:9:15} Re-evaluation of queued op [START of [ora.cssd 1 1] on [<nodename2>] : local=1, unplanned=06000000002403830]. found it no longer needed:CRS-2506: Operation on 'START of [ora.cssd 1 1] on [<nodename2>] : local=1, unplanned=06000000002403830' has been cancelled
. Finishing the op.
2014-07-25 11:15:45.432: [ INIT][28]{0:9:15} {0:9:15} Created alert : (:CRSPE00190:) : Resource restart failed!

######## CSSD start succeeded, but reported error CRS-2506 immediately and CSSD failed.

...

2014-07-25 11:16:10.711: [ CRSPE][28]{0:9:16} CRS-2676: Start of 'ora.crsd' on '<nodename2>' succeeded

2014-07-25 11:16:10.714: [ CRSPE][28]{0:35:12} Re-evaluation of queued op [START of [ora.crsd 1 1] on [<nodename2>] : local=1, unplanned=060000000024be760]. found it no longer needed:CRS-2506: Operation on 'START of [ora.crsd 1 1] on [<nodename2>] : local=1, unplanned=060000000024be760' has been cancelled
. Finishing the op.
2014-07-25 11:16:10.714: [ INIT][28]{0:35:12} {0:35:12} Created alert : (:CRSPE00190:) : Resource restart failed!

######## CRSD start succeeded, but reported error CRS-2506 immediately and CRSD failed.