rootcrs.sh -postpatch failed to start cssd
(Doc ID 2525624.1)
Last updated on JULY 20, 2024
Applies to:
Oracle Database - Enterprise Edition - Version 12.1.0.2 and laterInformation in this document applies to any platform.
Symptoms
rootcrs.sh -postpatch failed to start cssd.
# ./rootcrs.sh -postpatch
Using configuration parameter file: <GI_HOME>/crs/install/crsconfig_params
2019/02/25 11:33:46 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2019/02/25 11:33:59 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2019/02/25 11:34:00 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.conf'
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '<HOST1>'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on '<HOST1>'
CRS-2677: Stop of 'ora.drivers.acfs' on '<HOST1>' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '<HOST1>' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on '<HOST1>'
CRS-2672: Attempting to start 'ora.evmd' on '<HOST1>'
CRS-2676: Start of 'ora.evmd' on '<HOST1>' succeeded
CRS-2676: Start of 'ora.mdnsd' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on '<HOST1>'
CRS-2676: Start of 'ora.gpnpd' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on '<HOST1>'
CRS-2676: Start of 'ora.gipcd' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on '<HOST1>'
CRS-2676: Start of 'ora.cssdmonitor' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.cssd' on '<HOST1>'
CRS-2672: Attempting to start 'ora.diskmon' on '<HOST1>'
CRS-2676: Start of 'ora.diskmon' on '<HOST1>' succeeded
CRS-2883: Resource 'ora.cssdmonitor' failed during Clusterware stack start.
CRS-4406: Oracle High Availability Services synchronous start failed.
CRS-4000: Command Start failed, or completed with errors.
2019/02/25 11:45:17 CLSRSC-117: Failed to start Oracle Clusterware stack
Died at <GI_HOME>/crs/install/crspatch.pm line 941.
The command '<GI_HOME>/perl/bin/perl -I<GI_HOME>/perl/lib -I<GI_HOME>/crs/install <GI_HOME>/crs/install/rootcrs.pl -postpatch' execution failed
#
Using configuration parameter file: <GI_HOME>/crs/install/crsconfig_params
2019/02/25 11:33:46 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2019/02/25 11:33:59 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2019/02/25 11:34:00 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.conf'
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '<HOST1>'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on '<HOST1>'
CRS-2677: Stop of 'ora.drivers.acfs' on '<HOST1>' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '<HOST1>' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on '<HOST1>'
CRS-2672: Attempting to start 'ora.evmd' on '<HOST1>'
CRS-2676: Start of 'ora.evmd' on '<HOST1>' succeeded
CRS-2676: Start of 'ora.mdnsd' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on '<HOST1>'
CRS-2676: Start of 'ora.gpnpd' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on '<HOST1>'
CRS-2676: Start of 'ora.gipcd' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on '<HOST1>'
CRS-2676: Start of 'ora.cssdmonitor' on '<HOST1>' succeeded
CRS-2672: Attempting to start 'ora.cssd' on '<HOST1>'
CRS-2672: Attempting to start 'ora.diskmon' on '<HOST1>'
CRS-2676: Start of 'ora.diskmon' on '<HOST1>' succeeded
CRS-2883: Resource 'ora.cssdmonitor' failed during Clusterware stack start.
CRS-4406: Oracle High Availability Services synchronous start failed.
CRS-4000: Command Start failed, or completed with errors.
2019/02/25 11:45:17 CLSRSC-117: Failed to start Oracle Clusterware stack
Died at <GI_HOME>/crs/install/crspatch.pm line 941.
The command '<GI_HOME>/perl/bin/perl -I<GI_HOME>/perl/lib -I<GI_HOME>/crs/install <GI_HOME>/crs/install/rootcrs.pl -postpatch' execution failed
#
ocssd.trc
2019-02-25 11:35:17.872015 : CSSD:2777722336: clssscqueue_init: queue(0x14e1870), max(0)
2019-02-25 11:35:18.613306 : CSSD:2773931776: clssscagSelect: endpoint(0x64) authenticated with user(root)
2019-02-25 11:35:18.613392 : CSSD:2773931776: clssscagProcessInitialMsg: Handshake successful with agent 1
2019-02-25 11:35:18.613435 : CSSD:2773931776: clssscagProcAgReq: got a successful connection
2019-02-25 11:35:18.613489 : CSSD:2773931776: clssscagProcessInitialMsg: notify agent 1 that it is active
2019-02-25 11:35:18.613518 : CSSD:2773931776: clssscagProcessInitialMsg: connection from agent 1, endp 0x64 - agents joined 0
2019-02-25 11:35:18.790005 : CSSD:2773931776: clssscagSelect: endpoint(0x53) authenticated with user(root)
2019-02-25 11:35:18.790083 : CSSD:2773931776: clssscagProcessInitialMsg: Handshake successful with agent 0
2019-02-25 11:35:18.790088 : CSSD:2773931776: clssscagProcAgReq: got a successful connection
2019-02-25 11:35:18.790114 : CSSD:2773931776: clssscagProcessInitialMsg: connection from agent 0, endp 0x53 - agents joined 0
2019-02-25 11:45:16.876378 : CSSD:2773931776: clssscagProcAgReq: shutdown abort requested by the agent
2019-02-25 11:45:16.876397 : CSSD:2773931776: ###################################
2019-02-25 11:45:16.876400 : CSSD:2773931776: clssscExit: CSSD aborting from thread clssscAgListener
2019-02-25 11:45:16.876403 : CSSD:2773931776: ###################################
2019-02-25 11:45:16.876405 : CSSD:2773931776: (:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally
2019-02-25 11:45:16.876508 : CSSD:2773931776: ####### Begin Diagnostic Dump #######
2019-02-25 11:45:16.876512 : CSSD:2773931776: ### Begin diagnostic data for the Core layer ###
2019-02-25 11:45:16.876516 : CSSD:2773931776: Initialization state clssscInitNODENUM (0x00000001) not set
2019-02-25 11:45:16.876519 : CSSD:2773931776: Initialization state clssscInitMAINCTX_DONE (0x00000002) not set
2019-02-25 11:45:16.876521 : CSSD:2773931776: Initialization state clssscInitPROF_PARMS (0x00000004) not set
2019-02-25 11:45:16.876523 : CSSD:2773931776: Initialization state clssscInitSKGXN_DONE (0x00000008) not set
2019-02-25 11:45:16.876526 : CSSD:2773931776: Initialization state clssscInitKGZF_DONE (0x00000010) not set
2019-02-25 11:45:16.876528 : CSSD:2773931776: Initialization state clssscInitGMP_ENDPT (0x00000020) not set
2019-02-25 11:45:16.876530 : CSSD:2773931776: Initialization state clssscInitNM_MIN (0x00000040) not set
2019-02-25 11:45:16.876532 : CSSD:2773931776: Initialization state clssscInitGMC_CLNT_EP (0x00000080) not set
2019-02-25 11:45:16.876535 : CSSD:2773931776: Initialization state clssscInitNM_COMPL (0x00000100) not set
2019-02-25 11:45:16.876537 : CSSD:2773931776: Initialization state clssscInitGMP_MIN (0x00000200) not set
2019-02-25 11:45:16.876539 : CSSD:2773931776: Initialization state clssscInitBNMS_COMPL (0x00000400) not set
2019-02-25 11:45:16.876541 : CSSD:2773931776: Initialization state clssscInitBNMC_COMPL (0x00000800) not set
2019-02-25 11:45:16.876543 : CSSD:2773931776: Initialization state clssscInitALARM_DONE (0x00001000) not set
2019-02-25 11:45:16.876546 : CSSD:2773931776: Initialization state clssscInitCTRL_COMPL (0x00002000) not set
2019-02-25 11:45:16.876557 : CSSD:2773931776: Initialization state clssscInitFRST_RCFG (0x00004000) not set
2019-02-25 11:45:16.876559 : CSSD:2773931776: Initialization state clssscInitBCNM_READY (0x00008000) not set
2019-02-25 11:45:16.876561 : CSSD:2773931776: Initialization state clssscInitGMC_READY (0x00010000) not set
2019-02-25 11:45:16.876563 : CSSD:2773931776: Initialization state clssscInitHAVE_DBINFO (0x00020000) not set
2019-02-25 11:45:16.876565 : CSSD:2773931776: Initialization state clssscInitGNS_READY (0x00040000) not set
2019-02-25 11:45:16.876568 : CSSD:2773931776: Initialization state clssscInitCLSFA (0x00100000) not set
2019-02-25 11:45:16.876570 : CSSD:2773931776: Initialization state clssscInitHAVE_ICIN (0x00200000) not set
2019-02-25 11:45:16.876572 : CSSD:2773931776: Initialization state clssscInitCOMPLETE (0x00400000) not set
2019-02-25 11:45:16.876574 : CSSD:2773931776: Initialization not complete !Error!
2019-02-25 11:45:16.876576 : CSSD:2773931776: #### End diagnostic data for the Core layer ####
2019-02-25 11:45:16.876579 : CSSD:2773931776: ######## End Diagnostic Dump ########
2019-02-25 11:45:16.876581 : CSSD:2773931776: clssscExit: closing monitor listening endpoint
CLSB:2773931776: Oracle Clusterware infrastructure error in OCSSD (OS PID 10749): Fatal signal 6 has occurred in program ocssd thread 2773931776; nested signal count is 1
Trace file <ORACLE_BASE>/diag/crs/<HOST1>/crs/trace/ocssd.trc
Oracle Database 12c Clusterware Release 12.1.0.2.0 - Production Copyright 1996, 2014 Oracle. All rights reserved.
DDE: Flood control is not active
Incident 9 created, dump file: <ORACLE_BASE>/diag/crs/<HOST1>/crs/incident/incdir_9/ocssd_i9.trc
CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []
2019-02-25 11:35:18.613306 : CSSD:2773931776: clssscagSelect: endpoint(0x64) authenticated with user(root)
2019-02-25 11:35:18.613392 : CSSD:2773931776: clssscagProcessInitialMsg: Handshake successful with agent 1
2019-02-25 11:35:18.613435 : CSSD:2773931776: clssscagProcAgReq: got a successful connection
2019-02-25 11:35:18.613489 : CSSD:2773931776: clssscagProcessInitialMsg: notify agent 1 that it is active
2019-02-25 11:35:18.613518 : CSSD:2773931776: clssscagProcessInitialMsg: connection from agent 1, endp 0x64 - agents joined 0
2019-02-25 11:35:18.790005 : CSSD:2773931776: clssscagSelect: endpoint(0x53) authenticated with user(root)
2019-02-25 11:35:18.790083 : CSSD:2773931776: clssscagProcessInitialMsg: Handshake successful with agent 0
2019-02-25 11:35:18.790088 : CSSD:2773931776: clssscagProcAgReq: got a successful connection
2019-02-25 11:35:18.790114 : CSSD:2773931776: clssscagProcessInitialMsg: connection from agent 0, endp 0x53 - agents joined 0
2019-02-25 11:45:16.876378 : CSSD:2773931776: clssscagProcAgReq: shutdown abort requested by the agent
2019-02-25 11:45:16.876397 : CSSD:2773931776: ###################################
2019-02-25 11:45:16.876400 : CSSD:2773931776: clssscExit: CSSD aborting from thread clssscAgListener
2019-02-25 11:45:16.876403 : CSSD:2773931776: ###################################
2019-02-25 11:45:16.876405 : CSSD:2773931776: (:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally
2019-02-25 11:45:16.876508 : CSSD:2773931776: ####### Begin Diagnostic Dump #######
2019-02-25 11:45:16.876512 : CSSD:2773931776: ### Begin diagnostic data for the Core layer ###
2019-02-25 11:45:16.876516 : CSSD:2773931776: Initialization state clssscInitNODENUM (0x00000001) not set
2019-02-25 11:45:16.876519 : CSSD:2773931776: Initialization state clssscInitMAINCTX_DONE (0x00000002) not set
2019-02-25 11:45:16.876521 : CSSD:2773931776: Initialization state clssscInitPROF_PARMS (0x00000004) not set
2019-02-25 11:45:16.876523 : CSSD:2773931776: Initialization state clssscInitSKGXN_DONE (0x00000008) not set
2019-02-25 11:45:16.876526 : CSSD:2773931776: Initialization state clssscInitKGZF_DONE (0x00000010) not set
2019-02-25 11:45:16.876528 : CSSD:2773931776: Initialization state clssscInitGMP_ENDPT (0x00000020) not set
2019-02-25 11:45:16.876530 : CSSD:2773931776: Initialization state clssscInitNM_MIN (0x00000040) not set
2019-02-25 11:45:16.876532 : CSSD:2773931776: Initialization state clssscInitGMC_CLNT_EP (0x00000080) not set
2019-02-25 11:45:16.876535 : CSSD:2773931776: Initialization state clssscInitNM_COMPL (0x00000100) not set
2019-02-25 11:45:16.876537 : CSSD:2773931776: Initialization state clssscInitGMP_MIN (0x00000200) not set
2019-02-25 11:45:16.876539 : CSSD:2773931776: Initialization state clssscInitBNMS_COMPL (0x00000400) not set
2019-02-25 11:45:16.876541 : CSSD:2773931776: Initialization state clssscInitBNMC_COMPL (0x00000800) not set
2019-02-25 11:45:16.876543 : CSSD:2773931776: Initialization state clssscInitALARM_DONE (0x00001000) not set
2019-02-25 11:45:16.876546 : CSSD:2773931776: Initialization state clssscInitCTRL_COMPL (0x00002000) not set
2019-02-25 11:45:16.876557 : CSSD:2773931776: Initialization state clssscInitFRST_RCFG (0x00004000) not set
2019-02-25 11:45:16.876559 : CSSD:2773931776: Initialization state clssscInitBCNM_READY (0x00008000) not set
2019-02-25 11:45:16.876561 : CSSD:2773931776: Initialization state clssscInitGMC_READY (0x00010000) not set
2019-02-25 11:45:16.876563 : CSSD:2773931776: Initialization state clssscInitHAVE_DBINFO (0x00020000) not set
2019-02-25 11:45:16.876565 : CSSD:2773931776: Initialization state clssscInitGNS_READY (0x00040000) not set
2019-02-25 11:45:16.876568 : CSSD:2773931776: Initialization state clssscInitCLSFA (0x00100000) not set
2019-02-25 11:45:16.876570 : CSSD:2773931776: Initialization state clssscInitHAVE_ICIN (0x00200000) not set
2019-02-25 11:45:16.876572 : CSSD:2773931776: Initialization state clssscInitCOMPLETE (0x00400000) not set
2019-02-25 11:45:16.876574 : CSSD:2773931776: Initialization not complete !Error!
2019-02-25 11:45:16.876576 : CSSD:2773931776: #### End diagnostic data for the Core layer ####
2019-02-25 11:45:16.876579 : CSSD:2773931776: ######## End Diagnostic Dump ########
2019-02-25 11:45:16.876581 : CSSD:2773931776: clssscExit: closing monitor listening endpoint
CLSB:2773931776: Oracle Clusterware infrastructure error in OCSSD (OS PID 10749): Fatal signal 6 has occurred in program ocssd thread 2773931776; nested signal count is 1
Trace file <ORACLE_BASE>/diag/crs/<HOST1>/crs/trace/ocssd.trc
Oracle Database 12c Clusterware Release 12.1.0.2.0 - Production Copyright 1996, 2014 Oracle. All rights reserved.
DDE: Flood control is not active
Incident 9 created, dump file: <ORACLE_BASE>/diag/crs/<HOST1>/crs/incident/incdir_9/ocssd_i9.trc
CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []
Changes
Applying GIPSU to GI_HOME and missed to DBPSU in GI home caused this issue.
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |
| References |