CRS is not coming up as CTSSD is in OFFLINE status
(Doc ID 2051836.1)
Last updated on FEBRUARY 21, 2024
Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.3 and laterOracle Database Cloud Schema Service - Version N/A and later
Gen 1 Exadata Cloud at Customer (Oracle Exadata Database Cloud Machine) - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Information in this document applies to any platform.
The following entries/logs are for two node cluster, node004, the bad node and node007 where everything is running.
Symptoms
The following example uses two nodes (node004 and node007)
1. The "crsctl stat" on node004, the bad node shows that ctssd and rest of the higher stack, such as evmd,crsd,ASM etc are in OFFLINE status :
------------------
# /u01/app/11.2.0.3/grid/bin/crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE OFFLINE
ora.crf
1 ONLINE OFFLINE
ora.crsd
1 ONLINE OFFLINE
ora.cssd
1 ONLINE ONLINE node004
ora.cssdmonitor
1 ONLINE ONLINE node004
ora.ctssd
1 ONLINE OFFLINE
ora.diskmon
1 ONLINE ONLINE node004
ora.evmd
1 ONLINE OFFLINE
ora.gipcd
1 ONLINE ONLINE node004
ora.gpnpd
1 ONLINE ONLINE node004
ora.mdnsd
1 ONLINE ONLINE node004
root@node004:~#
-----------------
2. alertnode004.log (node where CRS is not coming up) :
------------
2015-08-31 15:54:22.595:
[cssd(5694)]CRS-1713:CSSD daemon is started in clustered mode
2015-08-31 15:54:27.827:
[ohasd(5349)]CRS-2765:Resource 'ora.crf' has failed on server 'node004'.
.
2015-08-31 15:55:02.343:
[cssd(5694)]CRS-1601:CSSD Reconfiguration complete. Active nodes are node004 node007 .
2015-08-31 15:55:04.934:
[ctssd(6094)]CRS-2407:The new Cluster Time Synchronization Service reference node is host node007. <<=============== Reference or master node of CTSSD
2015-08-31 15:55:04.935:
[ctssd(6094)]CRS-2401:The Cluster Time Synchronization Service started on host node004.
2015-08-31 15:55:08.975:
[ctssd(6094)]CRS-2402:The Cluster Time Synchronization Service aborted on host node004. Details at in /u01/app/11.2.0.3/grid/log/node004/ctssd/octssd.log.
---------------
3. NTPD is being used to sync up the clock on the cluster, and its running on the other(good) node, node007(see above) which is the master of CTSSD. So CTSSD is running in OBSERVER mode on node007.
The octssd.log of the bad node, node004 shows that it can't detect the vendor time sync software as "ntp.conf" doesn't exist :
-----------------
2015-08-31 15:55:09.688: [ CTSS][1]Oracle Database CTSS Release 11.2.0.3.0 Production Copyright 2006, 2011 Oracle. All rights reserved.
2015-08-31 15:55:10.653: [ CTSS][1]sclsctss_gvss8: Return [0] and NTP status [1].
2015-08-31 15:55:10.653: [ CTSS][1]ctss_check_vendor_sw: Vendor time sync software is not detected. status [1]. <<=============== NTPD or vendor time sync software was not detected.
-----------------
Changes
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |
| References |