CRS shows Running Instance Status as OFFLINE

(Doc ID 1673397.1)

Last updated on JANUARY 04, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.2 to 11.2.0.3 [Release 11.2]
Information in this document applies to any platform.

Symptoms

Database instance status shows OFFLINE in crsctl stat res -t output, but instance is up and running fine.

$ <GRID_HOME>/bin/crsctl stat res ora.racdb.db -t

--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.racdb.db
     1        OFFLINE OFFLINE                               Corrupted Controlfile
     2        ONLINE  ONLINE       rachost2                 Open
     3        ONLINE  ONLINE       rachost3                 Open


$ <GRID_HOME>/bin/srvctl status database -d racdb
Instance racdb1 is not running on node rachost1
Instance racdb2 is running on node rachost2
Instance racdb3 is running on node rachost3

 
$ sqlplus / as sysdba
...
SQL> select INSTANCE_NAME, STATUS from gv$instance;

INSTANCE_NAME        STATUS
----------------     ------------
racdb1               OPEN
racdb2               OPEN
racdb3               OPEN

The instance alert log possibly shows the following messages during the time of issue:

Hex dump of (file 0, block 39) in trace file /ee/oracle/diag/rdbms/racdb/racdb1/trace/racdb1_arc2_11600080.trc
Corrupt block relative dba: 0x00000027 (file 0, block 39)
Fractured block found during control file block read
Data in bad block:
  type: 21 format: 2 rdba: 0x00000027
  last change scn: 0xffff.00006bd5 seq: 0x1 flg: 0x04
  spare1: 0x0 spare2: 0x0 spare3: 0x0
  consistency value in tail: 0x6bd31501
  check value in block header: 0x1657
  computed block checksum: 0x6
Control file block read for fixed table access has failed due to concurrent controlfile updates.
This is not a real data corruption. Please retry the query.

As the result, database service will not be running on the OFFLINE instance.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms