Sessions Waiting On Enq: CT – State, Instance Aborted with ora-600 [krccic_2] when disabling BCT

(Doc ID 2384566.1)

Last updated on MAY 10, 2018

Applies to:

Oracle Database - Enterprise Edition - Version 12.1.0.2 and later
Information in this document applies to any platform.

Symptoms

 Rman job hung, sessions were waiting on enq: CT – state, the physical standby instance got terminated with ora-600 [krccic_2] when customer disabled block change tracking.

 

Standby alert log 

-------------------

  alter database disable block change tracking

  Tue Mar 06 12:21:44 2018

  CKPT (ospid: 98671) waits for event 'enq: CT - state' for 73 secs.

  Tue Mar 06 12:23:26 2018

  Block change tracking service stopping.

  Tue Mar 06 12:23:26 2018

  Stopping background process CTWR

  Errors in file /u01/app/oracle/diag/rdbms/cmaltqa/cmaltqa1/trace/cmaltqa1_ckpt_98671.trc (incident=335859) (PDBNAME=CDB$ROOT):

  ORA-00600: internal error code, arguments: [krccic_2], [], [], [], [], [], [], [], [], [], [], []

  Incident details in: /u01/app/oracle/diag/rdbms/cmaltqa/cmaltqa1/incident/incdir_335859/cmaltqa1_ckpt_98671_i335859.trc

  Use ADRCI or Support Workbench to package the incident.

  See Note 411.1 at My Oracle Support for error and packaging details.

  Tue Mar 06 12:23:33 2018

  Errors in file /u01/app/oracle/diag/rdbms/cmaltqa/cmaltqa1/trace/cmaltqa1_ckpt_98671.trc:

  ORA-00600: internal error code, arguments: [krccic_2], [], [], [], [], [], [], [], [], [], [], []

  Tue Mar 06 12:23:33 2018

  USER (ospid: 98671): terminating the instance due to error 469

 

rman_20180305_0319_lvl_1d_cmaltqa.out

-------------------------------------------------

  channel D14: backup set complete, elapsed time: 00:18:23

  channel D14: starting incremental level 1 datafile backup set

  channel D14: specifying datafile(s) in backup set

  input datafile file number=00041 name=/u02/app/oracle/oradata/datcmaltqa/cmaltqa/CMALTQA/280D1F6DB3A92A50E0534A070D0AFB91/datafile/o1_mf_syn_data_ds902fo3_.dbf

  backing up blocks 96468993 through 100663296

  channel D14: starting piece 24 at 2018-03-05 03:44:04

  RMAN-08606: WARNING: The change tracking file is invalid.

 

cmaltqa1_ckpt_98671_i335859.trc

---------------------------------------------------

[TOC00000]

Jump to table of contents

Dump continued from file: /u01/app/oracle/diag/rdbms/cmaltqa/cmaltqa1/trace/cmaltqa1_ckpt_98671.trc

[TOC00001]

ORA-00600: internal error code, arguments: [krccic_2], [], [], [], [], [], [], [], [], [], [], []

[TOC00001-END]

[TOC00002]

========= Dump for incident 335859 (ORA 600 [krccic_2]) ========

*** 2018-03-06 12:23:33.165

dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)

[TOC00003]

----- SQL Statement (None) -----

Current SQL information unavailable - no cursor.

[TOC00003-END]

[TOC00004]

----- Call Stack Trace -----

kgerinv_internal() kgerinv() kgeasnmierr() krccic() ksbciccb() ksbmsg() ksbcti() ksbabs() ksbrdp()

 

Changes

  No changes, the ora-600 [krccic_2] occurred when customer disabled block change tracking and sessions waited for event 'enq: CT - state' .

This is caused by bug 19297917 - DRA: RMAN VALIDATE DATABASE HANG WAITING ON CT ENQ.

RELEASE NOTES: 

  On a bad BCT file scenario, backup/validate can sometimes hang as backup processes are sharing enqueues and one channel tries to get exclusive

access to the change tracking enqueue. As rman cleanup of channels is NOT async, it ends up waiting indefinitely.

  The fix is not to wait forever on the change tracking enqueue, timeout and the last channel will surely succeed.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms