Frequent Instance termination by a user process in Data Guard environment: Error: kccpb_sanity_check_2 Control file sequence number mismatch! (Doc ID 1593043.1)

Last updated on FEBRUARY 11, 2015

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.2 to 12.1.0.1 [Release 11.2 to 12.1]
Information in this document applies to any platform.

Symptoms

On a RAC standby database, 1 instance gets terminated frequently by a user process.

alert log shows:

Mon Oct 07 07:00:17 2013
USER (ospid: 11948): terminating the instance
Mon Oct 07 07:00:17 2013
System state dump requested by (instance=1, osid=11948), summary=[abnormal instance termination].
System State dumped to trace file /u02/app/oracle/admin/diag/rdbms/rac_clust/RACDB1/trace/RACDB1_diag_20415.trc
Dumping diagnostic data in directory=[cdmp_20131007070017], requested by (instance=1, osid=11948), summary=[abnormal instance termination].
Instance terminated by USER, pid = 11948
...
Mon Oct 07 11:00:16 2013
USER (ospid: 30048): terminating the instance
Mon Oct 07 11:00:16 2013
System state dump requested by (instance=1, osid=30048), summary=[abnormal instance termination].
System State dumped to trace file /u02/app/oracle/admin/diag/rdbms/rac_clust/RACDB1/trace/RACDB1_diag_13380.trc
Mon Oct 07 11:00:17 2013
ORA-1092 : opitsk aborting process
Mon Oct 07 11:00:17 2013
License high water mark = 97
Dumping diagnostic data in directory=[cdmp_20131007110016], requested by (instance=1, osid=30048), summary=[abnormal instance termination].
Instance terminated by USER, pid = 30048
USER (ospid: 30594): terminating the instance
Instance terminated by USER, pid = 30594
...
Mon Oct 07 15:00:21 2013
USER (ospid: 29261): terminating the instance
Mon Oct 07 16:00:21 2013
System state dump requested by (instance=1, osid=29261), summary=[abnormal instance termination].
System State dumped to trace file /u02/app/oracle/admin/diag/rdbms/rac_clust/RACDB1/trace/RACDB1_diag_29917.trc
Dumping diagnostic data in directory=[cdmp_20131007160021], requested by (instance=1, osid=29261), summary=[abnormal instance termination].
Instance terminated by USER, pid = 29261

 

The user process trace shows:

RACDB1_ora_11948.trc
----------
*** CLIENT ID:() 2013-10-07 07:00:17.417
*** SERVICE NAME:(SYS$USERS) 2013-10-07 07:00:17.417
*** MODULE NAME:(rman@racn1 (TNS V1-V3)) 2013-10-07 07:00:17.417
*** ACTION NAME:(0000014 STARTED114) 2013-10-07 07:00:17.417

Error: kccpb_sanity_check_2
Control file sequence number mismatch!
fhcsq: 826343258 bhcsq: 826350398 cfn 8
kjzduptcctx: Notifying DIAG for crash event
----- Abridged Call Stack Trace -----
ksedsts()+461<-kjzdssdmp()+267<-kjzduptcctx()+232<-kjzdicrshnfy()+53<-ksuitm()+1332<-kccpb_sanity_check()+341<-kccbmp_get()+309<-kccsed_rbl()+111<-kcc_begin_alt()+674<-kccbckrm()+6356<-kccbck()+37<-kccmus()+1210<-krasear()+3128<-krbiragf()+644<-pevm_icd_call_common()+867
<-pfrinstr_ICAL()+168<-pfrrun_no_tool()+63<-pfrrun()+627<-plsql_run()+649<-pricar()+1003
----- End of Abridged Call Stack Trace -----

*** 2013-10-07 07:00:17.460
USER (ospid: 11948): terminating the instance
ksuitm: waiting up to [5] seconds before killing DIAG(20415)

  

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms