Database Instance Crashes with ORA-15064 ORA-03135 ORA-00240 on 11.2 (Doc ID 1487108.1)

Last updated on MAY 08, 2017

Applies to:

Oracle Database - Enterprise Edition - Version 11.1.0.6 to 11.2.0.4 [Release 11.1 to 11.2]
Information in this document applies to any platform.
Checked on 14-Jun-2013

Symptoms

You are on 11.2 and database instance is using ASM. You see the database instance crashing ( frequently ) and observe the following errors in database alert.log

 

 

ORA-00240: control file enqueue held for more than 120 seconds

NOTE: ASMB terminating

ORA-15064: communication failure with ASM instance
ORA-03135: connection lost contact
Process ID:
Session ID: 289 Serial number: 23
Errors in file /opt/oracledb/base/diag/rdbms/prodrep/prodrep1/trace/prodrep1_asmb_25047.trc:
ORA-15064: communication failure with ASM instance
ORA-03135: connection lost contact

System state dump requested by (instance=1, osid=25047 (ASMB)), summary=[abnormal instance termination].
ASMB (ospid: 25047): terminating the instance due to error 15064
System State dumped to trace file /opt/oracledb/base/diag/rdbms/prodrep/prodrep1/trace/prodrep1_diag_25011.trc
Sun Aug 26 03:16:47 2012
ORA-1092 : opitsk aborting process
Sun Aug 26 03:16:47 2012
ORA-1092 : opitsk aborting process
Sun Aug 26 03:16:47 2012
License high water mark = 43
USER (ospid: 28997): terminating the instance
Instance terminated by USER, pid = 28997

 

 

 

In the systemstate dump you might see the shared pool latch being held and waited for

 

 

  0: waiting for 'row cache cleanup'
  =0x0, =0x0, =0x0
  wait_id=12204573 seq_num=11593 snap_id=1
  wait times: snap=0.000000 sec, exc=0.000087 sec, total=2 min 11 sec
  wait times: max=infinite, heur=2 min 11 sec
  wait counts: calls=0 os=0
  in_wait=1 iflags=0x15a2
  There is at least one session blocking this session.
  Dumping 1 direct blocker(s):
  inst: 1, sid: 6, ser: 1
  Dumping final blocker:
  inst: 1, sid: 6, ser: 1
  Wait State:
  fixed_waits=0 flags=0x22 boundary=0x0000000000000000/-1

 

 

PROCESS 24: ASMB
  ----------------------------------------
  SO: 0xc0000002fafc6890, type: 2, owner: 0x0000000000000000, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
  proc=0xc0000002fafc6890, name=process, file=ksu.h LINE:12616 ID:, pg=0
  (process) Oracle pid:24, ser:1, calls cur/top: 0xc0000002f75132d8/0xc0000002f75132d8
  flags : (0x6) SYSTEM
  flags2: (0x0), flags3: (0x10)
  intr error: 0, call error: 0, sess error: 0, txn error 0
  intr queue: empty
  ksudlp FALSE at location: 0
  (post info) last post received: 0 0 35
  last post received-location: ksr2.h LINE:627 ID:ksrpublish
  last process to post me: c0000002fafcdd60 1 0
  last post sent: 1060 0 2
  last post sent-location: ksl2.h LINE:2362 ID:kslpsr
  last process posted by me: c0000002fafc25d0 1 6
  (latch info) wait_event=0 bits=0
  Process Group: DEFAULT, pseudo proc: 0xc0000002f96a6980
  O/S info: user: oracledb, term: UNKNOWN, ospid: 25047
  OSD pid info: Unix process pid: 25047, image: oracleindigo4 (ASMB)

 

  SO: 0xc0000002f9d47688, type: 4, owner: 0xc0000002fafc6890, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
  proc=0xc0000002fafc6890, name=session, file=ksu.h LINE:12624 ID:, pg=0
  (session) sid: 6 ser: 1 trans: 0x0000000000000000, creator: 0xc0000002fafc6890
  flags: (0x51) USR/- flags_idl: (0x1) BSY/-/-/-/-/-
  flags2: (0x409) -/-/INC
  DID: , short-term DID:
  txn branch: 0x0000000000000000
  oct: 0, prv: 0, sql: 0x0000000000000000, psql: 0x0000000000000000, user: 0/SYS
  ksuxds FALSE at location: 0
  service name: SYS$BACKGROUND
  Current Wait Stack:
  Not in wait; last wait ended 0.787339 sec ago
  There are 17 sessions blocked by this session.
  Dumping one waiter:
  inst: 1, sid: 7, ser: 3
  wait event: 'latch: shared pool'
  p1: 'address'=0xc000000072ee3c80
  p2: 'number'=0x133
  p3: 'tries'=0x0
  row_wait_obj#: 4294967295, block#: 0, row#: 0, file# 0
  min_blocked_time: 279 secs, waiter_cache_ver: 22472

 

 

 

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms