ORA-29770 : LMHB crashing database instance waiting for redo allocation (Doc ID 2153154.1)

Last updated on JULY 10, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.0 and later
Information in this document applies to any platform.

Symptoms

RAC database instance crashed due to ORA-29770 while LMS was blocked due to redo allocation.

 Database instance alertlog file,

Sat Sep 26 16:12:48 2015
LMS2 (ospid: 3605652) has not called a wait for sub 0 secs.
Errors in file
/oracle/app/oracle/diag/rdbms/thradb/thradb1/trace/thradb1_lmhb_3606190.trc
(incident=1840265):
ORA-29770: global enqueue process LMS2 (OSID 3605652) is hung for more than
70 seconds
Incident details in:
/oracle/app/oracle/diag/rdbms/thradb/thradb1/incident/incdir_1840265/thradb1_l
mhb_3606190_i1840265.trc
LOCK_DBGRP: GCR_SYSTEST debug event locked group GR+DB_THRADB by memno 1
ERROR: Some process(s) is not making progress.
LMHB (ospid: 3606190) is terminating the instance.
Please check LMHB trace file for more details.
Please also check the CPU load, I/O load and other system properties for
anomalous behavior
ERROR: Some process(s) is not making progress.
LMHB (ospid: 3606190): terminating the instance due to error 29770
Sat Sep 26 16:12:49 2015

 

 Database instance diag trace file from the hour of issue,


1: waited for 'latch: redo allocation'
wait times: snap=0.047113 sec, exc=0.047113 sec, total=0.047113 sec
occurred after 0.000578 sec of elapsed time
2: waited for 'latch: redo allocation'
wait times: snap=0.000370 sec, exc=0.000370 sec, total=0.000370 sec
occurred after 0.047350 sec of elapsed time
3: waited for 'latch: redo allocation'
wait times: snap=0.000702 sec, exc=0.000702 sec, total=0.000702 sec
occurred after 0.000264 sec of elapsed time
4: waited for 'latch: redo allocation'
wait times: snap=0.002430 sec, exc=0.002430 sec, total=0.002430 sec
occurred after 0.000578 sec of elapsed time
5: waited for 'latch: redo allocation'
wait times: snap=0.018627 sec, exc=0.018627 sec, total=0.018627 sec
occurred after 0.000670 sec of elapsed time
6: waited for 'latch: redo allocation'
wait times: snap=0.002851 sec, exc=0.002851 sec, total=0.002851 sec
occurred after 0.000334 sec of elapsed time

.

Database instance LMS trace file from the hour of issue,

.
The LMS2 short stack is:
.
Short stack dump:
ksedsts()+712<-ksdxfstk()+44<-ksdxcb()+896<-sspuser()+220<-__sighandler()<-thr
ead_wait()+556<-sskgpwwait()+52<-skgpwwait()+268<-kslges()+2348<-kslgetl()+928
<-kcrfw_redo_gen_ext()+5740<-kcbchg1_main()+1344<-kcbchg1()+284<-ktbgfc()+604<
-ktbgcl1()+5252<-ktrgcm()+12968<-ktrget2()+924<-kclgeneratecr_main()+8728<-kcl
generatecr()+1744<-kclgcr()+1252<-kclcrrf()+936<-kjblcrcbk()+1032<-kjbrscrping
()+6032<-kjbrcrcvt()+7364<-kjbmpocr()+15852<-kjmxmpm()+724<-kjmpmsgi()+4520<-k
jmsm()+6164<-ksbrdp()+3032<-opirip()+1540<-opidrv()+1124<-sou2o()+240<-opimai_
real()+292<-ssthrdmain()+452<-main()+208<-__start()+112

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms