ASM rebalance is getting hung state after running for sometime on RAC (Doc ID 2189396.1)

Last updated on OCTOBER 03, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 12.1.0.2 and later
Information in this document applies to any platform.

Symptoms

While adding disk in RAC ,after running for sometime it went into hung state and ARBn trace file does not show anymore allocation movement.

ARBn process was stuck .

 

INST_ID GROUP_NUMBER OPERA PASS STAT POWER ACTUAL SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE CON_ID
---------- ------------ ----- --------- ---- ---------- ---------- ---------- ---------- ---------- ----------- -------------------------------------------- ----------
2 7 REBAL REBALANCE WAIT 50 0
2 7 REBAL COMPACT WAIT 50 0
1 7 REBAL REBALANCE RUN 50 50 5072690 10920833 9887 591 0
1 7 REBAL COMPACT WAIT 50 50 0 0 0 0 0
3 7 REBAL REBALANCE WAIT 50 0
3 7 REBAL COMPACT WAIT 50 0

6 rows selected.

 

*** 2016-09-27 06:02:23.186
PROCESS 57: ARB0
----------------------------------------
SO: 0x7000100be682270, type: 2, owner: 0x0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3
proc=0x7000100be682270, name=process, file=ksu.h LINE:13950 ID:, pg=0 conuid=0
(process) Oracle pid:57, ser:18, calls cur/top: 0x7000100b5487c58/0x7000100b547f410
flags : (0x2) SYSTEM icon_uid:0
flags2: (0x0), flags3: (0x10)
intr error: 0, call error: 0, sess error: 0, txn error 0
intr queue: empty
ksudlp FALSE at location: 0
(post info) last post received: 0 0 130
last post received-location: kji.h LINE:4844 ID:kjxmhmackh: wake up FG/ARB waiting for LMD
last process to post me: 0x7000100be662560 1 6
last post sent: 0 0 149
last post sent-location: kjm.h LINE:1696 ID:kjmpost: post lmd
last process posted by me: 0x7000100be662560 1 6
waiter on post event: 0
(latch info) hold_bits=0x0
Process Group: DEFAULT, pseudo proc: 0x7000100be6c8400
O/S info: user: oracle, term: UNKNOWN, ospid: 19136588
OSD pid info:
Short stack dump:
ksedsts()+712<-ksdxfstk()+44<-ksdxcb()+896<-sspuser()+220<-__sighandler()<-thread_wait()+580<-sskgpwwait()+52<-skgpwwait()+268<-ksliwat()+7912<-kslwaitctx()+192<-kjupack()+1392<-kjusuc()+5816<-ksipgetctxia()+2020<-ksqcmi()+11504<-ksqgtlctx()+4252<-ksqgelctx()+508<-kfdGetAuEnqPriv()+1096<-kffsrdGet()+964<-kfdvaPopLock()+52<-kfdvaOpLock()+792<-kffRelocateClientData()+3052<-kffRelocateSubmit()+228<-kffRelocate()+3056<-kfdaExecute()+2104<-kfgbRebalExecute()+440<-kfgbDriver()+1824<-ksbabs()+4088<-kfgbRun()+556<-ksbrdp()+3032<-opirip()+1540<-opidrv()+1124<-sou2o()+240<-opimai_real()+292<-ssthrdmain()+452<-main()+208<-__start()+112
----------------------------------------
SO: 0x7000100bda365f8, type: 27, owner: 0x7000100be682270, flag: INIT/-/-/0x00 if: 0x1 c: 0x1
proc=0x7000100be682270, name=ksir PrivOp State Object, file=ksir2.h LINE:534 ID:, pg=0 conuid=0
(ksirso) Stage = 0 rmvsn = 2.0.0.0.0
Operation currently on the privileged list are
next_ksirtopnd = [7000100b5612fa8,7000100bda2d6c8]

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms