Node 1 asm instance not coming up due to fail rebalance from other node with wait: enq: AM - ASM file relocation

(Doc ID 1489778.1)

Last updated on OCTOBER 31, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.3 and later
Information in this document applies to any platform.

Symptoms

Node 1 ASM instance is not coming up, and throwing below error:

Thu Sep 06 00:38:43 2012 
LCK0 started with pid=22, OS id=323584
Thu Sep 06 01:21:58 2012 
sculkget: failed to lock /crs11g/grid/11.2.0.3/dbs/lkinst+ASM1 exclusive
sculkget: lock held by PID: 442500
Oracle Instance Shutdown operation failed. Another process may be attempting to startup or shutdown this Instance.
Failed to acquire instance startup/shutdown serialization primitive
Thu Sep 06 01:21:59 2012
Shutting down instance (abort)

Hang analysis shows "enq: AM - ASM file relocation" wait event is the culprit:

Verified Hangs in the System
  Root Chain Total Hang
 Hang Hang Inst Root #hung #hung Hang Hang Resolution
  ID Type Status Num Sess Sess Sess Conf Span Action
----- ---- -------- ---- ----- ----- ----- ------ ------ -------------------
  1 HANG VICSELTD 4 993 4 4 LOW GLOBAL IGNRD:HangOnASMInst
 Hang Ignored Reason: Hang occurs on an ASM instance and will therefore not
  be automatically resolved.

 inst# SessId Ser# OSPID PrcNm Event
 ----- ------ ----- --------- ----- -----
  1 2419 1 405776 FG rdbms ipc reply
  1 993 1 348546 CKPT DFS lock handle
  3 1737 1223 1565316 ARB0 DFS lock handle
  4 993 1 7708750 CKPT enq: AM - ASM file relocation

Victim Information
  Ignored
 HangID Inst# Sessid Ser Num OSPID Fatal BG Previous Hang Count
 ------ ----- ------ ------- --------- -------- ------------- -------
  1 4 993 1 7708750 Y New Hang 1

 

Hang Chain 1 Signature: 'enq: AM - ASM file relocation'<='DFS lock handle'<='DFS lock handle'<='rdbms ipc reply'

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms