My Oracle Support Banner

ASM Diskgroup Rebalance Fails with ORA-13031973 (Doc ID 2372427.1)

Last updated on FEBRUARY 23, 2022

Applies to:

Oracle Database Cloud Schema Service - Version N/A and later
Gen 1 Exadata Cloud at Customer (Oracle Exadata Database Cloud Machine) - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Oracle Database Backup Service - Version N/A and later
Oracle Database Cloud Exadata Service - Version N/A and later
Information in this document applies to any platform.

Symptoms

While performing ASM diskgroup rebalance, below error reported in the ASM instance alert log file.  ARB process re-spawns and fails again:

alert__ASM2.log (<ORACLE BASE>/diag/asm/+ASM2/trace/ ):

Sat Mar 03 02:17:58 2018
SQL> alter diskgroup DATA_DG01 rebalance power 5
.
Sat Mar 03 02:18:42 2018
ARB0 started with pid=39, OS id=31397
NOTE: assigning ARB0 to group 3/0x9ed92839 (DATA_DG01) with 3 parallel I/Os
.
Sat Mar 03 02:19:26 2018
NOTE: membership refresh pending for group 3/0x9ed92839 (DATA_DG01)
Sat Mar 03 02:19:27 2018
ERROR: ORA-13031973 thrown in ARB0 for group number 3
Errors in file /oracle_crs/crs/diag/asm/+asm/+ASM2/trace/+ASM2_arb0_31397.trc:
ORA-13031973: Message 13031973 not found; product=RDBMS; facility=ORA
NOTE: stopping process ARB0
NOTE: rebalance interrupted for group 3/0x9ed92839
(DATA_DG01)
.
SUCCESS: refreshed membership for 3/0x9ed92839 (DATA_DG01)
NOTE: Attempting voting file refresh on diskgroup DATA_DG01
NOTE: ASM did background COD recovery
for group 3/0x9ed92839 (DATA_DG01)
NOTE: starting rebalance of group 3/0x9ed92839 (DATA_DG01) at power 3
Starting background process ARB0
Sat Mar 03 02:20:14 2018
ARB0 started with pid=39, OS id=32501
NOTE: assigning ARB0 to group 3/0x9ed92839 (DATA_DG01) with 3 parallel I/Os

.
.
Sat Mar 03 04:57:29 2018
ARB0 started with pid=27, OS id=19622
NOTE: assigning ARB0 to group 3/0x9ed48b66 (DATA_DG01) with 3 parallel I/Os
.
Sat Mar 03 04:57:55 2018
NOTE: GroupBlock outside
rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=3
Sat Mar 03 04:57:58 2018
NOTE: membership refresh pending for group 3/0x9ed48b66 (DATA_DG01)
Sat Mar 03 04:58:01 2018
GMON querying group 3 at 27 for pid 18, osid 9782
SUCCESS: refreshed membership for 3/0x9ed48b66 (DATA_DG01)
Sat Mar 03 04:58:02 2018
ERROR: ORA-13031973 thrown in ARB0 for group number 3
Errors in file /oracle_crs/crs/diag/asm/+asm/+ASM2/trace/+ASM2_arb0_19622.trc:
ORA-13031973: Message 13031973 not found; product=RDBMS; facility=ORA
ORA-13031973: Message 13031973 not found; product=RDBMS; facility=ORA
NOTE: stopping process ARB0
NOTE: rebalance interrupted for group 3/0x9ed48b66 (DATA_DG01)
NOTE: Attempting voting file refresh on diskgroup DATA_DG01
NOTE: ASM did background COD recovery for group 3/0x9ed48b66 (DATA_DG01)
NOTE: starting rebalance of group 3/0x9ed48b66 (DATA_DG01) at power 3
Starting background process ARB0
Sat Mar 03 04:58:04 2018
ARB0 started with pid=27, OS id=20519
NOTE: assigning ARB0 to group 3/0x9ed48b66 (DATA_DG01) with 3 parallel I/Os

_ASM2_arb0_19622.trc (<ORACLE BASE>/diag/asm/+ASM2/trace/ ):

Trace file /oracle_crs/crs/diag/asm/+asm/+ASM2/trace/+ASM2_arb0_19622.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
ORACLE_HOME = /oracle_crs/product/11.2.0/crs_1
System name: Linux
Node name: rachost2
Release: 2.6.18-419.el5
Version: #1 SMP Wed Feb 22 22:40:57 EST 2017
Machine: x86_64
Instance name: +ASM2
Redo thread mounted by this instance: 0 <none>
Oracle process number: 27
Unix process pid: 19622, image: oracle@lrchs03apdbprn01 (ARB0)

*** 2018-03-03 04:57:55.588
*** SESSION ID:(1675.5) 2018-03-03 04:57:55.588
*** CLIENT ID:() 2018-03-03 04:57:55.588
*** SERVICE NAME:() 2018-03-03 04:57:55.588
*** MODULE NAME:() 2018-03-03 04:57:55.588
*** ACTION NAME:() 2018-03-03 04:57:55.588

ARB0 relocating file +DATA_DG01.390.890216177 (1 entries)
ARB0 relocating file +DATA_DG01.297.836776263 (1 entries)
ARB0 relocating file +DATA_DG01.414.865227119 (1 entries)
.
.
*** 2018-03-03 04:58:01.865
ARB0 relocating file +DATA_DG01.297.836776263 (1 entries)
ARB0 relocating file +DATA_DG01.297.836776263 (1 entries)
ARB0 relocating file +DATA_DG01.418.875314573 (1 entries)
.
ARB0 relocating file +DATA_DG01.297.836776263 (1 entries)
ARB0 relocating file +DATA_DG01.297.836776263 (1 entries)
ORA-13031973: Message 13031973 not found; product=RDBMS; facility=ORA
ORA-13031973: Message 13031973 not found; product=RDBMS; facility=ORA

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Symptoms
Cause
Solution
 Community Discussions


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.