My Oracle Support Banner

PQ Slave Initiating Offline of a Disk May Result in ORA-15078 or ORA-15130 ASM Diskgroup Forecibly Dismounted (Doc ID 2636990.1)

Last updated on JULY 27, 2020

Applies to:

Oracle Database - Enterprise Edition - Version 12.2.0.1 to 19.6.0.0.0 [Release 12.2 to 19]
Information in this document applies to any platform.

Symptoms

DB instance Alert log will show PQ slaves initiating offline of a disk:


10-12 10:30:52.859464 10606 PDB_PRODNCM(3):NOTE: process
_p0la_cdbrac12 (302818) initiating offline of disk 120.2322156497
(DATAC1_FD_00_NZEPC01F03CE034XM) with mask 0x7e in group 1 (DATAC1) with
client assisting


This may happen with several disks until the diskgroup is forcibly taken offline.  From the ASM alert log:


2019-10-13T12:45:32.691005-07:00
ERROR: too many offline disks in PST (grp 1)
2019-10-13T12:45:32.691874-07:00
NOTE: cache dismounting (not clean) group 1/0xB2A07837 (DATAC1)


DB instance alert log may report ORA-15178:

2019-10-12T10:45:28.595537-07:00
Errors in file /u01/app/oracle/diag/rdbms/cdbrac1/CDBRAC13/trace/CDBRAC13_ora_324683.trc:
ORA-15078: ASM diskgroup was forcibly dismounted

And other trace files may report ORA-15130:

*** 2019-10-13T12:45:33.395506-07:00
NOTE: handling error ORA-15130 while draining I/OsERROR: ORA-15130 thrown in
ARB0 for group number 1
ORA-15130: diskgroup "DATAC1" is being dismounted

The PQ slave trace file from the PQ slave that initiated the disk offline will have reconnects in it like :

Reconnect to box 0x172eccb0 with timeout 0 failed with error 66 (Connect is
still in progress)
ossdebug_dump_stack()+185<-ossnet_vrpc_completion()+2663<-ossnet_process_netms
g_completion()+157<-ossnet_wait_all()+377<-ossnet_wait()+356<-oss_wait_ext()+3
27<-oss_wait()+108<-kfk_reap_oss_async_io()+519<-kfk_reap_ios_from_subsys()+21
0<-kfk_reap_ios()+432<-kfk_io1()+1461
...
*** 2019-10-13T12:45:31.095291-07:00 (PDB_PRODNCM(3))
Device Reopen async completion request 0x175db300, device handle 0x1734be50,
error code 0
Reconnect to box 0x173107d0 with timeout 1072 failed with error 38 (SKGXP
timeout)

And may show a 15081 "failed xlate" error:

WARNING: failed xlate 3 kfiorq=0x7f3142e95fc0 fib=0x6e67af2d8 isread=1 sync=0
err=15081
NOTE: process _p07n_cdbexp11 (358769) initiating offline of disk
117.4043091467 (DATAC1_FD_05_NZEPC01F03CE033XM) with mask 0x7e in group 1
(DATAC1) with client assisting

 

Changes

 Possibly recently upgraded to 12.2+ or added more PQ slaves.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.