ACFS crashing due to wrong network interface configuration

(Doc ID 1506752.1)

Last updated on OCTOBER 31, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.0. and later
Information in this document applies to any platform.

Symptoms

While ASM was running, and the RDBMS was running with an ORACLE_HOME on ACFS,
this ACFS error suddenly took down both the ASM and RDBMS instances

Fix write in gcs resources
Sun Sep 23 06:16:20 2012
NOTE: SMON starting instance recovery for group ACFS domain 1 (mounted)
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
Reconfiguration complete
NOTE: starting recovery of thread=1 ckpt=4.187 group=1 (ACFS)
NOTE: SMON waiting for thread 1 recovery enqueue
NOTE: SMON about to begin recovery lock claims for diskgroup 1 (ACFS)
NOTE: SMON successfully validated lock domain 1
NOTE: advancing ckpt for group 1 (ACFS) thread=1 ckpt=4.187
NOTE: SMON did instance recovery for group ACFS domain 1
Reconfiguration started (old inc 101, new inc 103)
List of instances:
1 2 3 4 (myinst: 2)
Global Resource Directory frozen
Communication channels reestablished
Sun Sep 23 06:16:25 2012
* domain 0 valid = 1 according to instance 1
* domain 2 valid = 1 according to instance 1
* domain 3 valid = 1 according to instance 1
* domain 8 valid = 1 according to instance 1
* domain 1 valid = 1 according to instance 1
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Submitted all GCS remote-cache requests
Fix write in gcs resources
Reconfiguration complete
Mon Sep 24 03:16:57 2012
NOTE: ASMB process exiting, either shutdown is in progress
NOTE: or foreground connected to ASMB was killed.
Mon Sep 24 03:16:57 2012
NOTE: client exited [30738]
NOTE: force a map free for map id 3
Mon Sep 24 03:16:59 2012
PMON (ospid: 22910): terminating the instance due to error 481
Instance terminated by PMON, pid = 22910

File_name::alert_+ASM2_it311bhr

Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
Mon Sep 24 03:17:39 2012
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Submitted all GCS remote-cache requests
Fix write in gcs resources
Reconfiguration complete
Mon Sep 24 03:43:24 2012
ACFS ERROR: Cluster membership rebuild did not complete. Terminating ASM instance to avoid deadlock
Mon Sep 24 03:43:27 2012
PMON (ospid: 31904): terminating the instance due to error 15478
Mon Sep 24 03:43:27 2012
System state dump requested by (instance=1, osid=31904 (PMON)), summary=[abnormal instance termination].
System State dumped to trace file /app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_diag_31967.trc
Dumping diagnostic data in directory=[cdmp_20120924034327], requested by (instance=1, osid=31904 (PMON)), summary=[abnormal instance termination].
Instance terminated by PMON, pid = 31904

File_name :: alert_+ASM1_it311ahr

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms