My Oracle Support Banner

bug 18610307:: deadlock resulting rpc throttle causing node reboot (Doc ID 2030238.1)

Last updated on APRIL 24, 2018

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.2 and later
Information in this document applies to any platform.

Symptoms

oracle database is down automatically alongwith node.

resulting outage at cluster level.

In messages file,we see ,

 

 

May 17 02:46:14 xxxxxxxxxxxx kernel: [Oracle OKS] ASSERTION FAILURE: FALSE File: /scratch/builds/aime/aime_usm_922089/usm/src/oks/driver/./odlmrebuild.c Line: 3218
May 17 02:46:14 xxxxxxxxxxxx kernel: KsDumpStack: Stack call traceback
May 17 02:46:14 xxxxxxxxxxxx kernel: Pid: 26344, comm: acfsutil.bin Tainted: P W --------------- 2.6.32-279.37.2.el6.x86_64 #1
May 17 02:46:14 xxxxxxxxxxxx kernel: Call Trace:
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa078abdb>] ? KsDumpStack+0x3b/0x40 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa078e586>] ? KsDoAssertion+0xd6/0xf0 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa07a1020>] ? odlm_wait_for_membership_change+0x1d0/0x310 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa0799779>] ? odlm_wait_for_rbld+0xe9/0x140 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa07ad1e9>] ? new_lock_remote+0x4e9/0xac0 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa079e090>] ? create_lock+0x720/0xc60 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa079ed0f>] ? odlm_lock+0x2af/0x460 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa088401f>] ? OfsAcquireDLMLock_int+0x175f/0x23b0 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa08b6ae0>] ? OfsFreeMemory+0x70/0xb0 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa089a21f>] ? OfsDeallocateIOCB+0x5f/0x70 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa0941d1a>] ? OfsRecoveryGetLocalDirBlock+0x6a/0xe0 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa09424d8>] ? OfsRecoverNode+0x218/0x830 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa078cd56>] ? KsRwLockWrite+0x16/0x40 [oracleoks]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa0942f4c>] ? OfsDoPhase1Recovery+0x1cc/0x4d0 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa0945c68>] ? OfsWaitForRecoveryToComplete+0x118/0x440 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa08836c4>] ? OfsAcquireDLMLock_int+0xe04/0x23b0 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff81214541>] ? avc_has_perm+0x71/0x90
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa0879330>] ? ofs_dlm_blkrtn+0x0/0x170 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa0884e23>] ? OfsOpAcquireDLMLock_int+0xc3/0x960 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffffa095fab5>] ? ofs_getattr+0xb5/0x300 [oracleacfs]
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff8117b971>] ? vfs_getattr+0x51/0x80
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff8117ba00>] ? vfs_fstatat+0x60/0x80
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff8117bb4b>] ? vfs_stat+0x1b/0x20
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff8117bb74>] ? sys_newstat+0x24/0x50
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff810d3fa7>] ? audit_syscall_entry+0x1d7/0x200
May 17 02:46:14 xxxxxxxxxxxx kernel: [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
May 17 02:46:14 xxxxxxxxxxxx kernel: OKSK-00007: Information has been saved in the file /var/log/Kokslog.0 . Include the contents of this file if reporting a problem to Oracle.
May 17 02:46:14 xxxxxxxxxxxx kernel: OKSK-00007: Information has been saved in the file /var/log/Fokslog.0 . Include the contents of this file if reporting a problem to Oracle.
May 17 02:46:15 xxxxxxxxxxxx kernel: (v 45364) GUARD-02: 31144 khash_super_prune_nolock: SUPER PRUNE at 1431830775 (line 277)
May 17 02:46:19 xxxxxxxxxxxx kernel: (v 45364) GUARD-02: 28292 khash_super_prune_nolock: SUPER PRUNE at 1431830779 (line 277)
May 17 02:46:22 xxxxxxxxxxxx kernel: (v 45364) GUARD-02: 31185 khash_super_prune_nolock: SUPER PRUNE at 1431830782 (line 277)
May 17 02:46:24 xxxxxxxxxxxx kernel: ADVMK-00017: The ASM instance terminated unexpectedly. All ADVM volumes will be taken offline.

File_name :: messages-20150517

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process and therefore has not been subject to an independent technical review.
My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.