Due to ACFS performance issue the node rebootes

(Doc ID 2292712.1)

Last updated on FEBRUARY 20, 2018

Applies to:

Oracle Database - Enterprise Edition - Version 12.1.0.2 to 12.1.0.2 [Release 12.1]
Information in this document applies to any platform.

Symptoms

 

messages:

Line 2758: Feb 10 02:17:26 node1 kernel: ACFSK-0074: ASSERTION FAILURE:
snapFlags & POSSIBLE_REDO File:
/scratch/builds/josmith/josmith_oss/el5uek39400_x86_64/3/usm/src/ofs/driver/./gen/ofsgensnap.c Line: 3765.
Line 2773: Feb 10 02:25:50 node1 kernel: ACFSK-0050: Assert at/scratch/builds/josmith/josmith_oss/el5uek39400_x86_64/3/usm/src/ofs/driver/./
gen/ofsgensnap.c line 3765 repeated 3012 times.Line 2774: Feb 10 02:41:17 node1 kernel:
ACFSK-0050: Assert at/scratch/builds/josmith/josmith_oss/el5uek39400_x86_64/3/usm/src/ofs/driver/./gen/ofsgensnap.c line 3765 repeated 4752 times.
Line 2819: Feb 10 05:53:12 node1 kernel: ACFSK-0050: Assert at/scratch/builds/josmith/josmith_oss/el5uek39400_x86_64/3/usm/src/ofs/driver/./
gen/ofsgensnap.c line 3765 repeated 6672 times.

Feb 09 22:57:02 node1 kernel: [<ffffffff8105752b>] ?
__wake_up_common+0x5b/0x90
Feb 09 22:57:02 node1 kernel: [<ffffffffa0755d50>] ?
OfsFlushData+0x240/0x240 [oracleacfs]
Feb 09 22:57:02 node1 kernel: [<ffffffffa0590931>]
KsKthreadRun+0x81/0xb0 [oracleoks]
Feb 09 22:57:02 node1 kernel: [<ffffffff815129c4>]
kernel_thread_helper+0x4/0x10
Feb 09 22:57:02 node1 kernel: [<ffffffffa05908b0>] ? __KsPanic+0xc0/0xc0
[oracleoks]
Feb 09 22:57:02 node1 kernel: [<ffffffff815129c0>] ? gs_change+0x13/0x13
Feb 09 22:57:02 node1 kernel: INFO: task acfsdefrag3:34071 blocked formore than 120 seconds.
Feb 09 22:57:02 node1 kernel: "echo 0 >/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 09 22:57:02 node1 kernel: acfsdefrag3 D ffffffff81520660 034071 2 0x00000080
Feb 09 22:57:02 node1 kernel: ffff88099a7a9af0 0000000000000046ffffffff81512c4e 0000000000011180

acfsutil.log
=============

K 4294741.994/160131212707 apx_acfs_+apx1[33074] [Oracle OKS] Nodes in
cluster:
K 4294741.997/160131212707 apx_acfs_+apx1[33074] Node 1 (IP 169.254.247.97)
K 4294741.999/160131212707 apx_acfs_+apx1[33074] Node 2 (IP 169.254.150.68)

K 4294742.004/160131212707 apx_acfs_+apx1[33074] kcss_rbld_start: thread
0xffff880ab6bb74c0 ch 0xffffffffa05f8b98 client id 1
K 4294742.004/160131212707 oks_rbld[33085] rcbp 0xffffffffa05f8d60 step 0,
cur_leader 0x1 incarn 0x4
K 4294742.004/160131212707 oks_rbld[33085] odlm_stall_all: reason 5
K 4294742.005/160131212707 oks_rbld[33085] [Oracle OKS] rebuild: client: 1,
step: 0/10, elapsed: 1
K 4294742.005/160131212707 oks_rbld[33085] rcbp 0xffffffffa05f8d60 step 1,
cur_leader 0x1 incarn 0x4
K 4294742.005/160131212707 oks_rbld[33085] [Oracle OKS] rebuild: client: 1,
step: 1/10, elapsed: 0
K 4294742.005/160131212707 oks_rbld[33085] rcbp 0xffffffffa05f8d60 step 2,
cur_leader 0x1 incarn 0x4
K 4294742.005/160131212707 oks_rbld[33085] [Oracle OKS] rebuild: client: 1,
step: 2/10, elapsed: 0
K 4294742.005/160131212707 oks_rbld[33085] rcbp 0xffffffffa05f8d60 step 3,
cur_leader 0x1 incarn 0x4
F 4294742.005/160131212707 oks_rbld[33085] OfsMarkForRecovery: Node 4 reason
8

K 4294742.248/160131212707 oks_comm[9010] KsRecv: recv failed -104
K 4294742.248/160131212707 oks_comm[9010] odlm_comm_worker, KsRecv returns
-4, comm thread exiting
K 4294742.248/160131212707 oks_rbld[33085] odlm_comm_close: Shutdown chan 1
to node 2
K 4294742.248/160131212707 oks_rbld[33085] [Oracle OKS] rebuild: client: 1,
step: 10/10, elapsed: 34
K 4294742.248/160131212707 oks_rbld[33085] kcss_rbld_start: thread
0xffff882ee05d7940 ch 0xffffffffa05f8ab0 client id 0
K 4294742.248/160131212707 oks_rbld[33085] odlm_comm_close: Shutdown chan 2
to node 2
K 4294742.248/160131212707 oks_rbld[33084] rcbp 0xffffffffa05f8d20 step 1,
cur_leader 0x1 incarn 0x4

K 4294742.249/160131212707 oks_rbld[33084] KsWorkerWakeup/ADVM: failed to
kick a peer
K 4294742.249/160131212707 asmResilver2[9035] KsWorkerWakeup/ADVM: failed to
kick a peer
V 4294742.249 asmResilver2[33238] Asm_acqSegment: REDO.datastore-481:
odlm_lock returned 36
V 4294742.249 asmResilver2[33998] Asm_acqSegment: REDO.acldatstore-481:
odlm_lock returned 36

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms