Oracle Linux - The snapapi26 Module Causes NFS Hung

(Doc ID 2418879.1)

Last updated on JULY 09, 2018

Applies to:

Linux OS - Version Oracle Linux 6.9 and later
Linux x86-64

Symptoms

1. can see on NFS client end, the NFS server not responding.

nfs: server xx.xx.xx not responding, still trying
nfs: server xx.xx.xx not responding, timed out

2. On the same time, can see may blocked processes on NFS server.
kernel: INFO: task snapapid:15694 blocked for more than 120 seconds.
kernel: INFO: task service_process:17267 blocked for more than 120 seconds.
kernel: INFO: task snapapid:15694 blocked for more than 120 seconds.
kernel: INFO: task service_process:17267 blocked for more than 120 seconds.
kernel: INFO: task snapapid:15694 blocked for more than 120 seconds.
kernel: INFO: task service_process:17267 blocked for more than 120 seconds.
kernel: INFO: task snapapid:15694 blocked for more than 120 seconds.
kernel: INFO: task service_process:17267 blocked for more than 120 seconds.
kernel: INFO: task khugepaged:26 blocked for more than 120 seconds.
kernel: INFO: task snapapid:15694 blocked for more than 120 seconds.

3. Oracle Linux Server performance is down, and /var/log/messages shows blocked messages which includes kernel module [snapapi26]:

kernel: heartbeat_timer_func(kworker/0:0,0): Deadlock detected.dev=800010, cnt=4, state=4. Unfreezing... <<<<<
kernel: find_deadlocked(snapapid,15694): dev=800010 state=1024
kernel: do_resolver(snapapid,15694): Real cleanup started... s=ffff8801054fc000(800010)
kernel: INFO: task snapapid:15694 blocked for more than 120 seconds.
kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kernel: snapapid D ffff880116df67a0 0 15694 2 0x00000080
kernel: ffff88010d997ce0 0000000000000046 0000000000000000 0000000000000000
kernel: 00000000000121c0 ffff88010d997fd8 ffff88010d996010 00000000000121c0
kernel: ffff88010d997fd8 00000000000121c0 ffff880232a74300 ffff880116df6200
kernel: Call Trace:
kernel: [] schedule+0x3f/0x60
kernel: [] rwsem_down_failed_common+0xc5/0x160
kernel: [] ? snapapi_write+0x130/0x130 [snapapi26]
kernel: [] rwsem_down_write_failed+0x13/0x20
kernel: [] call_rwsem_down_write_failed+0x13/0x20
kernel: [] ? down_write+0x32/0x40
kernel: [] thaw_super+0x28/0xd0
kernel: [] thaw_bdev+0x6d/0x90
kernel: [] sn_thaw_bdev+0x4d/0x60 [snapapi26]
kernel: [] do_resolver+0x105/0x120 [snapapi26]
kernel: [] ? snapapi_write+0x130/0x130 [snapapi26]
kernel: [] resolver_loop+0x75/0xb0 [snapapi26]
kernel: [] ? snapapi_write+0x130/0x130 [snapapi26]
kernel: [] kthread+0x96/0xa0
kernel: [] kernel_thread_helper+0x4/0x10
kernel: [] ? kthread_worker_fn+0x1a0/0x1a0
kernel: [] ? gs_change+0x13/0x13

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms