My Oracle Support Banner

Exalogic Guest vServer Hang With "nfs_lookup_revalidate" Errors in /var/log/messages System Logs (Doc ID 2211383.1)

Last updated on OCTOBER 10, 2018

Applies to:

Oracle Exalogic Elastic Cloud Software - Version 2.0.6.2.160419 and later
Linux x86-64
Oracle Virtual Server x86-64

Symptoms

In Exalogic Virtual racks, when using NFSv4 on Guest vServers with UEK2 kernel, issue of Guest vServer hang is seen.

Following "nfs_lookup_revalidate" Bug Soft Lock error messages are seen in /var/log/messages at the time of the hang issue.  

Oct 19 09:22:05 testGuestVM kernel: BUG: soft lockup - CPU#11 stuck for 23s! [acroread:19949]
Oct 19 09:22:05 testGuestVM kernel: Modules linked in: dm_snapshot ovmapi nfs lockd fscache auth_rpcgss nfs_acl sunrpc rdma_ucm(U) ib_sdp(U) rdma_cm(U) iw_cm(U) ib_addr(U) bonding ib_ipoib(U) ib_cm(U) ipv6 ib_uverbs(U) ib_umad(U) mlx4_vnic(U) mlx4_vnic_helper(U) mlx4_ib(U) ib_sa(U) ib_mad(U) ib_core(U) mlx4_core(U) ppdev parport_pc parport microcode pcspkr i2c_piix4 i2c_core ext4 mbcache jbd2 xen_blkfront pata_acpi ata_generic ata_piix floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
Oct 19 09:22:05 testGuestVM kernel: CPU 11
Oct 19 09:22:05 testGuestVM kernel: Modules linked in: dm_snapshot ovmapi nfs lockd fscache auth_rpcgss nfs_acl sunrpc rdma_ucm(U) ib_sdp(U) rdma_cm(U) iw_cm(U) ib_addr(U) bonding ib_ipoib(U) ib_cm(U) ipv6 ib_uverbs(U) ib_umad(U) mlx4_vnic(U) mlx4_vnic_helper(U) mlx4_ib(U) ib_sa(U) ib_mad(U) ib_core(U) mlx4_core(U) ppdev parport_pc parport microcode pcspkr i2c_piix4 i2c_core ext4 mbcache jbd2 xen_blkfront pata_acpi ata_generic ata_piix floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
Oct 19 09:22:05 testGuestVM kernel:
Oct 19 09:22:05 testGuestVM kernel: Pid: 19949, comm: acroread Not tainted 2.6.39-400.276.1.el6uek.x86_64 #1 Xen HVM domU
Oct 19 09:22:05 testGuestVM kernel: RIP: 0010:[<ffffffff811860b8>] [<ffffffff811860b8>] shrink_dentry_list+0x38/0x1f0
Oct 19 09:22:05 testGuestVM kernel: RSP: 0000:ffff880f9bc95ac8 EFLAGS: 00000206
Oct 19 09:22:05 testGuestVM kernel: RAX: ffff880c2c6d4440 RBX: 0000000000000000 RCX: ffff880c2c6d43c0
Oct 19 09:22:05 testGuestVM kernel: RDX: ffff88032669ad38 RSI: ffff880f9bc95b58 RDI: ffff880c2c6d441c
Oct 19 09:22:05 testGuestVM kernel: RBP: ffff880f9bc95b18 R08: 0000000000000000 R09: 0000000000000000
Oct 19 09:22:05 testGuestVM kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff81517e0e
Oct 19 09:22:05 testGuestVM kernel: R13: ffff880f9bc95ab8 R14: ffffffff81517e0e R15: ffff880f9bc95aa8
Oct 19 09:22:05 testGuestVM kernel: FS: 00007f2439a1a700(0000) GS:ffff88100fd60000(0000) knlGS:0000000000000000
Oct 19 09:22:05 testGuestVM kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 19 09:22:05 testGuestVM kernel: CR2: 0000000001698d7c CR3: 0000000f9d627000 CR4: 00000000001006e0
Oct 19 09:22:05 testGuestVM kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 19 09:22:05 testGuestVM kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 19 09:22:05 testGuestVM kernel: Process acroread (pid: 19949, threadinfo ffff880f9bc94000, task ffff880e472f8200)
Oct 19 09:22:05 testGuestVM kernel: Stack:
Oct 19 09:22:05 testGuestVM kernel: 000000000000126e ffff88032669ad38 ffff880e35f69c5c ffff880c2c6d4440
Oct 19 09:22:05 testGuestVM kernel: ffffffff81040fdd ffff880c2c6d441c ffff880fa086a800 ffff880fa086a8e0
Oct 19 09:22:05 testGuestVM kernel: 0000000000000000 0000000000000000 ffff880f9bc95ba8 ffffffff81186400
Oct 19 09:22:05 testGuestVM kernel: Call Trace:
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81040fdd>] ? __ticket_spin_lock+0xd/0x20
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81186400>] __shrink_dcache_sb+0x190/0x1d0
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81186650>] shrink_dcache_parent+0x30/0x50
Oct 19 09:22:05 testGuestVM kernel: [<ffffffffa03a147b>] nfs_lookup_revalidate+0x3ab/0x540 [nfs]
Oct 19 09:22:05 testGuestVM kernel: [<ffffffffa03a1663>] nfs_open_revalidate+0x53/0x250 [nfs]
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff811782e3>] complete_walk+0xe3/0x120
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff8104c36d>] ? ptep_set_access_flags+0x6d/0x70
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff8117cd7b>] path_lookupat+0x6b/0x680
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81080c6d>] ? __sigqueue_free+0x3d/0x50
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff8117d3c1>] do_path_lookup+0x31/0xc0
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff8117e099>] user_path_at_empty+0x59/0xa0
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff8113a13b>] ? handle_mm_fault+0x15b/0x270
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81512278>] ? do_page_fault+0x1e8/0x470
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff8117e0f1>] user_path_at+0x11/0x20
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff811722cc>] vfs_fstatat+0x4c/0x90
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81080e4a>] ? do_sigaction+0x13a/0x1e0
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff811723eb>] vfs_stat+0x1b/0x20
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81172414>] sys_newstat+0x24/0x50
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff810d0587>] ? audit_syscall_entry+0x1d7/0x200
Oct 19 09:22:05 testGuestVM kernel: [<ffffffff81516a62>] system_call_fastpath+0x16/0x1b 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.