OCFS2 Node Keeps Rebooting with RIP dlm_do_master_request.isra (Doc ID 2261473.1)

Last updated on JULY 14, 2017

Applies to:

Linux OS - Version Oracle Linux 6.2 and later
Linux x86-64

Symptoms

Node keeps rebooting. Following call trace can be seen in system logs

 

[ 3083.081516] (updatedb,4960,21):o2net_send_tcp_msg:962 ERROR: sendmsg returned -13 instead of 96
[ 3083.081520] (updatedb,4960,21):dlm_do_master_request:1328 ERROR: status = -13
[ 3083.081521] (updatedb,4960,21):dlm_do_master_request:1329 ERROR: unhandled error!
[ 3083.081557] ------------[ cut here ]------------
[ 3083.081633] kernel BUG at fs/ocfs2/dlm/dlmmaster.c:1330!
[ 3083.081674] invalid opcode: 0000 [#1] SMP
[ 3083.081754] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables ip6table_filter i
p6_tables iptable_filter ocfs2 jbd2 ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager scsi_transport_iscsi ocfs2_stackglue configfs vfat fat dm_service_time coretemp mperf freq_table kvm_intel kvm ghash_clmulni_intel aesni_intel x
ts aes_x86_64 lrw gf128mul ablk_helper cryptd microcode mxm_wmi pcspkr shpchp wmi osst acpi_pad ch st dm_multipath nfsd auth_rpcgss nfs_acl lockd sunrpc sg ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crc32c_intel mgag200 i2c_a
lgo_bit drm_kms_helper ttm qla2xxx ixgbe drm ahci ptp libahci i2c_core scsi_transport_fc pps_core dca
[ 3083.084781] megaraid_sas scsi_tgt hwmon dm_mirror dm_region_hash dm_log dm_mod ipv6 autofs4
[ 3083.085131] CPU 21
[ 3083.085178] Pid: 4960, comm: updatedb Not tainted 3.8.13-98.7.1.el7uek.x86_64 #2 Oracle Corporation ORACLE SERVER X6-2L/ASM,MOBO TRAY,2U
[ 3083.085244] RIP: 0010:[<ffffffffa312c9e5>] [<ffffffffa312c9e5>] dlm_do_master_request.isra.17+0x4e5/0x730 [ocfs2_dlm]
[ 3083.085322] RSP: 0018:ffff880fd2b0b728 EFLAGS: 00010282
[ 3083.085356] RAX: 0000000000000045 RBX: fffffffffffffff3 RCX: 0000000000004c17
[ 3083.085405] RDX: 0000000000004c17 RSI: 0000000000000046 RDI: 0000000000000246
[ 3083.085474] RBP: ffff880fd2b0b800 R08: ffffffff81bfaa02 R09: 0000000000000709
[ 3083.085526] R10: 0000000000000004 R11: ffff880fd2b0b566 R12: ffff880fe335de40
[ 3083.085579] R13: ffff880fd2b0b788 R14: ffff881014a60800 R15: 0000000000000001
[ 3083.085633] FS: 00007f0f09ca8740(0000) GS:ffff88103f2a0000(0000) knlGS:0000000000000000
[ 3083.085690] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3083.085753] CR2: 00007f0f0af9c848 CR3: 0000001010ba4000 CR4: 00000000003407e0
[ 3083.085803] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3083.085853] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 3083.085904] Process updatedb (pid: 4960, threadinfo ffff880fd2b0a000, task ffff881006a682c0)
[ 3083.085957] Stack:
[ 3083.085990] fffffffffffffff3 ffff880fd2b0b780 00000000ffffffff ffff881022667900
[ 3083.086148] ffff881006a68920 ffff881006a682c0 ffff8810231e7ad8 ffff8810231e7ad4
[ 3083.086368] ffff881006a68920 ffff881006a682c0 0000000000000001 0000000022119760
[ 3083.086512] Call Trace:
[ 3083.086567] [<ffffffffa313013a>] dlm_get_lock_resource+0x98a/0x10a0 [ocfs2_dlm]
[ 3083.086615] [<ffffffffa3138e5f>] ? dlm_new_lock+0x2f/0x140 [ocfs2_dlm]
[ 3083.086660] [<ffffffffa31394c2>] dlmlock+0x552/0x14c0 [ocfs2_dlm]
[ 3083.086738] [<ffffffffa31e4752>] ? ocfs2_inode_cache_unlock+0x12/0x20 [ocfs2]
[ 3083.086793] [<ffffffffa32327c9>] ? ocfs2_metadata_cache_unlock+0x19/0x20 [ocfs2]
[ 3083.086846] [<ffffffffa323288a>] ? ocfs2_buffer_cached.isra.6+0x9a/0x1b0 [ocfs2]
[ 3083.086890] [<ffffffffa315a020>] ? o2dlm_lock_ast_wrapper+0x20/0x20 [ocfs2_stack_o2cb]
[ 3083.086953] [<ffffffffa315a000>] ? 0xffffffffa3159fff
[ 3083.087004] [<ffffffffa31e47b2>] ? ocfs2_inode_cache_io_unlock+0x12/0x20 [ocfs2]
[ 3083.087067] [<ffffffffa3232e39>] ? ocfs2_metadata_cache_io_unlock+0x19/0x20 [ocfs2]
[ 3083.087128] [<ffffffffa31c6260>] ? ocfs2_read_blocks+0x350/0x6c0 [ocfs2]
[ 3083.087180] [<ffffffffa315a2dc>] o2cb_dlm_lock+0x5c/0x80 [ocfs2_stack_o2cb]
[ 3083.087232] [<ffffffffa315a000>] ? 0xffffffffa3159fff
[ 3083.087276] [<ffffffffa315a020>] ? o2dlm_lock_ast_wrapper+0x20/0x20 [ocfs2_stack_o2cb]
[ 3083.087331] [<ffffffffa30ad392>] ocfs2_dlm_lock+0x42/0x50 [ocfs2_stackglue]
[ 3083.087388] [<ffffffffa31d88cb>] __ocfs2_cluster_lock.isra.31+0x37b/0x830 [ocfs2]
[ 3083.087462] [<ffffffff81285321>] ? vsnprintf+0x411/0x670
[ 3083.092979] [<ffffffff81285649>] ? snprintf+0x39/0x40
[ 3083.098289] [<ffffffffa31d900b>] ocfs2_open_lock+0xfb/0x1a0 [ocfs2]
[ 3083.103819] [<ffffffffa31e7d2c>] ocfs2_iget+0x45c/0x830 [ocfs2]
[ 3083.109306] [<ffffffffa31f3100>] ocfs2_lookup+0xc0/0x300 [ocfs2]
[ 3083.114486] [<ffffffff8119f6b8>] ? d_alloc+0x58/0x70
[ 3083.119736] [<ffffffff8119119d>] lookup_real+0x1d/0x50
[ 3083.124651] [<ffffffff81191723>] __lookup_hash+0x33/0x40
[ 3083.129649] [<ffffffff81576c0c>] lookup_slow+0x44/0xa9
[ 3083.134477] [<ffffffff8119465e>] path_lookupat+0x76e/0x7d0
[ 3083.139121] [<ffffffff8101b069>] ? read_tsc+0x9/0x20
[ 3083.143650] [<ffffffff81192d3f>] ? getname_flags+0x4f/0x190
[ 3083.148002] [<ffffffff811946eb>] filename_lookup+0x2b/0xc0
[ 3083.152128] [<ffffffff81197b14>] user_path_at_empty+0x54/0x90
[ 3083.156209] [<ffffffffa31d3011>] ? ocfs2_lock_res_free.part.27+0x41/0x4d0 [ocfs2]
[ 3083.160218] [<ffffffff81089a9a>] ? lg_local_unlock+0x1a/0x20
[ 3083.164342] [<ffffffff811a647e>] ? mntput_no_expire+0x3e/0x120
[ 3083.168306] [<ffffffff81197b61>] user_path_at+0x11/0x20
[ 3083.172115] [<ffffffff8118c5e0>] vfs_fstatat+0x50/0xb0
[ 3083.175937] [<ffffffff8118cbb7>] sys_newlstat+0x27/0x40
[ 3083.179513] [<ffffffff810df356>] ? __audit_syscall_exit+0x1f6/0x2a0
[ 3083.183037] [<ffffffff810df10c>] ? __audit_syscall_entry+0x9c/0xf0
[ 3083.186519] [<ffffffff81586479>] system_call_fastpath+0x16/0x1b
[ 3083.189861] Code: fe ff ff 74 68 83 fb fc 74 63 83 fb e4 74 5e 48 b8 40 02 00 00 00 00 00 10 48 85 05 e6 f1 fb ff 74 09 48 85 05 7d 0a fc ff 74 02 <0f> 0b 65 8b 0c 25 1c a0 00 00 65 48 8b 04 25 c0 b7 00 00 8b 90
[ 3083.199564] RIP [<ffffffffa312c9e5>] dlm_do_master_request.isra.17+0x4e5/0x730 [ocfs2_dlm]
[ 3083.203721] RSP <ffff880fd2b0b728>

 

 

 

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms