Kernel Panic at [Exception RIP: blk_mq_check_expired+24]
(Doc ID 2587893.1)
Last updated on OCTOBER 17, 2019
Applies to:
Linux OS - Version Oracle Linux 6.7 with Unbreakable Enterprise Kernel [4.1.12] and laterOracle Cloud Infrastructure - Version N/A and later
Linux x86-64
Symptoms
Systems can suffer random NULL-pointer panics when blk-mq handles timed out requests.
A typical panic string can look like this:
PANIC: "BUG: unable to handle kernel NULL pointer dereference at 0000000000000450"
A typical panic stack can look like this:
crash7latest> bt PID: 2327 TASK: ffff880fc0438e00 CPU: 7 COMMAND: "kworker/7:1H" #0 [ffff880fc0e47960] machine_kexec at ffffffff81062a6b #1 [ffff880fc0e479d0] crash_kexec at ffffffff8111cff2 #2 [ffff880fc0e47aa0] oops_end at ffffffff8101b938 #3 [ffff880fc0e47ad0] no_context at ffffffff81747c5c #4 [ffff880fc0e47b30] __bad_area_nosemaphore at ffffffff81747d44 #5 [ffff880fc0e47b80] bad_area_nosemaphore at ffffffff81747eb0 #6 [ffff880fc0e47b90] __do_page_fault at ffffffff810723a6 #7 [ffff880fc0e47c00] trace_do_page_fault at ffffffff81072869 #8 [ffff880fc0e47c40] do_async_page_fault at ffffffff8106b3de #9 [ffff880fc0e47c60] async_page_fault at ffffffff8175be4f [exception RIP: blk_mq_check_expired+24] RIP: ffffffff81327aa8 RSP: ffff880fc0e47d18 RFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000014 RCX: 0000000000000000 RDX: ffff880fc0e47dc0 RSI: ffff880fc25ab800 RDI: ffff880fc25ab800 RBP: ffff880fc0e47d18 R8: ffff880fc0e47dc0 R9: 0000000000000001 R10: 0000000000001111 R11: 0000000000000000 R12: 0000000000000005 R13: ffff880fc1270b50 R14: ffff880fc1396000 R15: ffff880fc2f86d40 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #10 [ffff880fc0e47d20] bt_for_each at ffffffff8132a177 #11 [ffff880fc0e47d80] blk_mq_tag_busy_iter at ffffffff8132a1d6 #12 [ffff880fc0e47db0] blk_mq_timeout_work at ffffffff813262c3 #13 [ffff880fc0e47e00] process_one_work at ffffffff810a7025 #14 [ffff880fc0e47e50] worker_thread at ffffffff810a7832 #15 [ffff880fc0e47ec0] kthread at ffffffff810ad4aa #16 [ffff880fc0e47f50] ret_from_fork at ffffffff81756a41 crash7latest>
A typical pattern in dmesg can look like:
[695101.496601] BUG: unable to handle kernel NULL pointer dereference at 0000000000000450 [695101.498435] IP: [<ffffffff81327aa8>] blk_mq_check_expired+0x18/0x90 [695101.499861] PGD 8000000809e83067 PUD 8171e2067 PMD 0 [695101.501101] Oops: 0000 [#1] SMP [695101.501911] Modules linked in: sunrpc ppdev crct10dif_pclmul crc32_pclmul ghash_clmulni_intel dm_mod aesni_intel lrw gf128mul glue_helper ablk_helper cryptd parport_pc parport i2c_piix4 acpi_cpufreq i2c_core pcspkr binfmt_misc ip_tables xfs libcrc32c crc32c_intel serio_raw nvme ena nvme_core [695101.508491] CPU: 7 PID: 2327 Comm: kworker/7:1H Not tainted 4.1.12-124.25.1.el7uek.x86_64 #2 [695101.510343] Hardware name: Amazon EC2 r5.2xlarge/, BIOS 1.0 10/16/2017 [695101.511796] Workqueue: kblockd blk_mq_timeout_work [695101.512915] task: ffff880fc0438e00 ti: ffff880fc0e44000 task.ti: ffff880fc0e44000 [695101.514559] RIP: 0010:[<ffffffff81327aa8>] [<ffffffff81327aa8>] blk_mq_check_expired+0x18/0x90
Changes
N/A
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Symptoms |
Changes |
Cause |
Solution |
References |