ODA: Kernel Panic INVALIDATE_BH_LRUS during 12.1.2.6.0 Infra Upgrade -- Looping Reboot (Doc ID 2140386.1)

Last updated on MAY 31, 2016

Applies to:

Oracle Database Appliance - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

During the 12.1.2.5 -> 12.1.2.6 INFRA upgrade, patching is expected to automatically reboot the nodes. 

However during the reboot one node keeps on rebooting in a loop after showing the following in console:

UG: soft lockup - CPU#29 stuck for 22s! [kworker/u:48:704]

Modules linked in: sd_mod(+) crc_t10dif floppy xen_blkfront mpt3sas scsi_transport_sas raid_class pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod

CPU 29

Modules linked in: sd_mod(+) crc_t10dif floppy xen_blkfront mpt3sas scsi_transport_sas raid_class pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod

 

Pid: 704, comm: kworker/u:48 Not tainted 2.6.39-400.264.13.el6uek.x86_64 #1 Xen HVM domU

RIP: 0010:[<ffffffff810a96a0>]  [<ffffffff810a96a0>] smp_call_function_many+0x1f0/0x260

RSP: 0018:ffff883b96b03bd0  EFLAGS: 00000202

RAX: 0000000000000080 RBX: ffff883b96b03b70 RCX: 0000000000000007

RDX: 0000000000000007 RSI: 0000000000000080 RDI: 0000000000000000

RBP: ffff883b96b03c10 R08: ffff883ba5604810 R09: 0000000000000080

R10: 0000000000000002 R11: ffff883b96b03a08 R12: ffffffff81517b0e

R13: ffffffff812fac36 R14: ffff883b96b03b40 R15: ffffffff812fa62e

FS:  0000000000000000(0000) GS:ffff883d0e9a0000(0000) knlGS:0000000000000000

CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b

CR2: 00007fff666bd900 CR3: 0000000001787000 CR4: 00000000001006e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

Process kworker/u:48 (pid: 704, threadinfo ffff883b96b02000, task ffff883b96b00340)

Stack:

01ff883b96b03be0 0000000000000000 ffff883b96b03c30 ffffffff8119bad0

0000000000000000 0000000000000001 ffff883b9cde4758 ffff883b969b9800

ffff883b96b03c20 ffffffff810a9732 ffff883b96b03c50 ffffffff810a976b

Call Trace:

[<ffffffff8119bad0>] ? buffer_cpu_notify+0xa0/0xa0

[<ffffffff810a9732>] smp_call_function+0x22/0x30

[<ffffffff810a976b>] on_each_cpu+0x2b/0x70

[<ffffffff8119b5ac>] invalidate_bh_lrus+0x1c/0x20

[<ffffffff811a1dc8>] kill_bdev+0x28/0x50

[<ffffffff811a2f51>] __blkdev_put+0x71/0x1d0

[<ffffffff81099340>] ? async_schedule+0x20/0x20

[<ffffffff811a30e8>] blkdev_put+0x38/0x150

[<ffffffff81246a7c>] register_disk+0x18c/0x1a0

[<ffffffff81099340>] ? async_schedule+0x20/0x20

[<ffffffff81246be5>] add_disk+0x155/0x2f0

[<ffffffffa00b77c1>] sd_probe_async+0x131/0x1f0 [sd_mod]

[<ffffffff810993c4>] async_run_entry_fn+0x84/0x160

[<ffffffff8108a7c6>] process_one_work+0x136/0x450

[<ffffffff8108bda0>] worker_thread+0x170/0x3c0

[<ffffffff8108bc30>] ? manage_workers+0x120/0x120

[<ffffffff81090fe6>] kthread+0x96/0xa0

[<ffffffff81517884>] kernel_thread_helper+0x4/0x10

[<ffffffff81090f50>] ? kthread_worker_fn+0x1a0/0x1a0

[<ffffffff81517880>] ? gs_change+0x13/0x13

Code: e8 a6 4c 46 00 0f ae f0 48 8b 7b 30 ff 15 21 f0 6e 00 80 7d c7 00 0f 84 97 fe ff ff f6 43 20 01 0f 84 8d fe ff ff 0f 1f 44 00 00 <f3> 90 f6 43 20 01 75 f8 e9 7b fe ff ff 0f 1f 00 4c 89 ea 4c 89

Call Trace:

[<ffffffff8119bad0>] ? buffer_cpu_notify+0xa0/0xa0

[<ffffffff810a9732>] smp_call_function+0x22/0x30

[<ffffffff810a976b>] on_each_cpu+0x2b/0x70

[<ffffffff8119b5ac>] invalidate_bh_lrus+0x1c/0x20

[<ffffffff811a1dc8>] kill_bdev+0x28/0x50

[<ffffffff811a2f51>] __blkdev_put+0x71/0x1d0

[<ffffffff81099340>] ? async_schedule+0x20/0x20

[<ffffffff811a30e8>] blkdev_put+0x38/0x150

[<ffffffff81246a7c>] register_disk+0x18c/0x1a0

[<ffffffff81099340>] ? async_schedule+0x20/0x20

[<ffffffff81246be5>] add_disk+0x155/0x2f0

[<ffffffffa00b77c1>] sd_probe_async+0x131/0x1f0 [sd_mod]

[<ffffffff810993c4>] async_run_entry_fn+0x84/0x160

[<ffffffff8108a7c6>] process_one_work+0x136/0x450

[<ffffffff8108bda0>] worker_thread+0x170/0x3c0

[<ffffffff8108bc30>] ? manage_workers+0x120/0x120

[<ffffffff81090fe6>] kthread+0x96/0xa0

[<ffffffff81517884>] kernel_thread_helper+0x4/0x10

[<ffffffff81090f50>] ? kthread_worker_fn+0x1a0/0x1a0

[<ffffffff81517880>] ? gs_change+0x13/0x13

BUG: soft lockup - CPU#30 stuck for 22s! [kworker/u:55:711]

Modules linked in: sd_mod(+) crc_t10dif floppy xen_blkfront m

 

Changes

 12.1.2.5->12.1.2.6

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms