Oracle Linux 7: Kernel Memory Leak for Veritas File System (Doc ID 2168986.1)

Last updated on JUNE 22, 2017

Applies to:

Linux OS - Version Oracle Linux 7.0 and later
Linux x86-64

Symptoms

Oracle Database becomes unresponsive, and below symptoms are noticed on OL7 with Veritas file system.

1. Out of Memory in OS logs
2. High usage of slab
3. Huge number of objects for kmalloc-1024

Jun 25 11:52:41 <hostname> kernel: Out of memory: Kill process 40247 (oracle) score 0 or sacrifice child
Jun 25 11:52:41 <hostname> kernel: Killed process 40247 (oracle) total-vm:157584396kB, anon-rss:0kB, file-rss:1648kB
Jun 25 11:52:41 <hostname> kernel: oracle invoked oom-killer: gfp_mask=0x42d0, order=3, oom_score_adj=0
Jun 25 11:52:41 <hostname> kernel: oracle cpuset=/ mems_allowed=0-3
Jun 25 11:52:41 <hostname> kernel: CPU: 6 PID: 112912 Comm: oracle Tainted: P OE ------------ 3.10.0-327.10.1.el7.x86_64 #1
Jun 25 11:52:41 <hostname> kernel: Hardware name: Dell Inc. PowerEdge R920/0Y4CNC, BIOS 1.3.2 09/30/2014
Jun 25 11:52:41 <hostname> kernel: ffff88240bdb0000 000000000a4eeef6 ffff880cf48a7910 ffffffff816352a4
Jun 25 11:52:41 <hostname> kernel: ffff880cf48a79a0 ffffffff8163023f ffff882e3a1e7fa0 ffff882e3a1e7fb8
Jun 25 11:52:41 <hostname> kernel: ffffffff00000202 fafeebff00000000 0000000000000002 ffffffff81128903
Jun 25 11:52:41 <hostname> kernel: Call Trace:
Jun 25 11:52:41 <hostname> kernel: [<ffffffff816352a4>] dump_stack+0x19/0x1b
Jun 25 11:52:41 <hostname> kernel: [<ffffffff8163023f>] dump_header+0x8e/0x214
Jun 25 11:52:41 <hostname> kernel: [<ffffffff81128903>] ? delayacct_end+0x83/0xb0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff8116ce4e>] oom_kill_process+0x24e/0x3b0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff81088d8e>] ? has_capability_noaudit+0x1e/0x30
Jun 25 11:52:41 <hostname> kernel: [<ffffffff8116d676>] out_of_memory+0x4b6/0x4f0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff81173855>] __alloc_pages_nodemask+0xa95/0xb90
Jun 25 11:52:41 <hostname> kernel: [<ffffffff811b4429>] alloc_pages_current+0xa9/0x170
Jun 25 11:52:41 <hostname> kernel: [<ffffffff81514bc0>] sk_page_frag_refill+0x70/0x160
Jun 25 11:52:41 <hostname> kernel: [<ffffffff81576ca3>] tcp_sendmsg+0x263/0xc20
Jun 25 11:52:41 <hostname> kernel: [<ffffffff815a1074>] inet_sendmsg+0x64/0xb0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff812889f3>] ? selinux_socket_sendmsg+0x23/0x30
Jun 25 11:52:41 <hostname> kernel: [<ffffffff8150ff37>] sock_aio_write+0x157/0x180
Jun 25 11:52:41 <hostname> kernel: [<ffffffff811ddd7d>] do_sync_write+0x8d/0xd0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff811de695>] vfs_write+0x1b5/0x1e0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff811df03f>] SyS_write+0x7f/0xe0
Jun 25 11:52:41 <hostname> kernel: [<ffffffff81645a49>] system_call_fastpath+0x16/0x1b

slab : 

zzz ***
MemTotal: 528142476 kB
MemFree: 789636 kB
MemAvailable: 995184 kB
Slab: 234003392 kB

Huge number of objects is used for kmalloc-1024  :

zzz ***
slabinfo - version: 2.1
# name <active_objs> <num_objs> : tunables : slabdata <active_slabs> <num_slabs>
kmalloc-1024 231175235 231175452 1024 32 8 : tunables 0 0 0 : slabdata 7601756 7601756 0
kmalloc-512 1569361 1623456 512 32 4 : tunables 0 0 0 : slabdata 50739 50739 0

 

 

 

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms