Oracle VM: Memory Shortage Caused By "lldpad" Service Triggers Out Of Memory (OOM) Killer

(Doc ID 2393770.1)

Last updated on MAY 04, 2018

Applies to:

Oracle VM - Version 3.4.1 and later
Information in this document applies to any platform.

Symptoms

The server shuts down or restarts in an uncontrolled manner.  Sometimes the crash dump shows the lldpad process as being terminated but other processes could be killed as well.  This condition manifests itself like this:

kernel: [17696860.904252] xend invoked oom-killer: gfp_mask=0x2000d0, order=2, oom_score_adj=0  
kernel: [17696860.904255] xend cpuset=/ mems_allowed=0
kernel: [17696860.904261] CPU: 4 PID: 3618 Comm: xend Not tainted 4.1.12-61.1.9.el6uek.x86_64 #2
kernel: [17696860.904262] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 02/17/2017
kernel: [17696860.904264] 0000000000000000 ffff880075f0ba48 ffffffff816c6570 ffff8801f8508e00
kernel: [17696860.904267] 0000000000000000 ffff880075f0ba98 ffffffff8118bf7e ffff880000000002
kernel: [17696860.904269] ffffffff002000d0 0000000000000000 ffff880200cd9c00 ffff880200cda6b0
kernel: [17696860.904271] Call Trace:
kernel: [17696860.904279] [] dump_stack+0x63/0x83
kernel: [17696860.904283] [] dump_header+0x8e/0xe0
kernel: [17696860.904286] [] oom_kill_process+0x1d7/0x3c0
(snip)
kernel: [17696860.904386] Swap cache stats: add 1830833, delete 1828101, find 1108490938/1110930693
kernel: [17696860.904386] Free swap = 0kB  
kernel: [17696860.904387] Total swap = 4194300kB
kernel: [17696860.904388] 1572863 pages RAM   
kernel: [17696860.904389] 0 pages HighMem/MovableOnly
kernel: [17696860.904390] 53245 pages reserved
kernel: [17696860.904390] 4096 pages cma reserved
kernel: [17696860.904391] 0 pages hwpoisoned
kernel: [17696860.904392] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
kernel: [17696860.904400] [ 909] 0 909 2678 333 9 3 43 -1000 udevd
kernel: [17696860.904403] [ 2479] 0 2479 208181 1747 42 4 0 -1000 multipathd
kernel: [17696860.904405] [ 2825] 0 2825 22786 289 14 3 30 -1000 auditd
kernel: [17696860.904406] [ 2859] 0 2859 61373 951 23 4 71 0 rsyslogd
kernel: [17696860.904408] [ 2874] 0 2874 2999 642 12 3 15 0 irqbalance
kernel: [17696860.904409] [ 2903] 32 2903 4759 381 15 3 22 0 rpcbind
kernel: [17696860.904411] [ 2927] 29 2927 6919 445 20 3 83 0 rpc.statd
kernel: [17696860.904413] [ 2968] 0 2968 2040778 1010062 3990 10 1027463 -1000 lldpad 
kernel: [17696860.904416] [ 3030] 0 3030 2104 249 10 3 17 0 fcoemon 

In this example,the server has 4G memory and 4G paging store.  Notice that all of swap has been filled.  The lldpad process is shown with an OOM killer score of -1000 so the lldpad process has avoided the OOM killer; another process will be killed instead.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms