EM 13.4: OMS Java Process Consuming High CPU and Causing Console Slowness / OMS Hung Intermittently (Doc ID 2702019.1)

Last updated on NOVEMBER 19, 2020

Applies to:

Symptoms

This issue can happen after upgrading the EM to 13.4 or in a new installation of EM 13.4

1. OMS java process is consuming high amount CPU usage intermittently.

2. During this time, users may observe console slowness. 

3. $../gc_inst/em/EMGC_OMS1/sysman/log/emctl.log may report the following errors too:

4. ReservedCodeCache will be nearly 100% full. This can be verified from:

• EM Console > Navigate to  Oracle Weblogic Server target EMGC_OMS*
• Click on Monitoring -> All metrics
• Search for "JVM Memory Pools"
• Expand JVM Memory Pools and select Memory Pool - Current usage
• Select "Code Cache" from the result page and this will load a graph
• This will be for last 24 hours. Verify the usage and see if it is closer to 100%
• Change "View Data" at the top of the table from 24 Hours to 7 days
• Verify the usage and see if it is closer to 100%

5.  From the thread dumps which are collected at the time of issue, JVM compiler threads are causing the high CPU.

This can be verified from the following steps:

5.1) Identify the OMS PID using following command

$ps -ef | grep EMGC_OMS1

$MW_HOME/oracle_common/jdk/bin/jstack <OMS PID> > /tmp/oms1_jstack.txt

5.2) Collect CPU usage of OMS threads

$ps -C java -L -o pcpu,cpu,nice,state,cputime,pid,tid | grep <OMS_PID> > /tmp/oms1_thred_cpu.txt

5.3) Convert hexadecimal nid to decimal

Search for "Compiler" key word to locate compiler threads in /tmp/oms1_jstack.txt
There will be multiple compiler threads.
Convert it's nid value from hexadecimal to decimal.

5.4) Find the cpu utilisation of compiler threads

Search for decimal nid value in /tmp/oms1_thred_cpu.txt
First column shows the CPU utilisation
Repeat the above 2 steps for remaining compiler nids

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.