EM12c, EM13c: Enterprise Manager Cloud Control Heap Usage Percent Metric for EMGC_OMSn target Reports Frequent Warning or Critical Alerts, Continuous Growth
Last updated on JANUARY 05, 2018
Applies to:Enterprise Manager Base Platform - Version 220.127.116.11.0 and later
Information in this document applies to any platform.
Frequent Warning or Critical Alerts and / or Email Notifications are generated from Enterprise Manager (EM) Cloud Control for Heap Usage % metric for EMGC_OMSn target. Eg:
Target type=Oracle WebLogic Server Target name=/EMGC_GCDomain/GCDomain/EMGC_OMS1 Categories=Capacity Message=The heap usage is 96%. Severity=Critical Event reported time=Jul 29, 2014 11:15:05 PM EDT Associated Incident Id=140 Associated Incident Status=New Associated Incident Owner= Associated Incident Acknowledged By Owner=No Associated Incident Priority=None Associated Incident Escalation Level=0 Event Type=Metric Alert Event name=jvm:heapUsedPercentage.value Metric Group=JVM Metrics Metric=Heap Usage (%) Metric value=96
Navigating to Cloud Control Console -> Targets -> All targets -> EMGC_OMSn target (eg: /EMGC_GCDomain/GCDomain/EMGC_OMS1) -> Weblogic Server -> Monitoring -> All Metrics -> JVM Metrics -> Heap Usage (%)
it can be observed that the Utilization percent grows continuously every day.
Additionally, the following symptoms may be observed:
1. Frequent log rotation for access.log files reported in <gc_inst>/user_projects/domains/GCDomain/servers/EMGC_OMSn/logs/EMGC_OMSn.out via entries such as:
<Jul 29, 2014 11:21:31 PM WEST> <Notice> <LoggingService> <BEA-320400> <The log file /app/oracle/oem/gc_inst/user_projects /domains/GCDomain/servers/EMGC_OMS1/logs/access.log will be rotated. Reopen the log file if tailing has stopped. This can happen on some platforms like Windows.>
<Jul 29, 2014 11:21:31 PM WEST> <Notice> <LoggingService> <BEA-320401> <The log file has been rotated to /app/oracle/oem/gc_inst/user_projects /domains/GCDomain/servers/EMGC_OMS1/logs/access.log01256. Log messages will continue to be logged in /app/oracle/oem/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/access.log.>
2. Errors such as "java.lang.OutOfMemoryError: Java heap space" and "danger of OutOfMemoryError" reported in <gc_inst>/user_projects/domains/GCDomain/servers/EMGC_OMSn/logs/EMGC_OMSn.out
<Jul 29, 2014 11:23:10 PM WEST> <Critical> <Health> <BEA-310003> <Free memory in the server is 3,416,688 bytes. There is danger of OutOfMemoryError> <Jul 29, 2014 11:47:40 PM WEST> <Error> <Server> <BEA-002608> <The ListenThread because of an error: java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: Java heap space
3. High number (about 30 times per min) of occurrences for GET requests for a particular page by an EM Administrator reported in <gc_inst>/user_projects/domains/GCDomain/servers/EMGC_OMSn/logs/access.log*
192.168.129.109 - OPERADOR [29/Jul/2014:11:01:35 +0000] "GET /em/faces/core-groups-systemHome?target=ORCL*
In a time period of around 3 hours, there could be as high as 2957 GET calls from the EM Administrator user.
There is an user session via browser established to the EM 12c console for the particular target which has Auto-Refresh turned on for continuous monitoring.
There is a Browser Plugin/Add-on configured to access Cloud Control Console page for a particular target for continuous monitoring (eg: plugin iMacros Sidebar)
Sign In with your My Oracle Support account
Don't have a My Oracle Support account? Click to get started
My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms