11g Grid Control Agent Consumes High Virtual / Resident Memory and may Restart Frequently (Doc ID 1194748.1)

Last updated on AUGUST 25, 2015

Applies to:

Enterprise Manager Base Platform - Version 11.1.0.1 to 11.1.0.1 [Release 11.1]
Information in this document applies to any platform.
Checked for relevance on 15-Feb-2014

Symptoms

The 11g EM Agent restarts at frequent intervals with the following entries in the <AGENT_HOME>/sysman/log/emagent.nohup file

emagent started successfully
URLTiming: Using SunX509
Exception in thread "oracle.dms.collector.GatherThread@15000"
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$EntryIterator.next(HashMap.java:834)
at java.util.HashMap$EntryIterator.next(HashMap.java:832)
at oracle.dms.collector.Gatherer.cleanup(Gatherer.java:235)
at oracle.dms.collector.GatherThread.run(GatherThread.java:85)
--- Recycling process. VMSize is 6385.46875 MB increased by 6327.09375 MB in
past 3 ---
----- Thu Jul 15 02:14:08 2010::Received restart request from EMAgent : 12200
-----
----- Thu Jul 15 02:14:08 2010::Stopping EMAgent : 12200 -----
(pid=12200): emagent now exiting normally


The above re-start occurs as the Virtual memory consumed by the Agent crossed the limit and hence it is automatically re-started by the daemon process.

-  The <AGENT_HOME>/sysman/log/emagent_fetchlet.trc also shows many errors related to the JNIFetchlet:

<timestamp> [nmefmgr_getJNIFetchlet] WARN emd.fetchlets getMetric.387 - oracle.sysman.emSDK.emd.fetchlet.FetchletException:
oracle.sysman.emSDK.emd.fetchlet.FetchletException: java.io.IOException:
Couldn't connect to any host
oracle.sysman.emSDK.emd.fetchlet.FetchletException:
oracle.sysman.emSDK.emd.fetchlet.FetchletException: java.io.IOException:
Couldn't connect to any host
at
..............
<timestamp> [nmefmgr_getJNIFetchlet] WARN JMX.generic logp.251 - IOException: cannot communicate to Target with
serviceURL=service:jmx:t3s://agentmachine.domain:7101/jndi/weblogic.management.mbeanservers.runtime for metric weblogic_j2eeserver:server::
Couldn't connect to any host Cause: javax.naming.NamingException: Couldn't connect to any host [Root exception is org.omg.CORBA.COMM_FAILURE: vmcid:
SUN minor code: 203 completed: No]

-  Historical data for the 'Resident Memory Utilization (%) ' or 'Virtual Memory Utilization Growth (%) '  metrics for this Agent show large memory usage over the period, when the Agent was up.
    These details can be accessed from the Grid Console Setup -> Agents -> Click on the Agent Name -> Click on 'All Metrics' -> Expand 'Agent Process Statistics' ->
Click on the 'Resident Memory Utilization (%) ' or 'Virtual Memory Utilization Growth (%) '.

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms