My Oracle Support Banner

EM 10g: Grid Control Agent Crashes on AIX with Message: ERROR=12:There is not enough memory available now (Doc ID 396368.1)

Last updated on NOVEMBER 09, 2019

Applies to:

Enterprise Manager Base Platform - Version 10.2.0.1 to 10.2.0.5 [Release 10.2]
IBM AIX on POWER Systems (64-bit)
***Checked for relevance on 03-08-2012***
Checked for relevance on 04-Jan-2014

Symptoms

The Grid Agent on AIX crashes with 'There is not enough memory available now'. If the Agent is re-started, it crashes again after a few minutes.

The <AGENT_HOME>/sysman/log/emagent.trcshows:

<timestamp> Thread-26981 INFO ThreadPool: High WaterMark is : 164
<timestamp> Thread-26981 ERROR ThreadPool: Could not create thread,
ERROR=12:There is not enough memory available now.

OR

<timestamp> Thread-1503 ERROR fetchlets.oslinetok: Process stderr =
exec(): 0509-036 Cannot load program /u01/app/oracle/product/agent.10.2.
0/agent10g/bin/nmupm because of the following errors:
0509-022 Cannot load module /u01/app/oracle/product/agent.10.2.
0/agent10g/lib32/libnmemso.so.
0509-150 Dependent module libttsh10.a(shr_ttsh10.o) could not be loaded.
0509-022 Cannot load module libttsh10.a(shr_ttsh10.o).
0509-026 System error: There is not enough memory available now.
0509-022 Cannot load module nmupm.
0509-150 Dependent module /u01/app/oracle/product/agent.10.2.0/agent10g
/lib32/libnmemso.so could not be loaded.
0509-022 Cannot load module .


Attaching the Agent to truss shows:

(pid=7065636): starting emagent version 10.2.0.2.0
(pid=7065636): emagent started successfully
kgefec: fatal error 0
OCI-21503: program terminated by fatal error
OCI-04030: out of process memory when trying to allocate 1036 bytes (koh dur
heap d,koh image)


The <AGENT_HOME>/sysman/log/emagent.nohup could have:

JVMDG217: Dump Handler is Processing a Signal - Please Wait.
JVMDG303: JVM Requesting Java core file
JVMDG304: Java core file written to /o01
/app/oracle/product/em/10/agent/sysman/emd/javacore5980178.1109168476.txt
JVMDG215: Dump Handler has Processed Exception Signal 11.
(pid=5980178): signal handler called due to abnormal condition; dumping core
due to signal SIGSEGV
(pid=5980178): signal handler called due to abnormal condition; dumping core
due to signal SIGSEGV
JVMDG217: Dump Handler is Processing a Signal - Please Wait.
JVMDG303: JVM Requesting Java core file
JVMDG304: Java core file written to /o01/app/orac
le/product/em/10/agent/sysman/emd/javacore5980178.1109168476.txt
JVMDG215: Dump Handler has Processed Error Signal 6.
----- Wed Feb 23 09:21:40 2005::EMAgent exited at Wed Feb 23 09:21:40 2005 with
signal 0 -----
----- Wed Feb 23 09:21:40 2005::EMAgent has exited due to an internal error
...

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.