EM12c: Adding JVMD Agents However Not All JVMs Are Showing In The Pool (Doc ID 1669262.1)

Last updated on FEBRUARY 27, 2017

Applies to:

APM - Application Performance Management - Version 12.1.0.3.0 to 12.1.0.6.0 [Release 12.1]
Information in this document applies to any platform.

Goal

We are deploying JVMD agents to a 3 server weblogic domain using the instructions in (Oracle® Enterprise Manager Cloud Control Basic Installation Guide 12c Release 3 (12.1.0.3) E22624-27), however after successful completion of the deployment job we see a handful on the JVMs in the resulting JVM pool.

We have tried deinstalling the JVMD agents and retrying using (Oracle® Enterprise Manager Cloud Control Advanced Installation and Configuration Guide 12c Release 3 (12.1.0.3) E24089-27) yet that has made no changes. We are left with the same JVMs in the pool after completion.

In the JVMDMANAGER logs, we see the following for one of the JVMs:

..
<Feb-26-2014 16:42:50.892> <INFO> <JVMD|tid-54|util.Ad4jTarget> <Updated target properties for /DEV_HYPERION121_int_EPMSystem/EPMSystem/CalcMgr1_jvm>
<Feb-26-2014 16:42:50.896> <WARN> <JVMD|tid-54|util.JvmdTargetConfigLoader> <getTargetGuid: Failed to get GUID for the target (targetName=/DEV_HYPERION121_int_EPMSystem/EPMSystem/CalcMgr1_jvm targetType=java_virtual_machine)>
<Feb-26-2014 16:42:50.896> <WARN> <JVMD|tid-54|util.JvmdTargetConfigLoader> <getTargetGuid: Failed to get GUID for the target (targetName=/DEV_HYPERION121_int_EPMSystem/EPMSystem/CalcMgr1_jvm targetType=java_virtual_machine)>
java.sql.SQLException: ORA-20206: Target does not exist: /DEV_HYPERION121_int_EPMSystem/EPMSystem/CalcMgr1_jvm:java_virtual_machine
ORA-06512: at "SYSMAN.MGMT_TARGET", line 1067
ORA-06512: at line 1

   at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:462)
   at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:405)
   at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:931)
   at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:481)
   at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:205)
   at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:548)
   at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:213)
   at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1111)
   at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1488)
   at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3769)
   at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3954)
   at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:9353)
   at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1539)
   at oracle.sysman.e2e.model.ad4j.util.JvmdTargetConfigLoader.getTargetGuid(JvmdTargetConfigLoader.java:76)
   at oracle.sysman.e2e.model.ad4j.util.JvmdTargetConfigLoader.updateTargetConfig(JvmdTargetConfigLoader.java:645)
   at oracle.sysman.e2e.model.ad4j.util.Ad4jTarget.setAd4jTargetWithAssociation(Ad4jTarget.java:1321)
   at oracle.sysman.e2e.model.ad4j.util.Ad4jTarget.updateAd4jTargetWithAssociation(Ad4jTarget.java:1454)
   at oracle.sysman.e2e.model.ad4j.remote.Jamlisten.informAgentConnect(Jamlisten.java:1149)
   at oracle.sysman.e2e.model.ad4j.business.jvmInfo.parseJvmString(jvmInfo.java:978)
   at oracle.sysman.e2e.model.ad4j.business.jvmInfo.initvar(jvmInfo.java:614)
   at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.processCachedHello(AD4JCommServlet.java:1003)
   at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.handleHandshakeComplete(AD4JCommServlet.java:801)
   at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.processNonHelloMsg(AD4JCommServlet.java:694)
   at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.processAgentRequest(AD4JCommServlet.java:575)
   at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.doRequest(AD4JCommServlet.java:313)
   at weblogic.servlet.http.AbstractAsyncServlet.service(AbstractAsyncServlet.java:99)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
   at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
   at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
   at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
   at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184)
   at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3732)
   at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3696)
   at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
   at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
   at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2273)
   at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2179)
   at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1490)
   at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
   at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
...


After downloading and consuming the latest bundle for OMS 12.1.0.5.3, patch 18108942 (which contains fix for bug 18083864) we are still having the same issue.

We now are seeing the following in the logs:

...
<Apr-04-2014 12:01:43.410> <WARN> <JVMD|tid-11|util.Ad4jTarget> <Could not update target properties>
java.sql.SQLException: ORA-20206: Target does not exist:  /DEV_HYPERION121_int_EPMSystem/EPMSystem/CalcMgr1_jvm:java_virtual_machine
ORA-06512: at "SYSMAN.MGMT_TARGET", line 1067
ORA-06512: at line 1

    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:462)
    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:405)
    at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:931)
    at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:481)
    at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:205)
    at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:548)
    at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:213)
    at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1111)
    at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1488)
    at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3769)
    at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3954)
    at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:9353)
    at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1539)
    at oracle.sysman.e2e.model.ad4j.util.JvmdTargetConfigLoader.getTargetGuid(JvmdTargetConfigLoader.java:75)
    at oracle.sysman.e2e.model.ad4j.util.JvmdTargetConfigLoader.updateTargetConfig(JvmdTargetConfigLoader.java:644)
    at oracle.sysman.e2e.model.ad4j.util.Ad4jTarget.setAd4jTargetWithAssociation(Ad4jTarget.java:1534)
    at oracle.sysman.e2e.model.ad4j.util.Ad4jTarget.updateAd4jTargetWithAssociation(Ad4jTarget.java:1642)
    at oracle.sysman.e2e.model.ad4j.remote.Jamlisten.informAgentConnect(Jamlisten.java:1345)
    at oracle.sysman.e2e.model.ad4j.business.jvmInfo.parseJvmString(jvmInfo.java:1012)
    at oracle.sysman.e2e.model.ad4j.business.jvmInfo.initvar(jvmInfo.java:657)
    at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.processCachedHello(AD4JCommServlet.java:1026)
    at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.handleHandshakeComplete(AD4JCommServlet.java:824)
    at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.processNonHelloMsg(AD4JCommServlet.java:717)
    at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.processAgentRequest(AD4JCommServlet.java:598)
    at oracle.sysman.e2e.model.ad4j.remote.servlet.AD4JCommServlet.doRequest(AD4JCommServlet.java:334)
    at weblogic.servlet.http.AbstractAsyncServlet.service(AbstractAsyncServlet.java:99)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
    at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3732)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3696)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
    at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2273)
    at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2179)
    at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1490)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:221) 

...

 

 

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms