Em13c: OMS Console Unavailable and status OMS Command shows "Unexpected error occurred. Check error and log files."

(Doc ID 2391043.1)

Last updated on APRIL 25, 2018

Applies to:

Enterprise Manager Base Platform - Version 13.2.0.0.0 and later
Information in this document applies to any platform.

Symptoms

EM 13c OMS Console is unavailable intermittently and emctl status oms command shows the following message:

Oracle Management Server is not functioning because of the following reason:
Unexpected error occurred. Check error and log files.

$../gc_inst/em/EMGC_OMS1/sysman/log/emctl.log file reports below errors:

2018-04-10 05:45:42,842 [main] ERROR oms.StatusOMSCmd processStatusOMS.352 - Exception when trying to hit OMS page.
java.net.SocketTimeoutException: Read timed out
2018-04-10 05:45:42,847 [main] ERROR commands.BaseCommand logAndPrint.651 - Oracle Management Server is not functioning because of the following reason:
Unexpected error occurred. Check error and log files.

Around this time, $../gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out file reports following errors:

JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError"- please wait.
JVMDUMP032I JVM requested Heap dump using '/u01/app/oracle/product/gc_inst1/user_projects/domains/GCDomain/heapdump.20180417.110321.4522676.0008.phd' in response to an event
JVMDUMP010I Heap dump written to /u01/app/oracle/product/gc_inst1/user_projects/domains/GCDomain/heapdump.20180417.110321.4522676.0008.phd
JVMDUMP032I JVM requested Java dump using '/u01/app/oracle/product/gc_inst1/user_projects/domains/GCDomain/javacore.20180417.110321.4522676.0009.txt' in response to an event
JVMDUMP010I Java dump written to /u01/app/oracle/product/gc_inst1/user_projects/domains/GCDomain/javacore.20180417.110321.4522676.0009.txt
JVMDUMP032I JVM requested Snap dump using '/u01/app/oracle/product/gc_inst1/user_projects/domains/GCDomain/Snap.20180417.110321.4522676.0010.trc' in response to an event
JVMDUMP010I Snap dump written to /u01/app/oracle/product/gc_inst1/user_projects/domains/

From Java core files (Example: $../gc_inst/user_projects/domains/GCDomain/javacore.20180417.110321.4522676.0009.txt), we can see the cause is due to:

4XESTACKTRACE at weblogic/security/providers/authentication/LDAPAtnLoginModuleImpl.searchGroupUnlimited(LDAPAtnLoginModuleImpl.java:638(Compiled Code))
4XESTACKTRACE at weblogic/security/providers/authentication/LDAPAtnLoginModuleImpl.searchGroupUnlimited(LDAPAtnLoginModuleImpl.java:638(Compiled Code))
4XESTACKTRACE at weblogic/security/providers/authentication/LDAPAtnLoginModuleImpl.searchGroupUnlimited(LDAPAtnLoginModuleImpl.java:638(Compiled Code))

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms