My Oracle Support Banner

OAM 11.1.2.2 Error "more than the configured time (StuckThreadMaxTime) of "600" seconds" in OAM Server Log File And Increasing Thread Count Causing Server To Become Unresponsive (Doc ID 1987247.1)

Last updated on AUGUST 09, 2018

Applies to:

Oracle Access Manager - Version 11.1.2.2.0 and later
Information in this document applies to any platform.

Symptoms

OAM Server is becoming unresponsive some time after restart, with error "more than the configured time (StuckThreadMaxTime) of "600" seconds" occurring in the OAM server log file and increasing thread count on the OAM managed server machine.

The OAM managed server must be restarted to resolve the issue temporarily.


The log file shows e.g.:

[
GET /oam/server/osso_login?Site2pstoreToken=v1.2~3E04B30DF4CD9E1D79~F5AB0E46AC7F49057C64274F923524BB9E12E63F44679E07DB1F276BE26B0087E7096021A3665EACBB475252276CB04533861270A0F122532061957BA6A39F0386886AEE7C7A96C823A34490BABEFFB1908235693FD794666B462D1780C6CD7F35F6A80464CB313AD94CAD8D702CD4FB41898A2534BA3675CB8E5920FB3CD1DFF7FBE3B47D134F744E9046C32E92F69428C5639A3C4BB5B690CE9C78B19BC84D2C9996479929AD22C44C840F1E9A09F80F2DCBD2DE7E358A0F4DC6075FFC38DDC46C5E78EDD3ECF89F3771497EDB62A027F0218C2E801EAD104F9F26E147EDCC HTTP/1.0
Accept: */*
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)
Accept-Encoding: gzip, deflate
Cookie: ....; __utma=22135204.107588595.1420545430.1420801513.1420805557.13; __utmz=22135204.1420545430.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); OAM_REQ_0=invalid; OAM_REQ_COUNT=VERSION_4~1; OAM_ID=VERSION_4~mQ5PQ21H4jQhF9vwTB7Efw==~b02WYwNNTObDez9P8yosrTzGLoUXxsjBNERUT8Xrr+NBgKspjrlbhZc8Qmjj3KADmhVyULE4p6DmFZE5D2Kwp3LuKf/zvJcAJ29PQh5D8HeB/tdFvCYobgKcnfSqjV+AZqrtXt6jvUjxxSlx1AEBCrhbbNVd0T4qttAvyF3Am4GLxzVEPlE5ItGRMR9ujPZBp22ZoX5lkKBfdu6eIH0xDT+DxImJt/sNeg0rfxR1dQ0PIb86uq0jnurGTGK/ol7h43HG3KNjojkmJY9QxbD0OeKYoCbsMxboqrA7U2qrOtQijVUT9L9JKSR+bemI0r4O

]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace: null>


The following have failed to resolve the issue:

1. Increased stuckthreadmaxtime from 600 to 6000. This merely reduced the frequency of the error.
2. Increased oam.coherence.distributed.threads from 8 to 12.
3. Increased OAM managed server JVM size to min 2GB and max 4GB.

The thread dump shows a lock / block in AESCipherFactory:

"[STUCK] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" id=133 WAITING on lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39c93a4a ExeuctionContext=[WLSExecutionContext instance: d118714ea35659d2:71959f8e:14ace7f7f16:-8000-0000000000008505,0
  mThreadId: 133
  mOrderIndex: -9223372036854775806
  mSuspended: false
  mCtxContentMap: null
  m_ctxGlobalMap: 0
  mCtxLocalMap: 3
  mInheritable: true
  mListeners: 1
family:WLSContextFamily instance:
  mECID: d118714ea35659d2:71959f8e:14ace7f7f16:-8000-0000000000008505
  mCtxMap: 1
  mGlobalMap: 0
  mPropagateKeys: null
  mLogKeys: null
  mLimitKeys: null
]
  at sun.misc.Unsafe.park(Native Method)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
  at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
  at oracle.security.am.common.utilities.crypto.CipherPool.lease(CipherPool.java:137)
  at oracle.security.am.common.utilities.crypto.AESCipherFactory.getCipherObject(AESCipherFactory.java:107)
  at oracle.security.am.common.utilities.crypto.AESEncrypter.decrypt(AESEncrypter.java:155)
  at oracle.security.am.engines.sso.token.server.SSOTokenV4Handler.decryptSessionToken(SSOTokenV4Handler.java:241)
  at oracle.security.am.engines.sso.token.server.AbstractSSOTokenHandler.parseSessionToken(AbstractSSOTokenHandler.java:182)
  at oracle.security.am.engines.sso.token.server.SSOTokenIssuerModuleImpl.validate(SSOTokenIssuerModuleImpl.java:223)
  at oracle.security.am.engines.tpe.TokenValidatorImpl.validate(TokenValidatorImpl.java:68)
  at oracle.security.am.engines.sso.adapter.TPEAdapter.validateToken(TPEAdapter.java:118)
  at oracle.security.am.pbl.protocol.plugin.oam.AMHelper.mapToSSOSession(AMHelper.java:529)
  at oracle.security.am.pbl.protocol.plugin.oam.AMHelper.getAMSessionToken(AMHelper.java:130)
  at oracle.security.am.pbl.protocol.plugin.oam.AMLoginRequestHandler.process(AMLoginRequestHandler.java:94)
  at oracle.security.am.pbl.PBLFlowManager.handleBaseEvent(PBLFlowManager.java:138)
  at oracle.security.am.pbl.PBLFlowManager.processRequest(PBLFlowManager.java:107)
  at oracle.security.am.pbl.transport.http.AMServlet.handleRequest(AMServlet.java:198)
  at oracle.security.am.pbl.transport.http.AMServlet.doPost(AMServlet.java:157)
  at oracle.security.am.pbl.transport.http.AMServlet.doGet(AMServlet.java:890)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  .....

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process and therefore has not been subject to an independent technical review.
My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.