OSB Applications Hanging Due To Huge Load Average In Server
(Doc ID 2206025.1)
Last updated on MAY 15, 2023
Applies to:
Oracle Service Bus - Version 11.1.1.7.0 and laterInformation in this document applies to any platform.
Symptoms
The issue is whenever user is trying to start any OSB WebLogic domain server (does not matter if it is Admin server or any OSB managed server), the start sequence is taking unusually long time - 50-60 minutes as compared to the usual 3-4 minutes. Also, during the start sequence the load average of the physical node is bumping up rapidly leading to slow performance of the entire physical node.
User does not see the server going to Hung or Crashing - the only exception being only when the managed server is getting up, the admin server once goes to WARNING state due to STUCK threads since the managed server takes too much time to start.
In the AdminServer.out seeing the following error:
<Nov 8, 2016 5:33:01 AM EST> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "691" seconds working on the request "Workmanager: weblogic.kernel.Default, Version: 0, Scheduled=true, Started=true, Started time: 691798 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:152)
java.net.SocketInputStream.read(SocketInputStream.java:122)
weblogic.utils.io.ChunkedInputStream.read(ChunkedInputStream.java:159)
java.io.InputStream.read(InputStream.java:101)
com.certicom.tls.record.ReadHandler.readFragment(Unknown Source)
com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
com.certicom.tls.record.ReadHandler.read(Unknown Source)
com.certicom.io.InputSSLIOStreamWrapper.read(Unknown Source)
sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
java.io.InputStreamReader.read(InputStreamReader.java:184)
java.io.BufferedReader.fill(BufferedReader.java:154)
java.io.BufferedReader.readLine(BufferedReader.java:317)
java.io.BufferedReader.readLine(BufferedReader.java:382)
weblogic.nodemanager.client.NMServerClient.checkResponse(NMServerClient.java:289)
weblogic.nodemanager.client.NMServerClient.checkResponse(NMServerClient.java:314)
weblogic.nodemanager.client.NMServerClient.start(NMServerClient.java:102)
weblogic.nodemanager.mbean.StartRequest.start(StartRequest.java:75)
weblogic.nodemanager.mbean.StartRequest.execute(StartRequest.java:47)
weblogic.kernel.WorkManagerWrapper$1.run(WorkManagerWrapper.java:63)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:263)
weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
>
Changes
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |
| References |