Fusion Applications OTBI: HTTP Error 500--Internal Server Error, Unable to connect to the BI Presentation Server, When I Tried To Create Analysis, caused by Stuck Threads (Doc ID 1633926.1)

Last updated on SEPTEMBER 21, 2016

Applies to:

Oracle Fusion Common CRM Cloud Service - Version 11.1.4.0.0 and later
Oracle Fusion Application Toolkit - Version 11.1.1.5.1 and later
Oracle Fusion Application Toolkit Cloud Service - Version 11.1.4.0.0 and later
Information in this document applies to any platform.

Symptoms

Fusion CRM Cloud 11.1.7 - error when attempt to Create Report/analysis

This was working fine yesterday


You encounter the error below when from "Reports and Analytics" I try to Create an analysis

ERROR1
-------------

Error 500--Internal Server Error
From RFC 2068 Hypertext Transfer Protocol -- HTTP/1.1:
10.5.1 500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

You will find below the error msg that you can see from Sales Page...

ERROR 2
---------------

Unable to connect to the BI Presentation Server. Please ensure that it is running, configured properly, and that the connection details within this application are correct.Exceeded failed connection tries. ConnectionId=BIPresentationServer
Unable to connect to the BI Presentation Server. Please ensure that it is running, configured properly, and that the connection details within this application are correct.Exceeded failed connection tries. ConnectionId=BIPresentationServer

ERROR3
--------------

When you tried to connect to Analytics with the url https://bi-llfsn9xwrf.oracleoutsourcing.com/analytics/saw.dll?catalog I recieved only a blank page with the message "Signing In ...."

 

DAGNOSTICS

--------------------

Cloud EM showed that all components are up and running 

Logs showed:


<Mar 7, 2014 1:34:57 PM UTC> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "638" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 638758 ms
[
POST /analytics-ws/saw.dll?SoapImpl=nQSessionService HTTP/1.1
Content-type: text/xml; charset=utf-8
Soapaction: "#impersonateex"
Accept: text/xml, multipart/related, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
User-Agent: Oracle JAX-WS 2.1.5
Content-Length: 3576
VIRTUAL_HOST_TYPE: internal
ECID-Context: 1.004x0GfncMrCKujK_2e9Uq0000Pp0009Yd;kYjE0ZDLIPGJj1QRXSPUoPROnTBTnVBSXNUB^SOR
Connection: Keep-Alive
X-Forwarded-For: 138.3.146.13
Proxy-Client-IP: 138.3.146.13
X-WebLogic-KeepAliveSecs: 30
X-WebLogic-Request-ClusterInfo: true
x-weblogic-cluster-hash: TmWDOFbCvdWrXJoDWhn/uy6Y6ok

]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-230 "[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
  jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
  jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
  java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
  java.net.SocketInputStream.read(SocketInputStream.java:107)
  com.siebel.analytics.web.sawconnect.SAWConnection$NotifyInputStream.read(SAWConnection.java:165)
  java.io.BufferedInputStream.fill(BufferedInputStream.java:189)
  java.io.BufferedInputStream.read(BufferedInputStream.java:236)
  ^-- Holding lock: java.io.BufferedInputStream@27c87e1e[thin lock]
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocol.readInt(SAWProtocol.java:188)
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.readChunkHeader(SAWProtocolInputStreamImpl.java:257)
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.startReadingNewMessage(SAWProtocolInputStreamImpl.java:49)
  com.siebel.analytics.web.sawconnect.SAWServletHttpBinding.forwardResponse(SAWServletHttpBinding.java:200)
  com.siebel.analytics.web.SAWBridge.processRequest(SAWBridge.java:133)
  com.siebel.analytics.web.SAWBridge.doPost(SAWBridge.java:235)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:700)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:815)
  weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:224)
  weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:108)
  weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:207)
  weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  com.siebel.analytics.web.integration.LoadBalancerHTTPFilter.doFilter(LoadBalancerHTTPFilter.java:175)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  com.siebel.analytics.web.integration.FirewallFilter.doFilter(FirewallFilter.java:69)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:119)
  oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:310)
  oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:452)
  oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)
  oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:107)
  oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:70)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.security.wls.filter.SSOSessionSynchronizationFilter.doFilter(SSOSessionSynchronizationFilter.java:259)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:90)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3711)
  weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3705)
  weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:308)
  weblogic.security.service.SecurityManager.runAs(SecurityManager.java:116)
  weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2218)
  weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2133)
  weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1458)
  weblogic.work.ExecuteThread.execute(ExecuteThread.java:250)
  weblogic.work.ExecuteThread.run(ExecuteThread.java:213)
}

>
<Mar 7, 2014 1:34:57 PM UTC> <Notice> <Diagnostics> <BEA-320068> <Watch 'StuckThread' with severity 'Notice' on server 'bi_server1' has triggered at Mar 7, 2014 1:34:57 PM UTC. Notification details:
WatchRuleType: Log
WatchRule: (SEVERITY = 'Error') AND ((MSGID = 'WL-000337') OR (MSGID = 'BEA-000337'))
WatchData: DATE = Mar 7, 2014 1:34:57 PM UTC SERVER = bi_server1 MESSAGE = [STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "638" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 638758 ms
[
POST /analytics-ws/saw.dll?SoapImpl=nQSessionService HTTP/1.1
Content-type: text/xml; charset=utf-8
Soapaction: "#impersonateex"
Accept: text/xml, multipart/related, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
User-Agent: Oracle JAX-WS 2.1.5
Content-Length: 3576
VIRTUAL_HOST_TYPE: internal
ECID-Context: 1.004x0GfncMrCKujK_2e9Uq0000Pp0009Yd;kYjE0ZDLIPGJj1QRXSPUoPROnTBTnVBSXNUB^SOR
Connection: Keep-Alive
X-Forwarded-For: 138.3.146.13
Proxy-Client-IP: 138.3.146.13
X-WebLogic-KeepAliveSecs: 30
X-WebLogic-Request-ClusterInfo: true
x-weblogic-cluster-hash: TmWDOFbCvdWrXJoDWhn/uy6Y6ok

]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-230 "[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
  jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
  jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
  java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
  java.net.SocketInputStream.read(SocketInputStream.java:107)
  com.siebel.analytics.web.sawconnect.SAWConnection$NotifyInputStream.read(SAWConnection.java:165)
  java.io.BufferedInputStream.fill(BufferedInputStream.java:189)
  java.io.BufferedInputStream.read(BufferedInputStream.java:236)
  ^-- Holding lock: java.io.BufferedInputStream@27c87e1e[thin lock]
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocol.readInt(SAWProtocol.java:188)
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.readChunkHeader(SAWProtocolInputStreamImpl.java:257)
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.startReadingNewMessage(SAWProtocolInputStreamImpl.java:49)
  com.siebel.analytics.web.sawconnect.SAWServletHttpBinding.forwardResponse(SAWServletHttpBinding.java:200)
  com.siebel.analytics.web.SAWBridge.processRequest(SAWBridge.java:133)
  com.siebel.analytics.web.SAWBridge.doPost(SAWBridge.java:235)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:700)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:815)
  weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:224)
  weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:108)
  weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:207)
  weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  com.siebel.analytics.web.integration.LoadBalancerHTTPFilter.doFilter(LoadBalancerHTTPFilter.java:175)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  com.siebel.analytics.web.integration.FirewallFilter.doFilter(FirewallFilter.java:69)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:119)
  oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:310)
  oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:452)
  oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)
  oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:107)
  oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:70)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.security.wls.filter.SSOSessionSynchronizationFilter.doFilter(SSOSessionSynchronizationFilter.java:259)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:90)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3711)
  weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3705)
  weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:308)
  weblogic.security.service.SecurityManager.runAs(SecurityManager.java:116)
  weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2218)
  weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2133)
  weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1458)
  weblogic.work.ExecuteThread.execute(ExecuteThread.java:250)
  weblogic.work.ExecuteThread.run(ExecuteThread.java:213)
}

SUBSYSTEM = WebLogicServer USERID = <WLS Kernel> SEVERITY = Error THREAD = [ACTIVE] ExecuteThread: '32' for queue: 'weblogic.kernel.Default (self-tuning)' MSGID = BEA-000337 MACHINE = llfsn9xwrf04.oracleoutsourcing.com TXID =  CONTEXTID = 23ca7f9d24f36eb1:-10b76ef7:14498de4e05:-8000-000000000002cde3 TIMESTAMP = 1394199297616  
WatchAlarmType: AutomaticReset
WatchAlarmResetPeriod: 600000
>
<Mar 7, 2014 1:34:58 PM UTC> <Emergency> <oracle.dfw.incident> <BEA-000000> <incident 463 created with problem key "BEA-000337 [WebLogicServer]">
[INFO ][memory ][Fri Mar  7 13:35:00 2014][1394199300010][03211] [YC#750] 65268.086-65268.111: YC 965107KB->734992KB (1048576KB), 0.025 s, sum of pauses 24.948 ms, longest pause 24.948 ms.
[INFO ][memory ][Fri Mar  7 13:35:01 2014][1394199301030][03211] [YC#751] 65269.112-65269.131: YC 970901KB->737778KB (1048576KB), 0.019 s, sum of pauses 18.610 ms, longest pause 18.610 ms.
[INFO ][memory ][Fri Mar  7 13:35:02 2014][1394199302475][03211] [YC#752] 65270.559-65270.577: YC 971897KB->738742KB (1048576KB), 0.018 s, sum of pauses 18.025 ms, longest pause 18.025 ms.
[INFO ][memory ][Fri Mar  7 13:35:03 2014][1394199303760][03211] [YC#753] 65271.843-65271.861: YC 972647KB->739482KB (1048576KB), 0.019 s, sum of pauses 18.298 ms, longest pause 18.298 ms.
<Mar 7, 2014 1:35:57 PM UTC> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "698" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 698773 ms
[
POST /analytics-ws/saw.dll?SoapImpl=nQSessionService HTTP/1.1
Content-type: text/xml; charset=utf-8
Soapaction: "#impersonateex"
Accept: text/xml, multipart/related, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
User-Agent: Oracle JAX-WS 2.1.5
Content-Length: 3576
VIRTUAL_HOST_TYPE: internal
ECID-Context: 1.004x0GfncMrCKujK_2e9Uq0000Pp0009Yd;kYjE0ZDLIPGJj1QRXSPUoPROnTBTnVBSXNUB^SOR
Connection: Keep-Alive
X-Forwarded-For: 138.3.146.13
Proxy-Client-IP: 138.3.146.13
X-WebLogic-KeepAliveSecs: 30
X-WebLogic-Request-ClusterInfo: true
x-weblogic-cluster-hash: TmWDOFbCvdWrXJoDWhn/uy6Y6ok

]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
Thread-230 "[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
  jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
  jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:24)
  java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
  java.net.SocketInputStream.read(SocketInputStream.java:107)
  com.siebel.analytics.web.sawconnect.SAWConnection$NotifyInputStream.read(SAWConnection.java:165)
  java.io.BufferedInputStream.fill(BufferedInputStream.java:189)
  java.io.BufferedInputStream.read(BufferedInputStream.java:236)
  ^-- Holding lock: java.io.BufferedInputStream@27c87e1e[thin lock]
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocol.readInt(SAWProtocol.java:188)
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.readChunkHeader(SAWProtocolInputStreamImpl.java:257)
  com.siebel.analytics.web.sawconnect.sawprotocol.SAWProtocolInputStreamImpl.startReadingNewMessage(SAWProtocolInputStreamImpl.java:49)
  com.siebel.analytics.web.sawconnect.SAWServletHttpBinding.forwardResponse(SAWServletHttpBinding.java:200)
  com.siebel.analytics.web.SAWBridge.processRequest(SAWBridge.java:133)
  com.siebel.analytics.web.SAWBridge.doPost(SAWBridge.java:235)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:700)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:815)
  weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:224)
  weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:108)
  weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:207)
  weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  com.siebel.analytics.web.integration.LoadBalancerHTTPFilter.doFilter(LoadBalancerHTTPFilter.java:175)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  com.siebel.analytics.web.integration.FirewallFilter.doFilter(FirewallFilter.java:69)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:119)
  oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:310)
  oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:452)
  oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)
  oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:107)
  oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:70)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.security.wls.filter.SSOSessionSynchronizationFilter.doFilter(SSOSessionSynchronizationFilter.java:259)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:90)
  weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:59)
  weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3711)
  weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3705)
  weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:308)
  weblogic.security.service.SecurityManager.runAs(SecurityManager.java:116)
  weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2218)
  weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2133)
  weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1458)
  weblogic.work.ExecuteThread.execute(ExecuteThread.java:250)
  weblogic.work.ExecuteThread.run(ExecuteThread.java:213)
}

>

FileName
----------------
biserver1.out

FileComment
----------------------
shows stuck threads during communications with OBIPS


Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms