Sites Server Experiencing Outages Intermittently Due To Stuck Threads in RUNNABLE state (Doc ID 2177776.1)

Last updated on MARCH 21, 2023

Applies to:

Symptoms

Sites management server is experiencing outages and performance issues intermittently with the following relevant thread dump output:

"[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00007f17b0019800 nid=0x26b8 runnable [0x00007f17a1cfe000]
java.lang.Thread.State: RUNNABLE
at com.openmarket.ICS.listloop.doAfterBody(listloop.java:115)
at jsp_servlet._jsp._cs_deployed._<some_custom_location>._navigation._jspService(__<some_custom_navigation>.java:1507)
at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184)
at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:526)
at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:447)
at COM.FutureTense.Servlet.ServletRequest.include(ServletRequest.java:1430)
at COM.FutureTense.Servlet.FRequestObj.include(FRequestObj.java:1422)
at COM.FutureTense.Servlet.JSPServices.runJSP(JSPServices.java:111)
at COM.FutureTense.Platform.FileSystem.FILESYSTEMJSPManager.runJSP(FILESYSTEMJSPManager.java:463)
at COM.FutureTense.Servlet.JSPServices.runJSPObject(JSPServices.java:50)
at COM.FutureTense.Platform.FileSystem.FILESYSTEMJSPManager$JSPDataFile.run(FILESYSTEMJSPManager.java:190)
at COM.FutureTense.Common.ContentServer.jspExecute(ContentServer.java:3036)
at COM.FutureTense.Common.ContentServer.evalTemplate(ContentServer.java:2630)
at COM.FutureTense.Common.ContentServer.processElement(ContentServer.java:2008)
at COM.FutureTense.Common.CS.CallElement(CS.java:1603)
at com.openmarket.xcelerate.jsp.render.CallElement.doEndTag(CallElement.java:107)
at com.openmarket.framework.jsp.Base.doEndTag(Base.java:143)
at jsp_servlet._jsp._cs_deployed._<some_custom_location>._common.__header._jspService(__header.java:1865)

The WebLogic Server log had this relevant information:

<[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <> <BEA-000337> <[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "622" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 622026 ms
...
Connection: Keep-Alive

]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
COM.FutureTense.Util.ftStatusCode.compose(ftStatusCode.java:464)
COM.FutureTense.Util.ftStatusCode.getString(ftStatusCode.java:54)
COM.FutureTense.CatalogManager.CatalogEngine.failure(CatalogEngine.java:4332)
COM.FutureTense.CatalogManager.CatalogEngine.cmdfailure(CatalogEngine.java:4285)
COM.FutureTense.Tree.TreeEngine.cmdfailure(TreeEngine.java:1126)
COM.FutureTense.Tree.TreeEngine.getNodes(TreeEngine.java:440)
COM.FutureTense.Common.TreeManager.processCmd(TreeManager.java:461)
COM.FutureTense.Common.TreeManager.execute(TreeManager.java:187)
COM.FutureTense.Common.ContentServer.runManager(ContentServer.java:3472)
COM.FutureTense.Common.CS.TreeManager(CS.java:1577)
COM.FutureTense.Common.CS.TreeManager(CS.java:1530)
com.openmarket.framework.objects.TreeCatalog.GetChildren(TreeCatalog.java:370)
com.openmarket.framework.objects.AbstractNode.Children(AbstractNode.java:148)
com.openmarket.xcelerate.asset.Asset.Children(Asset.java:6282)
com.openmarket.xcelerate.asset.Asset.LoadAllChildren(Asset.java:3549)
com.openmarket.xcelerate.asset.Asset.Read(Asset.java:3361)
com.openmarket.assetframework.complexasset.ComplexAsset.superread(ComplexAsset.java:1116)
com.openmarket.assetframework.complexasset.AssetManager.read(AssetManager.java:374)
com.openmarket.assetframework.common.BulkTransactionalAssetManager.read(BulkTransactionalAssetManager.java:533)
com.openmarket.assetframework.complexasset.ComplexAsset.Read(ComplexAsset.java:1099)
com.openmarket.xcelerate.asset.Asset.Load(Asset.java:2468)
com.fatwire.assetapi.data.AssetDataImpl.getAssociatedAssets(AssetDataImpl.java:418)
jsp_servlet._jsp._cs_deployed._<some_custom_location>._common.__get_data._jspService(__get_data.java:173)

Sites sites.log file showed the following relevant information:

Caused by: java.lang.OutOfMemoryError: Java heap space
at java.awt.image.DataBufferByte.<init>(DataBufferByte.java:58)
at java.awt.image.ComponentSampleModel.createDataBuffer(ComponentSampleModel.java:397)
at java.awt.image.Raster.createWritableRaster(Raster.java:938)
at javax.imageio.ImageTypeSpecifier.createBufferedImage(ImageTypeSpecifier.java:1056)
at javax.imageio.ImageReader.getDestination(ImageReader.java:2879)
at com.sun.imageio.plugins.png.PNGImageReader.readImage(PNGImageReader.java:1263)
at com.sun.imageio.plugins.png.PNGImageReader.read(PNGImageReader.java:1560)
at javax.imageio.ImageIO.read(ImageIO.java:1422)
at javax.imageio.ImageIO.read(ImageIO.java:1326)
at com.fatwire.mobility.util.MobilityUtils._createDeviceImageBean(MobilityUtils.java:431)
at com.fatwire.mobility.util.MobilityUtils.getDeviceImages(MobilityUtils.java:373)
at jsp_servlet._jsp._cs_deployed._ui._data._carousel.__skincarouselaction._jspService(__skincarouselaction.java:152)

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.