Deadlock on server due to weblogic replication and ADF application (Doc ID 2758743.1)

Last updated on JANUARY 08, 2024

Applies to:

Symptoms

Encountering a deadlock in ADF application:

Found one Java-level deadlock:
=============================
"[STUCK] ExecuteThread: '39' for queue: 'weblogic.kernel.Default (self-tuning)'":
 waiting to lock monitor 0x00007f3245a82698 (object 0x0000000625ee54a0, a weblogic.cluster.replication.WrappedRO),
 which is held by "[STUCK] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'"
"[STUCK] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'":
 waiting for ownable synchronizer 0x00000006262d1c78, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
 which is held by "[STUCK] ExecuteThread: '25' for queue: 'weblogic.kernel.Default (self-tuning)'"
"[STUCK] ExecuteThread: '25' for queue: 'weblogic.kernel.Default (self-tuning)'":
 waiting to lock monitor 0x00007f3274ff2478 (object 0x0000000626266720, a weblogic.servlet.internal.AttributeWrapper),
 which is held by "[STUCK] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'"

Java stack information for the threads listed above:
===================================================
"[STUCK] ExecuteThread: '39' for queue: 'weblogic.kernel.Default (self-tuning)'":
at weblogic.cluster.replication.WrappedRO.ensureStatus(WrappedRO.java:144)
- waiting to lock <0x0000000625ee54a0> (a weblogic.cluster.replication.WrappedRO)
at weblogic.cluster.replication.ReplicationManager.getPrimary(ReplicationManager.java:887)
at weblogic.cluster.replication.ReplicationManager.lookup(ReplicationManager.java:432)
at weblogic.servlet.internal.session.ReplicatedSessionContext.lookupSession(ReplicatedSessionContext.java:428)
at weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:270)
at weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:262)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getValidSession(ServletRequestImpl.java:3375)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper._getSessionInternal(ServletRequestImpl.java:2824)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getSessionInternal(ServletRequestImpl.java:2783)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getSession(ServletRequestImpl.java:2773)
at weblogic.servlet.internal.ServletRequestImpl.getSession(ServletRequestImpl.java:1459)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2207)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2196)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2174)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1632)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:256)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)
"[STUCK] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006262d1c78> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290)
at oracle.adf.model.dcframe.DataControlFrameImpl.lock(DataControlFrameImpl.java:510)
at oracle.adf.model.BindingContext.setCurrentFrame(BindingContext.java:2768)
at oracle.adf.model.BindingContext.setCurrentDataControlFrame(BindingContext.java:2616)
at oracle.adf.model.dcframe.DataControlFrameImpl.createSnapshot(DataControlFrameImpl.java:894)
at oracle.adf.model.BindingContext.createSnapshot(BindingContext.java:875)
at oracle.adf.model.BindingContext.writeObject(BindingContext.java:828)
at sun.reflect.GeneratedMethodAccessor9554.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1156)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at weblogic.common.internal.PassivationUtils.toByteArray(PassivationUtils.java:40)
at weblogic.common.internal.PassivationUtils.toByteArray(PassivationUtils.java:31)
at weblogic.common.internal.PassivationUtils.copy(PassivationUtils.java:73)
at weblogic.servlet.internal.AttributeWrapper.getObject(AttributeWrapper.java:111)
- locked <0x0000000626266720> (a weblogic.servlet.internal.AttributeWrapper)
at weblogic.servlet.internal.AttributeWrapper.getObject(AttributeWrapper.java:71)
at weblogic.servlet.internal.session.SessionData.notifyActivated(SessionData.java:1202)
at weblogic.servlet.internal.session.SessionData.notifyActivated(SessionData.java:1191)
at weblogic.servlet.internal.session.ReplicatedSessionData.becomePrimary(ReplicatedSessionData.java:225)
at weblogic.cluster.replication.WrappedRO.changeStatus(WrappedRO.java:163)
at weblogic.cluster.replication.WrappedRO.ensureStatus(WrappedRO.java:150)
- locked <0x0000000625ee54a0> (a weblogic.cluster.replication.WrappedRO)
at weblogic.cluster.replication.ReplicationManager$wroManager.create(ReplicationManager.java:333)
at weblogic.cluster.replication.ReplicationManager.registerLocally(ReplicationManager.java:486)
- locked <0x0000000626228328> (a weblogic.cluster.replication.ROID)
at weblogic.servlet.internal.session.ReplicatedSessionContext.lookupRO(ReplicatedSessionContext.java:200)
at weblogic.servlet.internal.session.ReplicatedSessionContext.lookupSession(ReplicatedSessionContext.java:414)
at weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:270)
at weblogic.servlet.internal.session.ReplicatedSessionContext.getSessionInternal(ReplicatedSessionContext.java:262)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getValidSession(ServletRequestImpl.java:3375)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper._getSessionInternal(ServletRequestImpl.java:2824)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getSessionInternal(ServletRequestImpl.java:2783)
at weblogic.servlet.internal.ServletRequestImpl$SessionHelper.getSession(ServletRequestImpl.java:2773)
at weblogic.servlet.internal.ServletRequestImpl.getSession(ServletRequestImpl.java:1459)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2207)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2196)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2174)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1632)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:256)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)
"[STUCK] ExecuteThread: '25' for queue: 'weblogic.kernel.Default (self-tuning)'":
at weblogic.servlet.internal.AttributeWrapper.getObject(AttributeWrapper.java:89)
- waiting to lock <0x0000000626266720> (a weblogic.servlet.internal.AttributeWrapper)
at weblogic.servlet.internal.AttributeWrapper.getObject(AttributeWrapper.java:71)
at weblogic.servlet.internal.session.AttributeWrapperUtils.unwrapObject(AttributeWrapperUtils.java:116)
at weblogic.servlet.internal.session.SessionData.getAttributeInternal(SessionData.java:455)
at weblogic.servlet.internal.session.ReplicatedSessionData.getAttribute(ReplicatedSessionData.java:704)
at com.sun.faces.context.SessionMap.containsKey(Unknown Source)
at oracle.adfinternal.controller.application.ManagedBeanELResolver.getValue(ManagedBeanELResolver.java:103)
at com.sun.faces.el.DemuxCompositeELResolver._getValue(Unknown Source)
at com.sun.faces.el.DemuxCompositeELResolver.getValue(Unknown Source)
at oracle.adf.share.el.VariableResolverELContext$1.getValue(VariableResolverELContext.java:51)
at com.sun.el.parser.AstIdentifier.getValue(AstIdentifier.java:103)
at com.sun.el.parser.AstValue.getValue(AstValue.java:179)
at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:224)
at oracle.adf.share.el.OracleExpressionEvaluatorImpl.evaluate(OracleExpressionEvaluatorImpl.java:148)
at oracle.adf.share.el.OracleExpressionEvaluatorImpl.evaluate(OracleExpressionEvaluatorImpl.java:103)
at oracle.adf.share.el.OracleExpressionEvaluatorImpl.evaluateHandleNull(OracleExpressionEvaluatorImpl.java:84)
at oracle.adf.model.binding.DCUtil.elEvaluate(DCUtil.java:855)
at oracle.adf.model.binding.DCInvokeMethodDef.resolveAsExpression(DCInvokeMethodDef.java:225)
at oracle.adf.model.binding.DCUtil.putValueInPath(DCUtil.java:549)
at oracle.bpel.services.datacontrol.XSDDataControl.invokeWorkflowOperation(XSDDataControl.java:382)
at oracle.bpel.services.datacontrol.XSDDataControl.invokeOperation(XSDDataControl.java:468)
at oracle.adf.model.bean.DCBeanDataControl.invokeMethod(DCBeanDataControl.java:481)
at oracle.adf.model.binding.DCInvokeMethod.callMethod(DCInvokeMethod.java:266)
at oracle.jbo.uicli.binding.JUCtrlActionBinding.doIt(JUCtrlActionBinding.java:1691)
- locked <0x0000000626264d30> (a oracle.jbo.uicli.binding.JUFormBinding)
at oracle.adf.model.binding.DCDataControl.invokeOperation(DCDataControl.java:2218)
at oracle.adf.model.bean.DCBeanDataControl.invokeOperation(DCBeanDataControl.java:564)
at oracle.adf.model.adapter.AdapterDCService.invokeOperation(AdapterDCService.java:316)
- locked <0x00000006261a7630> (a oracle.adf.model.adapter.AdapterDCService)
at oracle.jbo.uicli.binding.JUCtrlActionBinding.invoke(JUCtrlActionBinding.java:784)
at oracle.jbo.uicli.binding.JUMethodIteratorDef$JUMethodIteratorBinding.invokeMethodAction(JUMethodIteratorDef.java:170)
at oracle.jbo.uicli.binding.JUMethodIteratorDef$JUMethodIteratorBinding.executeQuery(JUMethodIteratorDef.java:397)
at oracle.jbo.uicli.binding.JUMethodIteratorDef$JUMethodIteratorBinding.executeQueryIfNeeded(JUMethodIteratorDef.java:338)
- locked <0x00000006261a7630> (a oracle.adf.model.adapter.AdapterDCService)
at oracle.adf.model.binding.DCBindingContainer.internalRefreshControl(DCBindingContainer.java:3286)
at oracle.adf.model.binding.DCBindingContainer.refreshControl(DCBindingContainer.java:2947)
at oracle.adf.model.binding.DCBindingContainer.refresh(DCBindingContainer.java:2933)
at oracle.adfinternal.controller.util.model.DataBindingActivityImpl.beforeExecute(DataBindingActivityImpl.java:142)
at oracle.adfinternal.controller.engine.ControlFlowEngine.executeActivity(ControlFlowEngine.java:1223)
at oracle.adfinternal.controller.engine.ControlFlowEngine.doRouting(ControlFlowEngine.java:1087)
at oracle.adfinternal.controller.engine.ControlFlowEngine.doRouting(ControlFlowEngine.java:979)
at oracle.adfinternal.controller.engine.ControlFlowEngine.invokeTaskFlow(ControlFlowEngine.java:253)
at oracle.adfinternal.controller.application.RemoteTaskFlowCallRequestHandler.invokeTaskFlowByUrl(RemoteTaskFlowCallRequestHandler.java:103)
at oracle.adfinternal.controller.application.RemoteTaskFlowCallRequestHandler.doCreateView(RemoteTaskFlowCallRequestHandler.java:64)
at oracle.adfinternal.controller.application.BaseRequestHandlerImpl.createView(BaseRequestHandlerImpl.java:59)
at org.apache.myfaces.trinidadinternal.application.InternalViewHandlingStrategy.createView(InternalViewHandlingStrategy.java:48)
at com.sun.faces.application.view.MultiViewHandler.createView(Unknown Source)
at javax.faces.application.ViewHandlerWrapper.createView(Unknown Source)
at org.apache.myfaces.trinidadinternal.application.ViewHandlerImpl.createView(ViewHandlerImpl.java:108)
at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._restoreView(LifecycleImpl.java:847)
at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._executePhase(LifecycleImpl.java:397)
at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:225)
at javax.faces.webapp.FacesServlet.service(Unknown Source)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:280)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:254)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:136)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:346)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:25)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.adf.model.servlet.ADFBindingFilter.doFilter(ADFBindingFilter.java:192)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.adfinternal.view.faces.webapp.rich.RegistrationFilter.doFilter(RegistrationFilter.java:105)
at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:502)
at oracle.adfinternal.view.faces.activedata.AdsFilter.doFilter(AdsFilter.java:60)
at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:502)
at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl._doFilterImpl(TrinidadFilterImpl.java:327)
at org.apache.myfaces.trinidadinternal.webapp.TrinidadFilterImpl.doFilter(TrinidadFilterImpl.java:229)
at org.apache.myfaces.trinidad.webapp.TrinidadFilter.doFilter(TrinidadFilter.java:92)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.adf.library.webapp.LibraryFilter.doFilter(LibraryFilter.java:202)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at nagiworld.net.filters.gzip.GZIPFilter.doFilter(Unknown Source)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.adfinternal.view.faces.caching.filter.AdfFacesCachingFilterImpl.doFilter(AdfFacesCachingFilterImpl.java:136)
at oracle.adf.view.rich.webapp.AdfFacesCachingFilter.doFilter(AdfFacesCachingFilter.java:66)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.bpel.services.workflow.client.worklist.util.WorkflowFilter.doFilter(WorkflowFilter.java:176)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.bpel.services.workflow.client.worklist.util.DisableUrlSessionFilter.doFilter(DisableUrlSessionFilter.java:70)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:137)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:315)
at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:460)
at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:120)
at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:217)
at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:81)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:220)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:79)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3456)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3422)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:323)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2280)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2196)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2174)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1632)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:256)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

Found 1 deadlock.

Changes

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.