Deadlock raised in WLS code while while rolling back XA transaction (Doc ID 1516929.1)

Last updated on NOVEMBER 05, 2016

Applies to:

Oracle Weblogic Server - Version 10.3.4 to 12.1.1.0
Information in this document applies to any platform.

Symptoms

The below deadlock occured and server went into FAILED State.

DEADLOCK DETECTED:

==================

[deadlocked thread] [ACTIVE] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)':
---------------------------------------------------------------------------------------------------
Thread '[ACTIVE] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'' is waiting to acquire lock 'weblogic.jdbc.wrapper.TxInfo@1c744860' that is held by thread '[ACTIVE] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)''

Stack trace:
------------
weblogic.jdbc.wrapper.TxInfo.isEnded(TxInfo.java:44)
weblogic.jdbc.wrapper.XAConnection.removeTxAssocIfNeeded(XAConnection.java:344)
weblogic.jdbc.wrapper.XAConnection.removeStatement(XAConnection.java:327)
weblogic.jdbc.wrapper.JTAConnection.removeStatement(JTAConnection.java:161)
weblogic.jdbc.wrapper.Statement.internalClose(Statement.java:365)
weblogic.jdbc.wrapper.Statement.doClose(Statement.java:417)
weblogic.jdbc.wrapper.Statement.close(Statement.java:398)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.closeStatement(DatabaseAccessor.java:382)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:678)
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:526)
org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:529)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:206)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:192)
org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:664)
org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2582)
org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2553)
org.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:439)
org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1076)
org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:740)
org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1036)
org.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:407)
org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1122)
org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2908)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1291)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1273)
org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1233)
org.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(EntityManagerImpl.java:778)
org.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(EntityManagerImpl.java:722)
org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:616)
org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:495)
hk.org.ha.lis.persistence.impl.AdaRequestDaoJpaImpl.selectAdaRequest(AdaRequestDaoJpaImpl.java:58)
hk.org.ha.lis.biz.service.AdaRequestService.selectAdaRequest(AdaRequestService.java:209)
hk.org.ha.lis.biz.service.AdaRequestService.selectAdaRequest(AdaRequestService.java:186)
hk.org.ha.lis.biz.service.AdaRequestService.addAdaRequest(AdaRequestService.java:229)
hk.org.ha.lis.biz.ada.impl.AnalyserOrderAppServiceImpl.createAdaOrder(AnalyserOrderAppServiceImpl.java:250)
hk.org.ha.lis.biz.registration.impl.PostRegistrationAppServiceImpl.postRegistrationProcess(PostRegistrationAppServiceImpl.java:207)
hk.org.ha.lis.biz.registration.impl.PostRegistrationAppServiceImpl.executeProcessTask(PostRegistrationAppServiceImpl.java:107)
hk.org.ha.lis.biz.AbstractWorkerAppService.processTask(AbstractWorkerAppService.java:137)
hk.org.ha.lis.biz.AbstractWorkerAppService.executeCoreProcess(AbstractWorkerAppService.java:926)
hk.org.ha.lis.ejb.WorkerAppServiceBean.executeCoreProcess(WorkerAppServiceBean.java:372)
sun.reflect.GeneratedMethodAccessor355.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)
weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:54)
com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:50)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
$Proxy161.executeCoreProcess(Unknown Source)
hk.org.ha.lis.ejb.WorkerAppServiceEjb_y79pvx_WorkerAppServiceLocalImpl.__WL_invoke(Unknown Source)
weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:39)
hk.org.ha.lis.ejb.WorkerAppServiceEjb_y79pvx_WorkerAppServiceLocalImpl.executeCoreProcess(Unknown Source)
hk.org.ha.lis.biz.AbstractWorkerAppService.coreProcess(AbstractWorkerAppService.java:715)
hk.org.ha.lis.biz.AbstractWorkerAppService.process(AbstractWorkerAppService.java:815)
hk.org.ha.lis.biz.AbstractBatchWorkerAppService.executeTaskProcess(AbstractBatchWorkerAppService.java:769)
hk.org.ha.lis.biz.AbstractBatchWorkerAppService.taskProcess(AbstractBatchWorkerAppService.java:263)
hk.org.ha.lis.biz.AbstractBatchWorkerAppService.batchWorkerProcess(AbstractBatchWorkerAppService.java:114)
hk.org.ha.lis.biz.AbstractBatchWorkerAppService.process(AbstractBatchWorkerAppService.java:587)
hk.org.ha.lis.ejb.EntryWorkerAppServiceBean.process(EntryWorkerAppServiceBean.java:58)
sun.reflect.GeneratedMethodAccessor372.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)
weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:54)
com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:50)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
$Proxy160.process(Unknown Source)
hk.org.ha.lis.ejb.EntryWorkerAppServiceEjb_malgqt_EntryWorkerAppServiceLocalImpl.__WL_invoke(Unknown Source)
weblogic.ejb.container.internal.SessionLocalMethodInvoker.invoke(SessionLocalMethodInvoker.java:39)
hk.org.ha.lis.ejb.EntryWorkerAppServiceEjb_malgqt_EntryWorkerAppServiceLocalImpl.process(Unknown Source)
hk.org.ha.lis.views.struts.common.RawDispatcher.execute(Unknown Source)
hk.org.ha.lis.views.struts.common.Dispatcher.execute(Unknown Source)
sun.reflect.GeneratedMethodAccessor24501.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:441)
com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:280)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:243)
com.opensymphony.xwork2.interceptor.DefaultWorkflowInterceptor.doIntercept(DefaultWorkflowInterceptor.java:165)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.validator.ValidationInterceptor.doIntercept(ValidationInterceptor.java:252)
org.apache.struts2.interceptor.validation.AnnotationValidationInterceptor.doIntercept(AnnotationValidationInterceptor.java:68)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ConversionErrorInterceptor.intercept(ConversionErrorInterceptor.java:122)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:195)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:179)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
org.apache.struts2.interceptor.MultiselectInterceptor.intercept(MultiselectInterceptor.java:75)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
org.apache.struts2.interceptor.CheckboxInterceptor.intercept(CheckboxInterceptor.java:94)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
org.apache.struts2.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:235)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ModelDrivenInterceptor.intercept(ModelDrivenInterceptor.java:89)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ScopedModelDrivenInterceptor.intercept(ScopedModelDrivenInterceptor.java:130)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
org.apache.struts2.interceptor.debugging.DebuggingInterceptor.intercept(DebuggingInterceptor.java:267)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:126)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:138)
com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:87)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:165)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:164)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.AliasInterceptor.intercept(AliasInterceptor.java:179)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor.intercept(ExceptionMappingInterceptor.java:176)
com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:237)
org.apache.struts2.impl.StrutsActionProxy.execute(StrutsActionProxy.java:52)
org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:488)
org.apache.struts2.dispatcher.ng.ExecuteOperations.executeAction(ExecuteOperations.java:77)
org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:91)
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3715)
weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)
weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2277)
weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2183)
weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1454)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
weblogic.work.ExecuteThread.run(ExecuteThread.java:176)

[deadlocked thread] [ACTIVE] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)':
---------------------------------------------------------------------------------------------------
Thread '[ACTIVE] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'' is waiting to acquire lock 'java.lang.Object@152d827b' that is held by thread '[ACTIVE] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)''

Stack trace:
------------
weblogic.jdbc.wrapper.XAConnection.closeAllStatements(XAConnection.java:357)
weblogic.jdbc.wrapper.XAConnection.cleanup(XAConnection.java:378)
weblogic.jdbc.wrapper.XAConnection.close(XAConnection.java:554)
weblogic.jdbc.common.internal.XAConnectionEnvFactory.refreshResource(XAConnectionEnvFactory.java:257)
weblogic.jdbc.common.internal.XAConnectionEnvFactory.refreshResource(XAConnectionEnvFactory.java:231)
weblogic.common.resourcepool.ResourcePoolImpl.refreshResource(ResourcePoolImpl.java:1781)
weblogic.common.resourcepool.ResourcePoolImpl.checkResource(ResourcePoolImpl.java:1682)
weblogic.common.resourcepool.ResourcePoolImpl.checkAndReturnResource(ResourcePoolImpl.java:1588)
weblogic.common.resourcepool.ResourcePoolImpl.checkAndReturnResource(ResourcePoolImpl.java:1577)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:395)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:342)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:329)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:417)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:324)
weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:94)
weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:63)
weblogic.jdbc.jta.DataSource.getXAConnectionFromPool(DataSource.java:1677)
weblogic.jdbc.jta.DataSource.rollback(DataSource.java:1132)
weblogic.transaction.internal.XAServerResourceInfo.rollback(XAServerResourceInfo.java:1493)
weblogic.transaction.internal.XAServerResourceInfo.rollback(XAServerResourceInfo.java:788)
weblogic.transaction.internal.ServerSCInfo$3.run(ServerSCInfo.java:863)
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
weblogic.work.ExecuteThread.run(ExecuteThread.java:176)

 

####<2012-Nov-20 06:27:10.218 HKT> <Critical> <Health> <cdceap7n> <LisPwhPrd_cdceap7n_32118> <[ACTIVE] ExecuteThread: '12' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1353364030218> <BEA-310006> <Critical Subsystem core has failed. Setting server state to FAILED.
Reason: Thread deadlock detected>

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms