SOA composites are not loading during server start due to 'stuck' threads
(Doc ID 2179175.1)
Last updated on MAY 15, 2023
Applies to:
Oracle SOA Suite - Version 11.1.1.7.0 and laterInformation in this document applies to any platform.
Symptoms
SOA Server comes to the running state but none of the composites are deployed and the EM Console shows all composites are down. Thread dumps show the below deployment thread is getting stuck:
"Workmanager: , Version: 0, Scheduled=false, Started=false, Wait time: 0 ms
" id=240 idx=0x3a0 tid=23919 prio=10 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/ReentrantLock$NonfairSync@0xb90cfab0
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:156)
at java/util/concurrent/locks/AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
at java/util/concurrent/locks/ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java/util/concurrent/locks/ReentrantLock.lock(ReentrantLock.java:262)
at oracle/wsm/policymanager/accessor/BeanAccessor.persistUsage(BeanAccessor.java:1225)
at oracle/wsm/policymanager/accessor/BeanAccessor.updateUsage(BeanAccessor.java:2487)
at oracle/wsm/policymanager/accessor/BeanAccessor.removeUsage(BeanAccessor.java:1950)
at oracle/wsm/config/ConfigurationManager$ContextualAccessor.removeUsage(ConfigurationManager.java:901)
at oracle/wsm/config/ConfigurationManager$VirtualAccessor.removeUsage(ConfigurationManager.java:1177)
at oracle/wsm/policyaccess/PolicyAccessPoint.undeployApplication(PolicyAccessPoint.java:621)
at oracle/wsm/policyaccess/impl/PolicyAccessorImpl$4.run(PolicyAccessorImpl.java:347)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:232)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:240)
at oracle/wsm/policyaccess/impl/PolicyAccessorImpl.processEvent(PolicyAccessorImpl.java:344)
at oracle/integration/platform/common/mgmt/PolicyCacheImpl.deregisterPolicyBinding(PolicyCacheImpl.java:691)
at oracle/j2ee/ws/server/provider/ProviderConfigImpl.addService(ProviderConfigImpl.java:442)
at oracle/j2ee/ws/server/provider/ProviderConfigImpl.addService(ProviderConfigImpl.java:288)
at oracle/integration/platform/blocks/soap/FabricProviderConfig.addService(FabricProviderConfig.java:125)
at oracle/integration/platform/blocks/soap/FabricProviderConfig.addService(FabricProviderConfig.java:214)
at oracle/integration/platform/blocks/soap/WebServiceEntryBindingComponent.load(WebServiceEntryBindingComponent.java:228)
at oracle/integration/platform/blocks/soap/WebServiceEntryBindingComponent.load(WebServiceEntryBindingComponent.java:152)
at oracle/integration/platform/blocks/deploy/CompositeDeploymentConnection.deployServices(CompositeDeploymentConnection.java:161)
at oracle/integration/platform/blocks/deploy/CompositeDeploymentConnection.deploy(CompositeDeploymentConnection.java:92)
at oracle/integration/platform/blocks/deploy/CompositeDeployerImpl.deploy(CompositeDeployerImpl.java:150)
^-- Holding lock: oracle/integration/platform/blocks/deploy/DeploymentResourceManager@0xc0361a78[biased lock]
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
at jrockit/vm/Reflect.invokeMethod(Ljava/lang/Object;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
at sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
at sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java/lang/reflect/Method.invoke(Method.java:597)
at org/springframework/aop/support/AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org/springframework/aop/framework/ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at oracle/integration/platform/blocks/deploy/DeploymentEventPublisher.invoke(DeploymentEventPublisher.java:86)
at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org/springframework/transaction/interceptor/TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org/springframework/aop/framework/JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com/sun/proxy/$Proxy357.deploy(Loracle/fabric/composite/model/CompositeModel;)V(Unknown Source)
at oracle/integration/platform/kernel/FabricKernelInitializerServlet$1.run(FabricKernelInitializerServlet.java:819)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:120)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:184)
at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'" id=166 idx=0x288 tid=22061 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/rjvm/ResponseImpl@0xc09ca038[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/rjvm/ResponseImpl.waitForData(ResponseImpl.java:90)
^-- Lock released while waiting: weblogic/rjvm/ResponseImpl@0xc09ca038[fat lock]
at weblogic/rjvm/ResponseImpl.getTxContext(ResponseImpl.java:130)
at weblogic/rjvm/BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:110)
at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:345)
at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259)
at oracle/wsm/policymanager/bean/ejb/impl/UsageTracker_oi3aq7_IRemoteUsageTrackerImpl_1036_WLStub.recordUsage(Ljava/lang/String;Ljava/util/Map;Ljava/util/Set;)V(Unknown Source)
at oracle/wsm/policymanager/client/UsageTrackerDelegate$4.run(UsageTrackerDelegate.java:168)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)
at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:324)
at oracle/security/jps/internal/jaas/AccActionExecutor.execute(AccActionExecutor.java:74)
at oracle/security/jps/internal/jaas/AbstractSubjectSecurity$ActionExecutorWrapper.execute(AbstractSubjectSecurity.java:242)
at oracle/security/jps/internal/jaas/CascadeActionExecutor$SubjectPrivilegedExceptionAction.run(CascadeActionExecutor.java:83)
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:146)
at weblogic/security/Security.runAs(Security.java:61)
at oracle/security/jps/wls/jaas/WlsActionExecutor.execute(WlsActionExecutor.java:51)
at oracle/security/jps/internal/jaas/CascadeActionExecutor.execute(CascadeActionExecutor.java:56)
at oracle/security/jps/internal/jaas/AbstractSubjectSecurity$ActionExecutorWrapper.execute(AbstractSubjectSecurity.java:242)
at oracle/wsm/policymanager/client/UsageTrackerDelegate.recordUsage(UsageTrackerDelegate.java:165)
at oracle/wsm/policymanager/accessor/BeanAccessor.persistUsage(BeanAccessor.java:1247)
at oracle/wsm/policymanager/accessor/BeanAccessor.access$1100(BeanAccessor.java:148)
at oracle/wsm/policymanager/accessor/BeanAccessor$UsageUpdaterTask.run(BeanAccessor.java:412)
at oracle/wsm/common/scheduler/TimerManagerWrapper$TimerListenerImpl.timerExpired(TimerManagerWrapper.java:62)
at weblogic/timers/internal/commonj/ListenerWrap.timerExpired(ListenerWrap.java:37)
at weblogic/timers/internal/TimerImpl.run(TimerImpl.java:273)
at weblogic/work/SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:256)
at weblogic/work/ExecuteThread.run(ExecuteThread.java:221)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
" id=240 idx=0x3a0 tid=23919 prio=10 alive, parked, native_blocked, daemon
-- Parking to wait for: java/util/concurrent/locks/ReentrantLock$NonfairSync@0xb90cfab0
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2230)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:156)
at java/util/concurrent/locks/AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
at java/util/concurrent/locks/ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java/util/concurrent/locks/ReentrantLock.lock(ReentrantLock.java:262)
at oracle/wsm/policymanager/accessor/BeanAccessor.persistUsage(BeanAccessor.java:1225)
at oracle/wsm/policymanager/accessor/BeanAccessor.updateUsage(BeanAccessor.java:2487)
at oracle/wsm/policymanager/accessor/BeanAccessor.removeUsage(BeanAccessor.java:1950)
at oracle/wsm/config/ConfigurationManager$ContextualAccessor.removeUsage(ConfigurationManager.java:901)
at oracle/wsm/config/ConfigurationManager$VirtualAccessor.removeUsage(ConfigurationManager.java:1177)
at oracle/wsm/policyaccess/PolicyAccessPoint.undeployApplication(PolicyAccessPoint.java:621)
at oracle/wsm/policyaccess/impl/PolicyAccessorImpl$4.run(PolicyAccessorImpl.java:347)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:232)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:240)
at oracle/wsm/policyaccess/impl/PolicyAccessorImpl.processEvent(PolicyAccessorImpl.java:344)
at oracle/integration/platform/common/mgmt/PolicyCacheImpl.deregisterPolicyBinding(PolicyCacheImpl.java:691)
at oracle/j2ee/ws/server/provider/ProviderConfigImpl.addService(ProviderConfigImpl.java:442)
at oracle/j2ee/ws/server/provider/ProviderConfigImpl.addService(ProviderConfigImpl.java:288)
at oracle/integration/platform/blocks/soap/FabricProviderConfig.addService(FabricProviderConfig.java:125)
at oracle/integration/platform/blocks/soap/FabricProviderConfig.addService(FabricProviderConfig.java:214)
at oracle/integration/platform/blocks/soap/WebServiceEntryBindingComponent.load(WebServiceEntryBindingComponent.java:228)
at oracle/integration/platform/blocks/soap/WebServiceEntryBindingComponent.load(WebServiceEntryBindingComponent.java:152)
at oracle/integration/platform/blocks/deploy/CompositeDeploymentConnection.deployServices(CompositeDeploymentConnection.java:161)
at oracle/integration/platform/blocks/deploy/CompositeDeploymentConnection.deploy(CompositeDeploymentConnection.java:92)
at oracle/integration/platform/blocks/deploy/CompositeDeployerImpl.deploy(CompositeDeployerImpl.java:150)
^-- Holding lock: oracle/integration/platform/blocks/deploy/DeploymentResourceManager@0xc0361a78[biased lock]
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
at jrockit/vm/Reflect.invokeMethod(Ljava/lang/Object;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
at sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
at sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java/lang/reflect/Method.invoke(Method.java:597)
at org/springframework/aop/support/AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org/springframework/aop/framework/ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at oracle/integration/platform/blocks/deploy/DeploymentEventPublisher.invoke(DeploymentEventPublisher.java:86)
at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org/springframework/transaction/interceptor/TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org/springframework/aop/framework/JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com/sun/proxy/$Proxy357.deploy(Loracle/fabric/composite/model/CompositeModel;)V(Unknown Source)
at oracle/integration/platform/kernel/FabricKernelInitializerServlet$1.run(FabricKernelInitializerServlet.java:819)
at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:120)
at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:184)
at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
"[ACTIVE] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'" id=166 idx=0x288 tid=22061 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: weblogic/rjvm/ResponseImpl@0xc09ca038[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at weblogic/rjvm/ResponseImpl.waitForData(ResponseImpl.java:90)
^-- Lock released while waiting: weblogic/rjvm/ResponseImpl@0xc09ca038[fat lock]
at weblogic/rjvm/ResponseImpl.getTxContext(ResponseImpl.java:130)
at weblogic/rjvm/BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:110)
at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:345)
at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259)
at oracle/wsm/policymanager/bean/ejb/impl/UsageTracker_oi3aq7_IRemoteUsageTrackerImpl_1036_WLStub.recordUsage(Ljava/lang/String;Ljava/util/Map;Ljava/util/Set;)V(Unknown Source)
at oracle/wsm/policymanager/client/UsageTrackerDelegate$4.run(UsageTrackerDelegate.java:168)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)
at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:324)
at oracle/security/jps/internal/jaas/AccActionExecutor.execute(AccActionExecutor.java:74)
at oracle/security/jps/internal/jaas/AbstractSubjectSecurity$ActionExecutorWrapper.execute(AbstractSubjectSecurity.java:242)
at oracle/security/jps/internal/jaas/CascadeActionExecutor$SubjectPrivilegedExceptionAction.run(CascadeActionExecutor.java:83)
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:146)
at weblogic/security/Security.runAs(Security.java:61)
at oracle/security/jps/wls/jaas/WlsActionExecutor.execute(WlsActionExecutor.java:51)
at oracle/security/jps/internal/jaas/CascadeActionExecutor.execute(CascadeActionExecutor.java:56)
at oracle/security/jps/internal/jaas/AbstractSubjectSecurity$ActionExecutorWrapper.execute(AbstractSubjectSecurity.java:242)
at oracle/wsm/policymanager/client/UsageTrackerDelegate.recordUsage(UsageTrackerDelegate.java:165)
at oracle/wsm/policymanager/accessor/BeanAccessor.persistUsage(BeanAccessor.java:1247)
at oracle/wsm/policymanager/accessor/BeanAccessor.access$1100(BeanAccessor.java:148)
at oracle/wsm/policymanager/accessor/BeanAccessor$UsageUpdaterTask.run(BeanAccessor.java:412)
at oracle/wsm/common/scheduler/TimerManagerWrapper$TimerListenerImpl.timerExpired(TimerManagerWrapper.java:62)
at weblogic/timers/internal/commonj/ListenerWrap.timerExpired(ListenerWrap.java:37)
at weblogic/timers/internal/TimerImpl.run(TimerImpl.java:273)
at weblogic/work/SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:256)
at weblogic/work/ExecuteThread.run(ExecuteThread.java:221)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
-- end of trace
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |