SOA composites are not loading during server start due to 'stuck' threads

(Doc ID 2179175.1)

Last updated on SEPTEMBER 06, 2016

Applies to:

Oracle SOA Suite - Version 11.1.1.7.0 and later
Information in this document applies to any platform.

Symptoms


SOA Server comes to the running state but none of the composites are deployed and the EM Console shows all composites are down. Thread dumps show the below deployment thread is getting stuck:


"Workmanager: , Version: 0, Scheduled=false, Started=false, Wait time: 0 ms
" id=240 idx=0x3a0 tid=23919 prio=10 alive, parked, native_blocked, daemon
  -- Parking to wait for: java/util/concurrent/locks/ReentrantLock$NonfairSync@0xb90cfab0
  at jrockit/vm/Locks.park0(J)V(Native Method)
  at jrockit/vm/Locks.park(Locks.java:2230)
  at sun/misc/Unsafe.park(ZJ)V(Native Method)
  at java/util/concurrent/locks/LockSupport.park(LockSupport.java:156)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
  at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
  at java/util/concurrent/locks/ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
  at java/util/concurrent/locks/ReentrantLock.lock(ReentrantLock.java:262)
  at oracle/wsm/policymanager/accessor/BeanAccessor.persistUsage(BeanAccessor.java:1225)
  at oracle/wsm/policymanager/accessor/BeanAccessor.updateUsage(BeanAccessor.java:2487)
  at oracle/wsm/policymanager/accessor/BeanAccessor.removeUsage(BeanAccessor.java:1950)
  at oracle/wsm/config/ConfigurationManager$ContextualAccessor.removeUsage(ConfigurationManager.java:901)
  at oracle/wsm/config/ConfigurationManager$VirtualAccessor.removeUsage(ConfigurationManager.java:1177)
  at oracle/wsm/policyaccess/PolicyAccessPoint.undeployApplication(PolicyAccessPoint.java:621)
  at oracle/wsm/policyaccess/impl/PolicyAccessorImpl$4.run(PolicyAccessorImpl.java:347)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:232)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:240)
  at oracle/wsm/policyaccess/impl/PolicyAccessorImpl.processEvent(PolicyAccessorImpl.java:344)
  at oracle/integration/platform/common/mgmt/PolicyCacheImpl.deregisterPolicyBinding(PolicyCacheImpl.java:691)
  at oracle/j2ee/ws/server/provider/ProviderConfigImpl.addService(ProviderConfigImpl.java:442)
  at oracle/j2ee/ws/server/provider/ProviderConfigImpl.addService(ProviderConfigImpl.java:288)
  at oracle/integration/platform/blocks/soap/FabricProviderConfig.addService(FabricProviderConfig.java:125)
  at oracle/integration/platform/blocks/soap/FabricProviderConfig.addService(FabricProviderConfig.java:214)
  at oracle/integration/platform/blocks/soap/WebServiceEntryBindingComponent.load(WebServiceEntryBindingComponent.java:228)
  at oracle/integration/platform/blocks/soap/WebServiceEntryBindingComponent.load(WebServiceEntryBindingComponent.java:152)
  at oracle/integration/platform/blocks/deploy/CompositeDeploymentConnection.deployServices(CompositeDeploymentConnection.java:161)
  at oracle/integration/platform/blocks/deploy/CompositeDeploymentConnection.deploy(CompositeDeploymentConnection.java:92)
  at oracle/integration/platform/blocks/deploy/CompositeDeployerImpl.deploy(CompositeDeployerImpl.java:150)
  ^-- Holding lock: oracle/integration/platform/blocks/deploy/DeploymentResourceManager@0xc0361a78[biased lock]
  at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
  at jrockit/vm/Reflect.invokeMethod(Ljava/lang/Object;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
  at sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
  at sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java/lang/reflect/Method.invoke(Method.java:597)
  at org/springframework/aop/support/AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
  at org/springframework/aop/framework/ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
  at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
  at oracle/integration/platform/blocks/deploy/DeploymentEventPublisher.invoke(DeploymentEventPublisher.java:86)
  at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  at org/springframework/transaction/interceptor/TransactionInterceptor.invoke(TransactionInterceptor.java:106)
  at org/springframework/aop/framework/ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
  at org/springframework/aop/framework/JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  at com/sun/proxy/$Proxy357.deploy(Loracle/fabric/composite/model/CompositeModel;)V(Unknown Source)
  at oracle/integration/platform/kernel/FabricKernelInitializerServlet$1.run(FabricKernelInitializerServlet.java:819)
  at oracle/integration/platform/blocks/executor/WorkManagerExecutor$1.run(WorkManagerExecutor.java:120)
  at weblogic/work/j2ee/J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:184)
  at weblogic/work/DaemonWorkThread.run(DaemonWorkThread.java:30)
  at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
  -- end of trace

"[ACTIVE] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'" id=166 idx=0x288 tid=22061 prio=5 alive, waiting, native_blocked, daemon
  -- Waiting for notification on: weblogic/rjvm/ResponseImpl@0xc09ca038[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at weblogic/rjvm/ResponseImpl.waitForData(ResponseImpl.java:90)
  ^-- Lock released while waiting: weblogic/rjvm/ResponseImpl@0xc09ca038[fat lock]
  at weblogic/rjvm/ResponseImpl.getTxContext(ResponseImpl.java:130)
  at weblogic/rjvm/BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:110)
  at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:345)
  at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259)
  at oracle/wsm/policymanager/bean/ejb/impl/UsageTracker_oi3aq7_IRemoteUsageTrackerImpl_1036_WLStub.recordUsage(Ljava/lang/String;Ljava/util/Map;Ljava/util/Set;)V(Unknown Source)
  at oracle/wsm/policymanager/client/UsageTrackerDelegate$4.run(UsageTrackerDelegate.java:168)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)
  at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:324)
  at oracle/security/jps/internal/jaas/AccActionExecutor.execute(AccActionExecutor.java:74)
  at oracle/security/jps/internal/jaas/AbstractSubjectSecurity$ActionExecutorWrapper.execute(AbstractSubjectSecurity.java:242)
  at oracle/security/jps/internal/jaas/CascadeActionExecutor$SubjectPrivilegedExceptionAction.run(CascadeActionExecutor.java:83)
  at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
  at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:146)
  at weblogic/security/Security.runAs(Security.java:61)
  at oracle/security/jps/wls/jaas/WlsActionExecutor.execute(WlsActionExecutor.java:51)
  at oracle/security/jps/internal/jaas/CascadeActionExecutor.execute(CascadeActionExecutor.java:56)
  at oracle/security/jps/internal/jaas/AbstractSubjectSecurity$ActionExecutorWrapper.execute(AbstractSubjectSecurity.java:242)
  at oracle/wsm/policymanager/client/UsageTrackerDelegate.recordUsage(UsageTrackerDelegate.java:165)
  at oracle/wsm/policymanager/accessor/BeanAccessor.persistUsage(BeanAccessor.java:1247)
  at oracle/wsm/policymanager/accessor/BeanAccessor.access$1100(BeanAccessor.java:148)
  at oracle/wsm/policymanager/accessor/BeanAccessor$UsageUpdaterTask.run(BeanAccessor.java:412)
  at oracle/wsm/common/scheduler/TimerManagerWrapper$TimerListenerImpl.timerExpired(TimerManagerWrapper.java:62)
  at weblogic/timers/internal/commonj/ListenerWrap.timerExpired(ListenerWrap.java:37)
  at weblogic/timers/internal/TimerImpl.run(TimerImpl.java:273)
  at weblogic/work/SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
  at weblogic/work/ExecuteThread.execute(ExecuteThread.java:256)
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:221)
  at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
  -- end of trace




 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms