Bad Performance Caused by JRockit Defect Regarding ConstantPool (Doc ID 1562076.1)

Last updated on DECEMBER 15, 2016

Applies to:

Oracle WebCenter Portal - Version 11.1.1.6.4 and later
Information in this document applies to any platform.

Symptoms

WebCenter Portal Server is running with bad performance. Thread dump shows that critical thread lock & wait problem.  In the following sample,  ExecuteThread: '400' is blocked trying to get locked object  java/lang/Object@0x16599b5c0, which is holding by ExecuteThread: '336'. Due to ExecuteThread: '336' is stucking at java/lang/Class.getConstantPool()Lsun/reflect/ConstantPool;(Native Method), the ExecuteThread: '400' and others almost 300 threads are waiting for lock same object.

"[ACTIVE] ExecuteThread: '336' for queue: 'weblogic.kernel.Default (self-tuning)'" id=2653 idx=0x79c tid=26060 prio=5 alive, native_waiting, daemon
   at java/lang/Class.getConstantPool()Lsun/reflect/ConstantPool;(Native Method)
   at java/lang/System$2.getConstantPool(System.java:1130)[optimized]
   at java/lang/reflect/Method.declaredAnnotations(Method.java:693)[inlined]
   at java/lang/reflect/Method.getAnnotation(Method.java:679)[inlined]
   at java/lang/reflect/AccessibleObject.isAnnotationPresent(AccessibleObject.java:168)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.notifyPreDestroy(BridgeImpl.java:2518)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.notifyPreDestroy(BridgeImpl.java:2508)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.access$200(BridgeImpl.java:129)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl$LRUMap.remove(BridgeImpl.java:2809)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl$LRUMap.removeEldestEntry(BridgeImpl.java:2797)[optimized]
   ^-- Holding lock: java/lang/reflect/Method@0x1a9414c20[biased lock]
   at java/util/LinkedHashMap.addEntry(LinkedHashMap.java:410)[optimized]
   at java/util/HashMap.put(HashMap.java:477)[optimized]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl$LRUMap.put(BridgeImpl.java:2817)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.putBridgeRequestScopeData(BridgeImpl.java:2057)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.saveBridgeRequestScopeData(BridgeImpl.java:2002)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.doFacesRender(BridgeImpl.java:1065)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.doFacesRequest(BridgeImpl.java:903)[optimized]
   ^-- Holding lock: java/lang/Object@0x16599b5c0[fat lock]
   at javax/portlet/faces/GenericFacesPortlet.doBridgeDispatch(GenericFacesPortlet.java:678)[inlined]
   at javax/portlet/faces/GenericFacesPortlet.doRenderDispatchInternal(GenericFacesPortlet.java:644)[optimized]
   at javax/portlet/faces/GenericFacesPortlet.doView(GenericFacesPortlet.java:280)[inlined]
   at javax/portlet/GenericPortlet.doDispatch(GenericPortlet.java:328)[inlined]
   at javax/portlet/faces/GenericFacesPortlet.doDispatch(GenericFacesPortlet.java:251)[inlined]
   at javax/portlet/GenericPortlet.render(GenericPortlet.java:233)[inlined]
   at oracle/portlet/bridge/adf/application/ADFBridgePortlet.render(ADFBridgePortlet.java:255)[optimized]
   at com/bea/portlet/container/PortletStub.doRender(PortletStub.java:1012)[optimized]
   at com/bea/portlet/container/FilterChainGenerator.runFilterChain(FilterChainGenerator.java:127)[inlined]
   at com/bea/portlet/container/PortletStub.render(PortletStub.java:438)[optimized]
   at com/bea/portlet/container/AppContainer.renderStub(AppContainer.java:1158)[inlined]
   at com/bea/portlet/container/AppContainer.invokeRender(AppContainer.java:1090)[optimized]
   at com/bea/wsrp/producer/adapter/javaportlet/JavaPortletAdapter.doGetMarkup(JavaPortletAdapter.java:294)[optimized]
   at com/bea/wsrp/producer/handlers/markup/MarkupServiceHandler.doGetMarkup(MarkupServiceHandler.java:449)[inlined]
   at com/bea/wsrp/producer/handlers/markup/MarkupServiceHandler.doService(MarkupServiceHandler.java:264)[optimized]
   at com/bea/wsrp/producer/handlers/AbstractServiceHandler.service(AbstractServiceHandler.java:71)[optimized]
   at com/bea/wsrp/producer/container/ProducerEndPoint.processNow(ProducerEndPoint.java:349)[inlined]
   at com/bea/wsrp/producer/container/ProducerEndPoint.processNow(ProducerEndPoint.java:250)[inlined]
   at com/bea/wsrp/producer/container/ProducerEndPoint.processNow(ProducerEndPoint.java:208)[optimized]
   at oracle/portlet/server/adapter/web/WSRP_v2_Markup_PortTypeSoapToEndpoint.getMarkup(WSRP_v2_Markup_PortTypeSoapToEndpoint.java:74)[optimized]
   at oasis/names/tc/wsrp/v2/bind/runtime/WSRP_v2_Markup_Binding_SOAP_Tie.invoke_getMarkup(WSRP_v2_Markup_Binding_SOAP_Tie.java:1055)[optimized]
   at oasis/names/tc/wsrp/v2/bind/runtime/WSRP_v2_Markup_Binding_SOAP_Tie.processingHook(WSRP_v2_Markup_Binding_SOAP_Tie.java:1460)[optimized]
   at oracle/j2ee/ws/server/StreamingHandler.handle(StreamingHandler.java:299)[optimized]
   at oracle/j2ee/ws/server/JAXRPCProcessor.doEndpointProcessing(JAXRPCProcessor.java:442)[inlined]
   at oracle/j2ee/ws/server/WebServiceProcessor.invokeEndpointImplementation(WebServiceProcessor.java:1112)[optimized]
   at oracle/j2ee/ws/server/JAXRPCProcessor.doRequestProcessing(JAXRPCProcessor.java:333)[optimized]
   at oracle/j2ee/ws/server/WebServiceProcessor.processRequest(WebServiceProcessor.java:233)[optimized]
   at oracle/j2ee/ws/server/JAXRPCProcessor.doService(JAXRPCProcessor.java:185)[optimized]
   at oracle/j2ee/ws/server/WebServiceServlet.doPost(WebServiceServlet.java:485)[optimized]
   at javax/servlet/http/HttpServlet.service(HttpServlet.java:727)[optimized]
   at javax/servlet/http/HttpServlet.service(HttpServlet.java:820)[optimized]
   at weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)[optimized]
   at weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)[inlined]
   at weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:301)[inlined]
   at weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/portlet/producer/container/endpoint/ProducerEndpointFilter.doFilter(ProducerEndpointFilter.java:36)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/security/wls/filter/SSOSessionSynchronizationFilter.doFilter(SSOSessionSynchronizationFilter.java:291)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/wcps/client/PersonalizationFilter.doFilter(PersonalizationFilter.java:75)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/webcenter/content/integration/servlets/ContentServletFilter.doFilter(ContentServletFilter.java:168)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/adf/library/webapp/LibraryFilter.doFilter(LibraryFilter.java:179)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/security/jps/ee/http/JpsAbsFilter$1.run(JpsAbsFilter.java:119)[optimized]
   at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)[inlined]
   at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:315)[inlined]
   at oracle/security/jps/ee/util/JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:442)[inlined]
   at oracle/security/jps/ee/http/JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)[inlined]
   at oracle/security/jps/ee/http/JpsAbsFilter.doFilter(JpsAbsFilter.java:171)[optimized]
   at oracle/security/jps/ee/http/JpsFilter.doFilter(JpsFilter.java:71)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/security/jps/ee/http/JpsAbsFilter$1.run(JpsAbsFilter.java:119)[optimized]
   at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)[inlined]
   at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:315)[inlined]
   at oracle/security/jps/ee/util/JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:442)[inlined]
   at oracle/security/jps/ee/http/JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)[inlined]
   at oracle/security/jps/ee/http/JpsAbsFilter.doFilter(JpsAbsFilter.java:171)[optimized]
   at oracle/security/jps/ee/http/JpsFilter.doFilter(JpsFilter.java:71)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at oracle/dms/servlet/DMSServletFilter.doFilter(DMSServletFilter.java:139)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]
   at weblogic/servlet/internal/RequestEventsFilter.doFilter(RequestEventsFilter.java:27)[optimized]
   at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[inlined]
   at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3730)[inlined]
   at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3696)[optimized]
   at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized]
   at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:120)[inlined]
   at weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2273)[inlined]
   at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:2179)[optimized]
   at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1490)[optimized]
   at weblogic/work/ExecuteThread.execute(ExecuteThread.java:256)[inlined]
   at weblogic/work/ExecuteThread.run(ExecuteThread.java:221)[optimized]
   at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
   -- end of trace

"[ACTIVE] ExecuteThread: '400' for queue: 'weblogic.kernel.Default (self-tuning)'" id=2722 idx=0x888 tid=27958 prio=5 alive, blocked, native_blocked, daemon
   -- Blocked trying to get lock: java/lang/Object@0x16599b5c0[fat lock]
   at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
   at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1411)[optimized]
   at jrockit/vm/Locks.lockFat(Locks.java:1512)[optimized]
   at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1054)[optimized]
   at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1005)[optimized]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.putBridgeRequestScopeData(BridgeImpl.java:2018)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.saveBridgeRequestScopeData(BridgeImpl.java:2002)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.doFacesRender(BridgeImpl.java:1065)[inlined]
   at org/apache/myfaces/portlet/faces/bridge/BridgeImpl.doFacesRequest(BridgeImpl.java:903)[optimized]
   at javax/portlet/faces/GenericFacesPortlet.doBridgeDispatch(GenericFacesPortlet.java:678)[inlined]
   at javax/portlet/faces/GenericFacesPortlet.doRenderDispatchInternal(GenericFacesPortlet.java:644)[optimized]
   at javax/portlet/faces/GenericFacesPortlet.doView(GenericFacesPortlet.java:280)[inlined]
   at javax/portlet/GenericPortlet.doDispatch(GenericPortlet.java:328)[inlined]
   at javax/portlet/faces/GenericFacesPortlet.doDispatch(GenericFacesPortlet.java:251)[inlined]
   at javax/portlet/GenericPortlet.render(GenericPortlet.java:233)[inlined]
   at oracle/portlet/bridge/adf/application/ADFBridgePortlet.render(ADFBridgePortlet.java:255)[optimized]
   at com/bea/portlet/container/PortletStub.doRender(PortletStub.java:1012)[optimized]
   at com/bea/portlet/container/FilterChainGenerator.runFilterChain(FilterChainGenerator.java:127)[inlined]
   at com/bea/portlet/container/PortletStub.render(PortletStub.java:438)[optimized]
   at com/bea/portlet/container/AppContainer.renderStub(AppContainer.java:1158)[inlined]
   at com/bea/portlet/container/AppContainer.invokeRender(AppContainer.java:1090)[optimized]
   at com/bea/wsrp/producer/adapter/javaportlet/JavaPortletAdapter.doGetMarkup(JavaPortletAdapter.java:294)[optimized]
   at com/bea/wsrp/producer/handlers/markup/MarkupServiceHandler.doGetMarkup(MarkupServiceHandler.java:449)[inlined]
   at com/bea/wsrp/producer/handlers/markup/MarkupServiceHandler.doService(MarkupServiceHandler.java:264)[optimized]
   at com/bea/wsrp/producer/handlers/AbstractServiceHandler.service(AbstractServiceHandler.java:71)[optimized]
   at com/bea/wsrp/producer/container/ProducerEndPoint.processNow(ProducerEndPoint.java:349)[inlined]
   at com/bea/wsrp/producer/container/ProducerEndPoint.processNow(ProducerEndPoint.java:250)[inlined]
   at com/bea/wsrp/producer/container/ProducerEndPoint.processNow(ProducerEndPoint.java:208)[optimized]
   at oracle/portlet/server/adapter/web/WSRP_v2_Markup_PortTypeSoapToEndpoint.getMarkup(WSRP_v2_Markup_PortTypeSoapToEndpoint.java:74)[optimized]
   at oasis/names/tc/wsrp/v2/bind/runtime/WSRP_v2_Markup_Binding_SOAP_Tie.invoke_getMarkup(WSRP_v2_Markup_Binding_SOAP_Tie.java:1055)[optimized]
   at oasis/names/tc/wsrp/v2/bind/runtime/WSRP_v2_Markup_Binding_SOAP_Tie.processingHook(WSRP_v2_Markup_Binding_SOAP_Tie.java:1460)[optimized]
   at oracle/j2ee/ws/server/StreamingHandler.handle(StreamingHandler.java:299)[optimized]
   at oracle/j2ee/ws/server/JAXRPCProcessor.doEndpointProcessing(JAXRPCProcessor.java:442)[inlined]
   at oracle/j2ee/ws/server/WebServiceProcessor.invokeEndpointImplementation(WebServiceProcessor.java:1112)[optimized]
   at oracle/j2ee/ws/server/JAXRPCProcessor.doRequestProcessing(JAXRPCProcessor.java:333)[optimized]
   at oracle/j2ee/ws/server/WebServiceProcessor.processRequest(WebServiceProcessor.java:233)[optimized]
   at oracle/j2ee/ws/server/JAXRPCProcessor.doService(JAXRPCProcessor.java:185)[optimized]
   at oracle/j2ee/ws/server/WebServiceServlet.doPost(WebServiceServlet.java:485)[optimized]
   at javax/servlet/http/HttpServlet.service(HttpServlet.java:727)[optimized]

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms