My Oracle Support Banner

Large Weblogic cluster shows excessive garbage collection (Doc ID 1513226.1)

Last updated on FEBRUARY 25, 2019

Applies to:

Oracle WebCenter Sites - Version 11.1.1.6.1 and later
Information in this document applies to any platform.

Symptoms

A large cluster of WebCenter Sites nodes (~60 of them) exhibits a memory leak (slow heap growth over time, until the heap is exhausted, and garbage collection does not free any heap). Also there are errors showing Unicast errors with members going in and out of the cluster. There is no traffic at all to the cluster and no user activity. The WLS cluster is configured using unicast.

WLS logs show:

####<09-Dec-2012 23:59:41 o'clock GMT> <Info> <Cluster> <cbl12074app08> <ms41-cbpcsd01> <[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1355097581380> <BEA-000111> <Adding ms3-cbpcsd01 with ID -2167371751009697640S:ms3-cbpcsd01.customer.com:[61003,61003,-1,-1,-1,-1,-1]:cbpcsd01:ms3-cbpcsd01 to cluster: cluster_cbpcsd01 view.>
####<09-Dec-2012 23:59:41 o'clock GMT> <Error> <Kernel> <cbl12074app08> <ms41-cbpcsd01> <[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1355097581381> <BEA-000802> <ExecuteRequest failed
java.lang.NullPointerException.
java.lang.NullPointerException
at weblogic.cluster.RemoteMemberInfo.add(RemoteMemberInfo.java:400)
at weblogic.cluster.RemoteMemberInfo.updateRuntimeState(RemoteMemberInfo.java:358)
at weblogic.cluster.MemberManager.updateMemberRuntimeState(MemberManager.java:577)
at weblogic.cluster.RuntimeStateMessage.execute(RuntimeStateMessage.java:33)
at weblogic.cluster.HTTPExecuteRequest$1.run(HTTPExecuteRequest.java:96)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.cluster.HTTPExecuteRequest.run(HTTPExecuteRequest.java:93)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)


####<10-Dec-2012 00:12:43 o'clock GMT> <Info> <Cluster> <cbl12074app08> <ms41-cbpcsd01> <[ACTIVE] ExecuteThread: '29' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1355098363609> <BEA-000128> <Updating -3363555987484002118S:ms2-cbpcsd01.customer.com:[61002,61002,-1,-1,-1,-1,-1]:cbpcsd01:ms2-cbpcsd01 in the cluster.>
####<10-Dec-2012 00:12:43 o'clock GMT> <Error> <Kernel> <cbl12074app08> <ms41-cbpcsd01> <[ACTIVE] ExecuteThread: '29' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1355098363610> <BEA-000802> <ExecuteRequest failed
java.lang.NullPointerException.
java.lang.NullPointerException
at weblogic.cluster.UpgradeUtils.clusterMembersChanged(UpgradeUtils.java:84)
at weblogic.cluster.MemberManager$3.run(MemberManager.java:523)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

####<10-Dec-2012 05:54:11 o'clock GMT> <Error> <Kernel> <cbl12074app08> <ms41-cbpcsd01> <[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1355118851424> <BEA-000802> <ExecuteRequest failed
java.lang.NullPointerException.
java.lang.NullPointerException
at weblogic.iiop.ClusterServices.clusterMembersChanged(ClusterServices.java:77)
at weblogic.cluster.MemberManager$3.run(MemberManager.java:523)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

Changes

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.