Restarting a Cluster Fails to Start All Managed Server with Error: "oracle.ias.cache.NetworkException: J2EE JOC-058 distributed cache initialization failure"
(Doc ID 1259754.1)
Last updated on SEPTEMBER 01, 2023
Applies to:
Oracle WebCenter Portal - Version 11.1.1.2.0 to 11.1.1.9.171222 [Release 11g]Information in this document applies to any platform.
Symptoms
In an environment with a WLS_Spaces cluster consisting of three managed servers, when restarting the cluster, one of the managed servers in the cluster will not start.
In order to start all managed servers, it is required to shut down all three and then start them individually.
The WLS_Spaces_diagnostic.log shows the following related message:
[2010-10-27T10:46:17.030-05:00] [WLS_Spaces0] [ERROR] [] [oracle.as.cache.groupv2.Distributor] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [ecid: <ECID>] [APP: webcenter] DI Unable to find or become a group coordinator within 300000 seconds
[2010-10-27T10:46:19.067-05:00] [WLS_Spaces0] [ERROR] [] [oracle.as.cache] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [ecid: <ECID>] [APP: webcenter] [[
oracle.ias.cache.NetworkException: J2EE JOC-058 distributed cache initialization failure
J2EE JOC-043 base exception:
J2EE JOC-823 unable to contact the packet-distributor at [localhost:<PORT>] segID=0
at oracle.ias.cache.groupv2.Client.init(Client.java:86)
at oracle.ias.cache.groupv2.GroupMember.<init>(GroupMember.java:383)
at oracle.ias.cache.groupv2.GroupMember.create(GroupMember.java:93)
at oracle.ias.cache.groupv2.GrpCommunication.init(GrpCommunication.java:141)
at oracle.ias.cache.groupv2.GrpCommunication.init(GrpCommunication.java:121)
at oracle.ias.cache.Net.init(Net.java:104)
at oracle.ias.cache.CacheInternal.initNet(CacheInternal.java:2920)
at oracle.ias.cache.CacheInternal.init(CacheInternal.java:695)
at oracle.ias.cache.CacheInternal.init(CacheInternal.java:343)
at oracle.ias.cache.CacheInternal.createNamedCache(CacheInternal.java:2596)
at oracle.ias.cache.Cache.createNamedCache(Cache.java:661)
at oracle.mds.internal.cache.JOCCacheProvider.createNamedCacheInternal(JOCCacheProvider.java:292)
at oracle.mds.internal.cache.JOCCacheProvider.createNamedCache(JOCCacheProvider.java:254)
at oracle.mds.internal.cache.JOCCacheProvider.<init>(JOCCacheProvider.java:87)
at oracle.mds.core.MDSInstance.initCache(MDSInstance.java:1584)
at oracle.mds.core.MDSInstance.<init>(MDSInstance.java:1710)
at oracle.mds.core.MDSInstance.createMDSInstanceWithCustomizedConfig(MDSInstance.java:1226)
at oracle.mds.core.MDSInstance.getOrCreateInstance(MDSInstance.java:569)
at oracle.adf.share.config.ADFMDSConfig.createMDSInstanceFromelement(ADFMDSConfig.java:243)
at oracle.adf.share.config.MDSConfigFactory.createApplication(MDSConfigFactory.java:121)
at oracle.adf.share.config.ADFConfigFactory.findOrCreateADFConfig(ADFConfigFactory.java:159)
at oracle.adf.share.config.ADFConfigFactory.findOrCreateADFConfig(ADFConfigFactory.java:78)
at oracle.adf.share.config.ADFConfigFactory.findOrCreateADFConfig(ADFConfigFactory.java:56)
at oracle.adf.share.ADFContext.getADFConfigEx(ADFContext.java:691)
at oracle.adf.share.ADFContext.getADFConfig(ADFContext.java:675)
at oracle.webcenter.framework.internal.service.DefaultServiceContext.getDefaultDataSourceName(DefaultServiceContext.java:277)
at oracle.webcenter.framework.internal.service.WebcenterJNDITranslator.getWebCenterJNDIName(WebcenterJNDITranslator.java:92)
at oracle.webcenter.framework.internal.service.WebcenterJNDITranslator.translateDatasourceJNDI(WebcenterJNDITranslator.java:73)
at com.bea.p13n.util.JndiHelper.translateDatasourceName(JndiHelper.java:131)
at com.bea.p13n.util.JndiHelper.lookupDataSource(JndiHelper.java:147)
at com.bea.p13n.jobmanager.internal.JobManagerImpl.getConnection(JobManagerImpl.java:2393)
at com.bea.p13n.jobmanager.internal.JobManagerImpl.shouldStartJobManager(JobManagerImpl.java:1329)
at com.bea.p13n.jobmanager.internal.JobManagerImpl.getInstance(JobManagerImpl.java:624)
at com.bea.p13n.jobmanager.internal.JobManagerLocalBean.ejbCreate(JobManagerLocalBean.java:21)
at com.bea.p13n.jobmanager.internal.JobManager_jb37cr_Impl.ejbCreate(JobManager_jb37cr_Impl.java:162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at weblogic.ejb.container.pool.StatelessSessionPool.createBean(StatelessSessionPool.java:216)
at weblogic.ejb.container.pool.Pool.createInitialBeans(Pool.java:300)
at weblogic.ejb.container.manager.StatelessManager.initializePool(StatelessManager.java:448)
at weblogic.ejb.container.deployer.EJBDeployer.initializePools(EJBDeployer.java:1634)
at weblogic.ejb.container.deployer.EJBDeployer.start(EJBDeployer.java:1464)
at weblogic.ejb.container.deployer.EJBModule.start(EJBModule.java:601)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:83)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:1267)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:83)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:409)
at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:54)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
at weblogic.deploy.internal.targetserver.BasicDeployment.activate(BasicDeployment.java:184)
at weblogic.deploy.internal.targetserver.BasicDeployment.activateFromServerLifecycle(BasicDeployment.java:361)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doActivate(DeploymentAdapter.java:51)
at weblogic.management.deploy.internal.DeploymentAdapter.activate(DeploymentAdapter.java:196)
at weblogic.management.deploy.internal.AppTransition$2.transitionApp(AppTransition.java:30)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:233)
at weblogic.management.deploy.internal.ConfiguredDeployments.activate(ConfiguredDeployments.java:169)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:123)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:173)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:89)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)ms/thread.jspa?threadID=957741
[2010-10-27T10:46:19.067-05:00] [WLS_Spaces0] [ERROR] [] [oracle.as.cache] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [ecid: <ECID>] [APP: webcenter] [[
oracle.ias.cache.NetworkException: J2EE JOC-058 distributed cache initialization failure
J2EE JOC-043 base exception:
J2EE JOC-823 unable to contact the packet-distributor at [localhost:<PORT>] segID=0
at oracle.ias.cache.groupv2.Client.init(Client.java:86)
at oracle.ias.cache.groupv2.GroupMember.<init>(GroupMember.java:383)
at oracle.ias.cache.groupv2.GroupMember.create(GroupMember.java:93)
at oracle.ias.cache.groupv2.GrpCommunication.init(GrpCommunication.java:141)
at oracle.ias.cache.groupv2.GrpCommunication.init(GrpCommunication.java:121)
at oracle.ias.cache.Net.init(Net.java:104)
at oracle.ias.cache.CacheInternal.initNet(CacheInternal.java:2920)
at oracle.ias.cache.CacheInternal.init(CacheInternal.java:695)
at oracle.ias.cache.CacheInternal.init(CacheInternal.java:343)
at oracle.ias.cache.CacheInternal.createNamedCache(CacheInternal.java:2596)
at oracle.ias.cache.Cache.createNamedCache(Cache.java:661)
at oracle.mds.internal.cache.JOCCacheProvider.createNamedCacheInternal(JOCCacheProvider.java:292)
at oracle.mds.internal.cache.JOCCacheProvider.createNamedCache(JOCCacheProvider.java:254)
at oracle.mds.internal.cache.JOCCacheProvider.<init>(JOCCacheProvider.java:87)
at oracle.mds.core.MDSInstance.initCache(MDSInstance.java:1584)
at oracle.mds.core.MDSInstance.<init>(MDSInstance.java:1710)
at oracle.mds.core.MDSInstance.createMDSInstanceWithCustomizedConfig(MDSInstance.java:1226)
at oracle.mds.core.MDSInstance.getOrCreateInstance(MDSInstance.java:569)
at oracle.adf.share.config.ADFMDSConfig.createMDSInstanceFromelement(ADFMDSConfig.java:243)
at oracle.adf.share.config.MDSConfigFactory.createApplication(MDSConfigFactory.java:121)
at oracle.adf.share.config.ADFConfigFactory.findOrCreateADFConfig(ADFConfigFactory.java:159)
at oracle.adf.share.config.ADFConfigFactory.findOrCreateADFConfig(ADFConfigFactory.java:78)
at oracle.adf.share.config.ADFConfigFactory.findOrCreateADFConfig(ADFConfigFactory.java:56)
at oracle.adf.share.ADFContext.getADFConfigEx(ADFContext.java:691)
at oracle.adf.share.ADFContext.getADFConfig(ADFContext.java:675)
at oracle.webcenter.framework.internal.service.DefaultServiceContext.getDefaultDataSourceName(DefaultServiceContext.java:277)
at oracle.webcenter.framework.internal.service.WebcenterJNDITranslator.getWebCenterJNDIName(WebcenterJNDITranslator.java:92)
at oracle.webcenter.framework.internal.service.WebcenterJNDITranslator.translateDatasourceJNDI(WebcenterJNDITranslator.java:73)
at com.bea.p13n.util.JndiHelper.translateDatasourceName(JndiHelper.java:131)
at com.bea.p13n.util.JndiHelper.lookupDataSource(JndiHelper.java:147)
at com.bea.p13n.jobmanager.internal.JobManagerImpl.getConnection(JobManagerImpl.java:2393)
at com.bea.p13n.jobmanager.internal.JobManagerImpl.shouldStartJobManager(JobManagerImpl.java:1329)
at com.bea.p13n.jobmanager.internal.JobManagerImpl.getInstance(JobManagerImpl.java:624)
at com.bea.p13n.jobmanager.internal.JobManagerLocalBean.ejbCreate(JobManagerLocalBean.java:21)
at com.bea.p13n.jobmanager.internal.JobManager_jb37cr_Impl.ejbCreate(JobManager_jb37cr_Impl.java:162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at weblogic.ejb.container.pool.StatelessSessionPool.createBean(StatelessSessionPool.java:216)
at weblogic.ejb.container.pool.Pool.createInitialBeans(Pool.java:300)
at weblogic.ejb.container.manager.StatelessManager.initializePool(StatelessManager.java:448)
at weblogic.ejb.container.deployer.EJBDeployer.initializePools(EJBDeployer.java:1634)
at weblogic.ejb.container.deployer.EJBDeployer.start(EJBDeployer.java:1464)
at weblogic.ejb.container.deployer.EJBModule.start(EJBModule.java:601)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:83)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:1267)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:83)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:409)
at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:54)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
at weblogic.deploy.internal.targetserver.BasicDeployment.activate(BasicDeployment.java:184)
at weblogic.deploy.internal.targetserver.BasicDeployment.activateFromServerLifecycle(BasicDeployment.java:361)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doActivate(DeploymentAdapter.java:51)
at weblogic.management.deploy.internal.DeploymentAdapter.activate(DeploymentAdapter.java:196)
at weblogic.management.deploy.internal.AppTransition$2.transitionApp(AppTransition.java:30)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:233)
at weblogic.management.deploy.internal.ConfiguredDeployments.activate(ConfiguredDeployments.java:169)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:123)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:173)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:89)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)ms/thread.jspa?threadID=957741
Changes
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |