Durable Subscriber Reconnection Failure With The Default Client ID Policy Set to "Restricted" and The Subscription Sharing Policy Set to "Exclusive".
Last updated on DECEMBER 11, 2017
Applies to:
Oracle WebLogic Server - Version 10.3.4 and laterInformation in this document applies to any platform.
**** Checked for relevance Dec 27th, 2013 ****
Symptoms
The following note delivers a patch to provide retro-compatibility when using Uniform Distributed Topics, using manual failover for Durable Subscriber, to emulate behavior existing in WebLogic Server versions prior 10.3.4. To understand what is the use case and if eventually this patch applies to your environment, please carefully read the use case described bellow.
The following is the JMS use case:
- Distributed topic across two managed servers.
- Producer sends a message to the cluster.
- Consumer connects durably to server A.
- Server A goes down.
- Consumer reconnects durably to server B.
- Server A is brought back online.
- Consumer now has two durable connections as the same subscriber as unable to consume messages.
All of this is using the default Client ID Policy to "Restricted" and Subscription Sharing Policy to "Exclusive".
Complete test case available in here.
Bellow a complete sequence of events:
1. ON MESSAGE CONSUMER SIDE
$ java -cp .:${CLASSPATH} JmsTestCaseSubscriber 1 2 t3://10.149.30.57:7021,10.149.30.57:7022
topic name is jmsTest!jms1@test.update
Connected
mouse
Message has been received.
dog
Message has been received.
JMS Connection has been broken, trying to reestablish... <=================================== MS1 KILLED
javax.naming.NameNotFoundException: Unable to resolve 'jms1@test.update'. Resolved 'jms1@test' [Root exception is javax.naming.NameNotFoundException: Unable to resolve 'jms1@test.update'. Resolved 'jms1@test']; remaining name 'update'
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:348)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259)
at weblogic.jndi.internal.ServerNamingNode_1033_WLStub.lookup(Unknown Source)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:405)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:393)
at javax.naming.InitialContext.lookup(InitialContext.java:392)
at JmsTestCaseSubscriber.connectJms(JmsTestCaseSubscriber.java:67)
at JmsTestCaseSubscriber.onException(JmsTestCaseSubscriber.java:112)
at weblogic.jms.client.JMSExceptionContext$1.run(JMSExceptionContext.java:54)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.jms.client.JMSExceptionContext.invokeListener(JMSExceptionContext.java:50)
at weblogic.jms.client.JMSConnection.onException(JMSConnection.java:1388)
at weblogic.jms.client.JMSSession.onException(JMSSession.java:2925)
at weblogic.jms.client.JMSConsumer.pushException(JMSConsumer.java:754)
at weblogic.jms.client.JMSConsumer.invoke(JMSConsumer.java:771)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:961)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
Caused by: javax.naming.NameNotFoundException: Unable to resolve 'jms1@test.update'. Resolved 'jms1@test'
at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1139)
at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:252)
at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:182)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:206)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:214)
at weblogic.jndi.internal.RootNamingNode_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
topic name is jmsTest!jms2@test.update <========================= FAILOVER OK
Connected
cat <========================= MESSAGE RECEIVED OK ON UDT
Message has been received.
[slcruser@sllinux7 TEST_CASE]$ java -cp .:${CLASSPATH} JmsTestCaseSubscriber 1 2 t3://10.149.30.57:7021,10.149.30.57:7022 <=============== SUBSCRIBER KILLED
[slcruser@sllinux7 TEST_CASE]$ java -cp .:${CLASSPATH} JmsTestCaseSubscriber 1 2 t3://10.149.30.57:7021,10.149.30.57:7022 <=============== SUBSCRIBER ALIVE AGAIN
topic name is jmsTest!jms1@test.update <========================= AFTER BRINGING UP MS1 AGAIN; IT REUSES THE CONNECTION TO MS1
Connected <========================= IT RECEIVES THE DUPLICATE MESSAGE
cat
Message has been received.
topic name is jmsTest!jms1@test.update
Connected
mouse
Message has been received.
dog
Message has been received.
JMS Connection has been broken, trying to reestablish... <=================================== MS1 KILLED
javax.naming.NameNotFoundException: Unable to resolve 'jms1@test.update'. Resolved 'jms1@test' [Root exception is javax.naming.NameNotFoundException: Unable to resolve 'jms1@test.update'. Resolved 'jms1@test']; remaining name 'update'
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:348)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259)
at weblogic.jndi.internal.ServerNamingNode_1033_WLStub.lookup(Unknown Source)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:405)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:393)
at javax.naming.InitialContext.lookup(InitialContext.java:392)
at JmsTestCaseSubscriber.connectJms(JmsTestCaseSubscriber.java:67)
at JmsTestCaseSubscriber.onException(JmsTestCaseSubscriber.java:112)
at weblogic.jms.client.JMSExceptionContext$1.run(JMSExceptionContext.java:54)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.jms.client.JMSExceptionContext.invokeListener(JMSExceptionContext.java:50)
at weblogic.jms.client.JMSConnection.onException(JMSConnection.java:1388)
at weblogic.jms.client.JMSSession.onException(JMSSession.java:2925)
at weblogic.jms.client.JMSConsumer.pushException(JMSConsumer.java:754)
at weblogic.jms.client.JMSConsumer.invoke(JMSConsumer.java:771)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:961)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
Caused by: javax.naming.NameNotFoundException: Unable to resolve 'jms1@test.update'. Resolved 'jms1@test'
at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1139)
at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:252)
at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:182)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:206)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:214)
at weblogic.jndi.internal.RootNamingNode_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
topic name is jmsTest!jms2@test.update <========================= FAILOVER OK
Connected
cat <========================= MESSAGE RECEIVED OK ON UDT
Message has been received.
[slcruser@sllinux7 TEST_CASE]$ java -cp .:${CLASSPATH} JmsTestCaseSubscriber 1 2 t3://10.149.30.57:7021,10.149.30.57:7022 <=============== SUBSCRIBER KILLED
[slcruser@sllinux7 TEST_CASE]$ java -cp .:${CLASSPATH} JmsTestCaseSubscriber 1 2 t3://10.149.30.57:7021,10.149.30.57:7022 <=============== SUBSCRIBER ALIVE AGAIN
topic name is jmsTest!jms1@test.update <========================= AFTER BRINGING UP MS1 AGAIN; IT REUSES THE CONNECTION TO MS1
Connected <========================= IT RECEIVES THE DUPLICATE MESSAGE
cat
Message has been received.
2. ON MESSAGE PRODUCER SIDE
$ java -cp .:${CLASSPATH} TopicSend t3://10.149.30.57:7021,10.149.30.57:7022
Enter message ("quit" to quit):
mouse
JMS Message Sent: mouse
Enter message ("quit" to quit):
dog
JMS Message Sent: dog <================== AT THIS POINT MS1 WAS KILLED.
Enter message ("quit" to quit): <================== MESSAGE SENT WHILE MS1 WAS DOWN (STEP 6)
cat
JMS Message Sent: cat
Enter message ("quit" to quit):
Enter message ("quit" to quit):
mouse
JMS Message Sent: mouse
Enter message ("quit" to quit):
dog
JMS Message Sent: dog <================== AT THIS POINT MS1 WAS KILLED.
Enter message ("quit" to quit): <================== MESSAGE SENT WHILE MS1 WAS DOWN (STEP 6)
cat
JMS Message Sent: cat
Enter message ("quit" to quit):
At this point an error message like the following will be received at consumer's side:
javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
at weblogic.jms.client.JMSSession.consumerCreate(JMSSession.java:2982)
at weblogic.jms.client.JMSSession.setupConsumer(JMSSession.java:2749)
at weblogic.jms.client.JMSSession.createConsumer(JMSSession.java:2691)
at weblogic.jms.client.JMSSession.createDurableSubscriber(JMSSession.java:2487)
at weblogic.jms.client.JMSSession.createDurableSubscriber(JMSSession.java:2460)
at weblogic.jms.client.WLSessionImpl.createDurableSubscriber(WLSessionImpl.java:1204)
at JmsTestCaseSubscriber.connectJms(JmsTestCaseSubscriber.java:80)
at JmsTestCaseSubscriber.<init>(JmsTestCaseSubscriber.java:46)
at JmsTestCaseSubscriber.main(JmsTestCaseSubscriber.java:160)
Caused by: javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
at weblogic.jms.frontend.FEConsumer.<init>(FEConsumer.java:296)
at weblogic.jms.frontend.FESession$2.run(FESession.java:1076)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.jms.frontend.FESession.consumerCreate(FESession.java:1072)
at weblogic.jms.frontend.FESession.invoke(FESession.java:3027)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:961)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
Caused by: javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:87)
at weblogic.jms.dispatcher.Request.getResult(Request.java:52)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1124)
at weblogic.messaging.dispatcher.DispatcherImpl.syncRequest(DispatcherImpl.java:184)
at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSync(DispatcherImpl.java:219)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java :43)
... 13 more
Caused by: javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.backend.BEConsumerImpl.registerDurableSubscription(BEConsumerImpl.java:1032)
at weblogic.jms.backend.BEConsumerImpl.init(BEConsumerImpl.java:348)
at weblogic.jms.backend.BEConsumerImpl.<init>(BEConsumerImpl.java:286)
at weblogic.jms.backend.BETopicImpl.createConsumer(BETopicImpl.java:479)
at weblogic.jms.backend.BETopicImpl.createConsumer(BETopicImpl.java:395)
at weblogic.jms.backend.BESessionImpl.createBEConsumer(BESessionImpl.java:468)
at weblogic.jms.backend.BESessionImpl.createConsumer(BESessionImpl.java:478)
at weblogic.jms.backend.BESessionImpl.invoke(BESessionImpl.java:296)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.javat(DispatcherImpl.java:184)
at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSync(DispatcherImpl.java:219)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:43)
at weblogic.jms.frontend.FEConsumer.<init>(FEConsumer.java:296)
at weblogic.jms.frontend.FESession$2.run(FESession.java:1076)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.jms.frontend.FESession.consumerCreate(FESession.java:1072)
at weblogic.jms.frontend.FESession.invoke(FESession.java:3027)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:961)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:112)
... 2 more
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
at weblogic.jms.client.JMSSession.consumerCreate(JMSSession.java:2982)
at weblogic.jms.client.JMSSession.setupConsumer(JMSSession.java:2749)
at weblogic.jms.client.JMSSession.createConsumer(JMSSession.java:2691)
at weblogic.jms.client.JMSSession.createDurableSubscriber(JMSSession.java:2487)
at weblogic.jms.client.JMSSession.createDurableSubscriber(JMSSession.java:2460)
at weblogic.jms.client.WLSessionImpl.createDurableSubscriber(WLSessionImpl.java:1204)
at JmsTestCaseSubscriber.connectJms(JmsTestCaseSubscriber.java:80)
at JmsTestCaseSubscriber.<init>(JmsTestCaseSubscriber.java:46)
at JmsTestCaseSubscriber.main(JmsTestCaseSubscriber.java:160)
Caused by: javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
at weblogic.jms.frontend.FEConsumer.<init>(FEConsumer.java:296)
at weblogic.jms.frontend.FESession$2.run(FESession.java:1076)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.jms.frontend.FESession.consumerCreate(FESession.java:1072)
at weblogic.jms.frontend.FESession.invoke(FESession.java:3027)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:961)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:111)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
Caused by: javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:87)
at weblogic.jms.dispatcher.Request.getResult(Request.java:52)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1124)
at weblogic.messaging.dispatcher.DispatcherImpl.syncRequest(DispatcherImpl.java:184)
at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSync(DispatcherImpl.java:219)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java :43)
... 13 more
Caused by: javax.jms.JMSException: Durable subscription test.updateTestSubscription is in use and cannot be shared
at weblogic.jms.backend.BEConsumerImpl.registerDurableSubscription(BEConsumerImpl.java:1032)
at weblogic.jms.backend.BEConsumerImpl.init(BEConsumerImpl.java:348)
at weblogic.jms.backend.BEConsumerImpl.<init>(BEConsumerImpl.java:286)
at weblogic.jms.backend.BETopicImpl.createConsumer(BETopicImpl.java:479)
at weblogic.jms.backend.BETopicImpl.createConsumer(BETopicImpl.java:395)
at weblogic.jms.backend.BESessionImpl.createBEConsumer(BESessionImpl.java:468)
at weblogic.jms.backend.BESessionImpl.createConsumer(BESessionImpl.java:478)
at weblogic.jms.backend.BESessionImpl.invoke(BESessionImpl.java:296)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.javat(DispatcherImpl.java:184)
at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSync(DispatcherImpl.java:219)
at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:43)
at weblogic.jms.frontend.FEConsumer.<init>(FEConsumer.java:296)
at weblogic.jms.frontend.FESession$2.run(FESession.java:1076)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)
at weblogic.jms.frontend.FESession.consumerCreate(FESession.java:1072)
at weblogic.jms.frontend.FESession.invoke(FESession.java:3027)
at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:961)
at weblogic.messaging.dispatcher.DispatcherServerRef.invoke(DispatcherServerRef.java:276)
at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:141)
at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:34)
at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:112)
... 2 more
Cause
Sign In with your My Oracle Support account |
|
Don't have a My Oracle Support account? Click to get started |
My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms