WebLogic Server SSL Configuration "Client Certs Requested And Enforced" - Cluster Member Reconnection Attempt Fails With tls.record.handshake.HandshakeHandler.fireAlert (Doc ID 1357484.1)

Last updated on NOVEMBER 05, 2016

Applies to:

Oracle WebLogic Server - Version 10.3 and later
Information in this document applies to any platform.

Symptoms

Consider this configuration; compare to your situation.

1. Start BL1 (Business Layer) and BL2, which are clustered together.
2. Start PL (Presentation Layer) server which acts as client.
3. Configure 2-way SSL between PL and BL servers.
4. Send 10 requests from PL and both BLs services the requests (its equally load balanced)
5. Stop one of the BL server (say BL1)
6. send 10 requests again from PL -> BL2 services the requests (as expected since only one BL is available and everything works fine.)
7. Now start BL1
8. Send 10 requests from PL -> here you would be thrown with exception.


Attempts to reconnect to the cluster fail when SSL is set to Client Certs Requested And Enforced using two way SSL.

On PL server you can see the exception as below.

<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Trust status (0): NONE>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Performing hostname validation checks: localhost>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <isMuxerActivated: false>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <13293943 SSL3/TLS MAC>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <13293943 received HANDSHAKE>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <HANDSHAKEMESSAGE: CertificateRequest>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <isMuxerActivated: false>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <13293943 SSL3/TLS MAC>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <13293943 received HANDSHAKE>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <HANDSHAKEMESSAGE: ServerHelloDone>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <No suitable identity certificate chain has been found.>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <write HANDSHAKE, offset = 0, length = 7>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Ignoring not supported JCE Mac: SunJCE version 1.6 for algorithm HmacMD5>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Will use default Mac for algorithm HmacMD5>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Ignoring not supported JCE Mac: SunJCE version 1.6 for algorithm HmacSHA1>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Will use default Mac for algorithm HmacSHA1>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Will use default Mac for algorithm MD5>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Using JCE Cipher: SunJCE version 1.6 for algorithm RC4>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Ignoring not supported JCE Mac: SunJCE version 1.6 for algorithm HmacMD5>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000 <BEA-000000> <Will use default Mac for algorithm HmacMD5>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Ignoring not supported JCE Mac: SunJCE version 1.6 for algorithm HmacSHA1>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <Will use default Mac for algorithm HmacSHA1>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <write HANDSHAKE, offset = 0, length = 16>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <isMuxerActivated: false>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <13293943 SSL3/TLS MAC>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <13293943 received ALERT>
<Jun 30, 2011 7:59:41 PM IST> <Debug> <SecuritySSL> <BEA-000000> <NEW ALERT with Severity: FATAL, Type: 40
java.lang.Exception: New alert stack
at com.certicom.tls.record.alert.Alert.<init>(Unknown Source)
at com.certicom.tls.record.alert.AlertHandler.handleAlertMessages(Unknown Source)
at com.certicom.tls.record.MessageInterpreter.interpretContent(Unknown Source)
at com.certicom.tls.record.MessageInterpreter.decryptMessage(Unknown Source)
at com.certicom.tls.record.ReadHandler.processRecord(Unknown Source)
at com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
at com.certicom.tls.record.ReadHandler.readUntilHandshakeComplete(Unknown Source)
at com.certicom.tls.interfaceimpl.TLSConnectionImpl.completeHandshake(Unknown Source)
at com.certicom.tls.record.WriteHandler.write(Unknown Source)
at com.certicom.io.OutputSSLIOStreamWrapper.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at weblogic.rjvm.t3.MuxableSocketT3.connect(MuxableSocketT3.java:396)
at weblogic.rjvm.t3.ConnectionFactoryT3S.createConnection(ConnectionFactoryT3S.java:37)
at weblogic.rjvm.ConnectionManager.createConnection(ConnectionManager.java:1773)
at weblogic.rjvm.ConnectionManager.findOrCreateConnection(ConnectionManager.java:1416)
at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:437)
at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:396)
at weblogic.rjvm.RJVMImpl.ensureConnectionEstablished(RJVMImpl.java:303)
at weblogic.rjvm.RJVMImpl.getOutputStream(RJVMImpl.java:347)
at weblogic.rjvm.RJVMImpl.getRequestStreamInternal(RJVMImpl.java:609)
at weblogic.rjvm.RJVMImpl.getRequestStream(RJVMImpl.java:577)
at weblogic.rjvm.RJVMImpl.getOutboundRequest(RJVMImpl.java:797)
at weblogic.rmi.internal.BasicRemoteRef.getOutboundRequest(BasicRemoteRef.java:157)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:341)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.jccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.db.oraclecase.loadbalancing.common.EJBMethodWrapper.execute(Unknown Source)
at com.db.oraclecase.loadbalancing.web.CaseBean.callEjbs(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.el.parser.AstValue.invoke(AstValue.java:157)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:283)
at com.sun.faces.faceletonListenerImpl.processAction(ActionListenerImpl.java:102)
at javax.faces.component.UICommand.broadcast(UICommand.java:315)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:775)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1267)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:82)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:312)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3592)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2202)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2108)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1432)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)


On BL servers you can see exception as below.

####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <1800872 received HANDSHAKE>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <HANDSHAKEMESSAGE: Certificate>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156nel>> <> <> <1308930674156> <BEA-000000> <Required peer certificates not supplied by peer>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <weblogic user specified trustmanager validation status 4>
####<Jun 24, 2011 9:21:14 PM IST> <Warning> <Security> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-090508> <Certificate chain received from localhost - 127.0.0.1 was incomplete.>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for qIVE] ExecuteThread: '0' for queue:
'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <User defined JSSE trustmanagers not allowed to override>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <SSLTrustValidator returns: 68>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <Trust failure (68): CERT_CHAIN_INCOMPLETE>
####<Jun 24, 2011 9:21:14 PM IST> <Debug> <SecuritySSL> <server> <MS1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1308930674156> <BEA-000000> <NEW ALERT with Severity: FATAL, Type: 40
java.lang.Exception: New alert stack
at com.certicom.tls.record.alert.Alert.<init>(Unknown Source)
at com.certicom.tls.record.handshake.HandshakeHandler.fireAlert(Unknown Source)
at com.certicom.tls.record.handshake.ServerStateSentHelloDone.handle(Unknown Source)
at com.certicom.tls.record.handshake.HandshakeHandler.handleHandshakeMessage(Unknown Source)
at com.certicom.tls.record.handshake.HandshakeHandler.handleHandshakeMessages(Unknown Source)
at com.certicom.tls.record.MessageInterpreter.interpretContent(Unknown Source)
at com.certicom.tls.record.MessageInterpreter.decryptMessage(Unknown Source)
at com.certicom.tls.record.ReadHandler.processRecord(Unknown Source)
at com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
at com.certicom.tls.record.ReadHandler.readUntilHandshakeComplete(Unknown Source)
at com.certicom.tls.interfaceimpl.TLSConnectionImpl.completeHandshake(Unknown Source)
at javax.net.ssl.impl.SSLSocketImpl.startHandshake(Unknown Source)
at weblogic.server.channels.DynamicSSLListenThread$1.run(DynamicSSLListenThread.java:130)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)


The exception only occurs when only one of the backend BL server is restarted.

Even these parameters were added and tested, but still the server throws the same exception (as above):

-Dweblogic.t3s.setQOSOnStub="102" -Dweblogic.t3.setQOSOnStub="102"

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms