OVD 11g HA Instances Stop Responding and Solaris Capped Zones Hung. Must Reboot the Zones to Gain Access and Recover.
(Doc ID 1952777.1)
Last updated on MARCH 30, 2022
Applies to:
Oracle Virtual Directory - Version 11.1.1.0 and laterOracle Solaris on SPARC (64-bit)
Symptoms
Oracle Virtual Directory (OVD) 11g, e.g. 11.1.1.7.0 version.
Twelve (12) Solaris 11.2 capped zones: 4 CPUs with 8 GB of memory
Two OVD instances stop responding to LDAP requests at the same time. OVD hung and Solaris capped zone hung at 50%. It is not possible to kill the OVD processes and also unable to manage any other processes. No traces can be obtained because the only means to access and recover is to reboot the zones.
Sequence of events:
• The unix LDAP authentication fails for all LDAP clients
• Verification if 2 OVD zones are up: OK
• Connection to zones and attempts of requests to 2 OVDs: No response.
• Verification of servers ovd: No response
• Attempts to stop normally the instances OVD: OVD processes cannot be stopped
• Kill and kill -9 of OVD process: No success
• Attempt to restart for OVD1: No success (blocked on shutdown, seems due to OVD process)
• Attempt to restart for OVD2: sane as above
• Halt of the two zones
• Restart of zones
• OVD starts and is back to work
OVD's are hanging every two days.
After setting OVD debug at trace:32, was able to obtain one set of logs. The diagnostic.log shows:
[2014-11-12T23:17:28.417+01:00] [octetstring] [TRACE] [] [com.octetstring.vde.DoSManager] [tid: xx] [ecid: <ECID>] [SRC_CLASS: com.octetstring.vde.util.VDELogger] [SRC_METHOD: debug] Current operations per connection (/<IP_ADDRESS>, 0/0).
[2014-11-12T23:17:39.472+01:00] [octetstring] [NOTIFICATION] [OVD-60053] [com.octetstring.vde.ConnectionHandler] [tid: xxxxx] [ecid: <ECID>] Error accepting connection from server socket: Remote host closed connection during handshake.[[
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1230)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1214)
at com.octetstring.vde.ConnectionHandler.initializeConnection(ConnectionHandler.java:174)
at com.octetstring.vde.ConnectionHandler.run(ConnectionHandler.java:218)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
... 8 more
]]
[2014-11-12T23:18:03.417+01:00] [octetstring] [ERROR] [OVD-60055] [com.octetstring.vde.frontend.LDAP] [tid: xx] [ecid: <ECID>] Out of memory error: Printing stack trace.[[
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.addEntry(HashMap.java:753)
at java.util.HashMap.put(HashMap.java:385)
at java.util.ListResourceBundle.loadLookup(ListResourceBundle.java:180)
at java.util.ListResourceBundle.handleKeySet(ListResourceBundle.java:144)
at java.util.ResourceBundle.containsKey(ResourceBundle.java:1669)
at oracle.core.ojdl.logging.ODLFormatter.getString(ODLFormatter.java:925)
at oracle.core.ojdl.logging.ODLFormatter.toLogMessage(ODLFormatter.java:486)
at oracle.core.ojdl.logging.ODLFormatter.toLogMessage(ODLFormatter.java:229)
at oracle.core.ojdl.logging.ODLHandler.publish(ODLHandler.java:764)
at oracle.core.ojdl.logging.ODLHandler.publish(ODLHandler.java:738)
at oracle.core.ojdl.logging.ODLLogger.doLog(ODLLogger.java:878)
at oracle.core.ojdl.logging.ODLLogger.logRec(ODLLogger.java:914)
at oracle.core.ojdl.logging.ODLLogger.logRec(ODLLogger.java:901)
at oracle.core.ojdl.logging.ODLLogger.log(ODLLogger.java:666)
at com.octetstring.vde.util.VDELogger.info(VDELogger.java:219)
at com.octetstring.vde.frontend.LDAP.run(LDAP.java:319)
at java.lang.Thread.run(Thread.java:662)
]]
[2014-11-12T23:17:35.790+01:00] [octetstring] [TRACE] [] [com.octetstring.vde.Connection] [tid: xxxxx] [ecid:<ECID>] [SRC_CLASS: com.octetstring.vde.util.VDELogger] [SRC_METHOD: debug] ConnID:xxxx Added mesgID:1 to Q, QSize:1
[2014-11-12T23:18:54.117+01:00] [octetstring] [NOTIFICATION] [OVD-60053] [com.octetstring.vde.ConnectionHandler] [tid: xxxxx] [ecid: <ECID>] Error accepting connection from server socket: Remote host closed connection during handshake.[[
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1230)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1214)
at com.octetstring.vde.ConnectionHandler.initializeConnection(ConnectionHandler.java:174)
at com.octetstring.vde.ConnectionHandler.run(ConnectionHandler.java:218)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
... 8 more
]]
...<end>...
Reviewed: OVD Diagnostic Logs Show SSL Handshake Errors and "Remote host closed connection during handshake" Document 1268605.1
And:
JVM tuning in OVD Dev's blog
OVD 11g Search Fails To Returns All Results / Search Halts And Throws Operations Error / OVD Log Shows java.lang.OutOfMemoryError Exception Document 1088258.1
However, all tuning has already been applied.
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |
| References |