OUD11g / 12c - Server Instance Consumes 100% CPU, Becomes Unresponsive and Does Not Accept Connections
(Doc ID 2312410.1)
Last updated on MAY 09, 2023
Applies to:
Oracle Unified Directory - Version 11.1.2.2.0 and laterInformation in this document applies to any platform.
Symptoms
OUD Server is consuming 100% CPU, is unresponsive and not accepting connections.
OUD Server ran for an hour exceeding 100% of CPU usage, then dropped down to approximately 2% CPU usage but still would not accept connections.
The following type of messages can be seen in the logs:
[14/Sep/2017:11:02:16 -0600] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.opends.server.replication.server.ReplicationServer (alert type org.opends.server.replication.ServerMissingTrimmedChanges, alert ID 14942419): Alert in replication server app_1:<REPL_ID> with server id <REPL_SRVR-1>: in domain dc=<SUFFIX_DN>, the remote server app_2:<REPL_ID> with server id <SRVR_ID-1> is missing some changes from directory server id <SRVR_ID-2> that have been purged from the changes database because they were too old. This may happen if the remote server has been stopped for too much time or if it is very late replaying the changes. If the remote server is a directory server, its backend should be re-initialized. If the remote server is another replication server, potentially many directory servers may lack some changes and their backend should be re-initialized
[14/Sep/2017:11:02:16 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> o=<INT_USER>,dc=<SUFFIX_DN> has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:02:16 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> cn=schema has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for dc=<SUFFIX_DN> in local server id <SRVR_ID-3>
[14/Sep/2017:11:02:16 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for o=<INT_USER>,dc=<SUFFIX_DN> in local server id <SRVR_ID-4>
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for cn=schema in local server id <SRVR_ID-5>
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138878 msg=Replication is up and running for domain cn=admin data with replication server id <SRVR_ID-1> app_2/<IP>:<REPL_ID> - local server id is <SRVR_ID-6> - data generation is 95401
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942386 msg=Directory server <SRVR_ID-6> was attempting to connect to replication server <REPL_SRVR-1> but has disconnected in handshake phase
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942386 msg=Directory server <SRVR_ID-6> was attempting to connect to replication server <REPL_SRVR-1> but has disconnected in handshake phase
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138921 msg=SSL connection attempt from <host_name> (<IP>) failed: Remote host closed connection during handshake
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138921 msg=SSL connection attempt from <host_name> (<IP>) failed: Remote host closed connection during handshake
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138921 msg=SSL connection attempt from <host_name> (<IP>) failed: Remote host closed connection during handshake
[14/Sep/2017:11:02:52 -0600] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.opends.server.replication.server.ReplicationServer (alert type org.opends.server.replication.ServerMissingTrimmedChanges, alert ID 14942419): Alert in replication server app_1:<REPL_ID> with server id <REPL_SRVR-1>: in domain dc=<SUFFIX_DN>, the remote server app_2:<REPL_ID> with server id <SRVR_ID-1> is missing some changes from directory server id <SRVR_ID-2> that have been purged from the changes database because they were too old. This may happen if the remote server has been stopped for too much time or if it is very late replaying the changes. If the remote server is a directory server, its backend should be re-initialized. If the remote server is another replication server, potentially many directory servers may lack some changes and their backend should be re-initialized
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=NOTICE msgID=15138913 msg=Replication Heartbeat Monitor for RS <IP>:<REPL_ID> cn=schema in RS <SRVR_ID-1> is closing the session because it could not detect a heartbeat
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=NOTICE msgID=15138913 msg=Replication Heartbeat Monitor for RS <IP>:<REPL_ID> o=<INT_USER>,dc=<SUFFIX_DN> in RS <SRVR_ID-1> is closing the session because it could not detect a heartbeat
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for cn=admin data in local server id <SRVR_ID-6>
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> dc=<SUFFIX_DN> has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:03:26 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> cn=admin data has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:02:16 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> o=<INT_USER>,dc=<SUFFIX_DN> has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:02:16 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> cn=schema has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for dc=<SUFFIX_DN> in local server id <SRVR_ID-3>
[14/Sep/2017:11:02:16 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for o=<INT_USER>,dc=<SUFFIX_DN> in local server id <SRVR_ID-4>
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for cn=schema in local server id <SRVR_ID-5>
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138878 msg=Replication is up and running for domain cn=admin data with replication server id <SRVR_ID-1> app_2/<IP>:<REPL_ID> - local server id is <SRVR_ID-6> - data generation is 95401
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942386 msg=Directory server <SRVR_ID-6> was attempting to connect to replication server <REPL_SRVR-1> but has disconnected in handshake phase
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942386 msg=Directory server <SRVR_ID-6> was attempting to connect to replication server <REPL_SRVR-1> but has disconnected in handshake phase
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138921 msg=SSL connection attempt from <host_name> (<IP>) failed: Remote host closed connection during handshake
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138921 msg=SSL connection attempt from <host_name> (<IP>) failed: Remote host closed connection during handshake
[14/Sep/2017:11:02:51 -0600] category=SYNC severity=NOTICE msgID=15138921 msg=SSL connection attempt from <host_name> (<IP>) failed: Remote host closed connection during handshake
[14/Sep/2017:11:02:52 -0600] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.opends.server.replication.server.ReplicationServer (alert type org.opends.server.replication.ServerMissingTrimmedChanges, alert ID 14942419): Alert in replication server app_1:<REPL_ID> with server id <REPL_SRVR-1>: in domain dc=<SUFFIX_DN>, the remote server app_2:<REPL_ID> with server id <SRVR_ID-1> is missing some changes from directory server id <SRVR_ID-2> that have been purged from the changes database because they were too old. This may happen if the remote server has been stopped for too much time or if it is very late replaying the changes. If the remote server is a directory server, its backend should be re-initialized. If the remote server is another replication server, potentially many directory servers may lack some changes and their backend should be re-initialized
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=NOTICE msgID=15138913 msg=Replication Heartbeat Monitor for RS <IP>:<REPL_ID> cn=schema in RS <SRVR_ID-1> is closing the session because it could not detect a heartbeat
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=NOTICE msgID=15138913 msg=Replication Heartbeat Monitor for RS <IP>:<REPL_ID> o=<INT_USER>,dc=<SUFFIX_DN> in RS <SRVR_ID-1> is closing the session because it could not detect a heartbeat
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942388 msg=The connection to replication server app_2/<IP>:<REPL_ID> <SRVR_ID-1> has been unexpectedly dropped by the replication server for cn=admin data in local server id <SRVR_ID-6>
[14/Sep/2017:11:02:52 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> dc=<SUFFIX_DN> has badly disconnected from this replication server <REPL_SRVR-1>
[14/Sep/2017:11:03:26 -0600] category=SYNC severity=SEVERE_ERROR msgID=14942389 msg=Replication Server <SRVR_ID-1> app_2:<REPL_ID> cn=admin data has badly disconnected from this replication server <REPL_SRVR-1>
Changes
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Changes |
| Cause |
| Solution |