OUD11g - The Replication Server Shuts Down Due to "llegalStateException: Database closed while still referenced by other handles"

(Doc ID 2319721.1)

Last updated on OCTOBER 25, 2017

Applies to:

Oracle Unified Directory - Version 11.1.2.2.0 to 11.1.2.2.3 [Release 11g]
Information in this document applies to any platform.

Symptoms

On : 11.1.2.2.0 version, configuration

Replication server suddenly stops. Seeing these messages in logs -

[10/Oct/2017:21:53:36 -0700] category=SYNC severity=SEVERE_ERROR msgID=-1 msg=Error during the replication server database trimming process. The replication ser
ver is going to shutdown(JE 5.0.86) Lock expired. Locker 1812336796 445944789_Replication DraftCN db _Txn: waited for lock on database=draftcndb LockAddr:179786
1199 LSN=0x168/0x8689e5 type=WRITE grant=WAIT_PROMOTION timeoutMillis=3600000 startTime=1507694016372 endTime=1507697616373
Owners: [<LockInfo locker="1143199497 -1_Worker Thread 15_ThreadLocker" type="READ"/>, <LockInfo locker="1812336796 445944789_Replication DraftCN db _Txn" type=
"READ"/>]
Waiters: []
Transaction 1812336796 445944789_Replication DraftCN db _Txn owns LockAddr:1797861199 <LockInfo locker="1812336796 445944789_Replication DraftCN db _Txn" type="
READ"/>
Transaction 1812336796 445944789_Replication DraftCN db _Txn waits for LockAddr:1797861199
(LockManager.java:664 LockManager.java:623 SyncedLockManager.java:97 LockManager.java:390 LockManager.java:276 Txn.java:510 Locker.java:443 CursorImpl.java:263
6 CursorImpl.java:1353 Cursor.java:2095 Cursor.java:2059 Cursor.java:696 DraftCNDB.java:731 DraftCNDbHandler.java:477 DraftCNDbHandler.java:383 DraftCNDbHandler
.java:337 Thread.java:744)
[10/Oct/2017:21:53:36 -0700] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.opends.server.replication.server.ReplicationServer (alert type org.opends.server.replication.ReplicationServerDown, alert ID -1):  Error during the replication server database trimming process. The replication server is going to shutdown(JE 5.0.86) Lock expired. Locker 1812336796 445944789_Replication DraftCN db _Txn: waited for lock on database=draftcndb LockAddr:1797861199 LSN=0x168/0x8689e5 type=WRITE grant=WAIT_PROMOTION timeoutMillis=3600000 startTime=1507694016372 endTime=1507697616373
Owners: [<LockInfo locker="1143199497 -1_Worker Thread 15_ThreadLocker" type="READ"/>, <LockInfo locker="1812336796 445944789_Replication DraftCN db _Txn" type="READ"/>]
Waiters: []
Transaction 1812336796 445944789_Replication DraftCN db _Txn owns LockAddr:1797861199 <LockInfo locker="1812336796 445944789_Replication DraftCN db _Txn" type="READ"/>
Transaction 1812336796 445944789_Replication DraftCN db _Txn waits for LockAddr:1797861199
(LockManager.java:664 LockManager.java:623 SyncedLockManager.java:97 LockManager.java:390 LockManager.java:276 Txn.java:510 Locker.java:443 CursorImpl.java:2636 CursorImpl.java:1353 Cursor.java:2095 Cursor.java:2059 Cursor.java:696 DraftCNDB.java:731 DraftCNDbHandler.java:477 DraftCNDbHandler.java:383 DraftCNDbHandler.java:337 Thread.java:744)
[10/Oct/2017:21:53:36 -0700] category=SYNC severity=NOTICE msgID=15138879 msg=Replication server <host>/<ip>:<port>1940 has properly disconnected for cn=schema in local server id 5855. Trying to reconnect to a suitable replication server
[10/Oct/2017:21:53:36 -0700] category=SYNC severity=NOTICE msgID=15138839 msg=Could not connect to any replication server on suffix cn=schema among the following RS candidates {}, retrying...
[10/Oct/2017:21:53:37 -0700] category=SYNC severity=NOTICE msgID=15138879 msg=Replication server <host>/<ip>:<port> 1940 has properly disconnected for cn=admin data in local server id 9483. Trying to reconnect to a suitable replication server
[10/Oct/2017:21:53:37 -0700] category=SYNC severity=NOTICE msgID=15138839 msg=Could not connect to any replication server on suffix cn=admin data among the following RS candidates {}, retrying...
[10/Oct/2017:21:53:38 -0700] category=SYNC severity=NOTICE msgID=15138879 msg=Replication server <host>/<ip>:<port> 1940 has properly disconnected for dc=company,dc=division,dc=com in local server id 887. Trying to reconnect to a suitable replication server
[10/Oct/2017:21:53:38 -0700] category=SYNC severity=NOTICE msgID=15138839 msg=Could not connect to any replication server on suffix dc=company,dc=division,dc=com among the following RS candidates {}, retrying...
[10/Oct/2017:21:53:39 -0700] category=CORE severity=FATAL_ERROR msgID=327820 msg=An uncaught exception during processing for thread Replication DraftCN db  has caused it to terminate abnormally.  The stack trace for that exception is:  java.lang.IllegalStateException: Unclosed Database: draftcnlimitsdb
Unclosed Database: draftcndb

While closing Database draftcndb encountered exception: java.lang.IllegalStateException: Database closed while still referenced by other handles. 1 open cursors.
       at com.sleepycat.je.Database.closeInternalWork(Database.java:646)
       at com.sleepycat.je.Database.closeInternal(Database.java:508)
       at com.sleepycat.je.Database.close(Database.java:438)
       at com.sleepycat.je.Environment.checkOpenDbs(Environment.java:518)
       at com.sleepycat.je.Environment.checkForCloseErrors(Environment.java:414)
       at com.sleepycat.je.Environment.close(Environment.java:375)
       at org.opends.server.replication.server.ReplicationDbEnv.shutdown(ReplicationDbEnv.java:688)
       at org.opends.server.replication.server.ReplicationServer.shutdown(ReplicationServer.java:827)
       at org.opends.server.replication.server.DraftCNDbHandler.run(DraftCNDbHandler.java:358)
       at java.lang.Thread.run(Thread.java:744)

Databases left open: 2

 com.sleepycat.je.Environment.close(Environment.java:390)
 org.opends.server.replication.server.ReplicationDbEnv.shutdown(ReplicationDbEnv.java:688)
 org.opends.server.replication.server.ReplicationServer.shutdown(ReplicationServer.java:827)
 org.opends.server.replication.server.DraftCNDbHandler.run(DraftCNDbHandler.java:358)
 java.lang.Thread.run(Thread.java:744)

[10/Oct/2017:21:53:39 -0700] category=CORE severity=NOTICE msgID=458891 msg=The Directory Server has sent an alert notification generated by class org.opends.server.api.DirectoryThread (alert type org.opends.server.UncaughtException, alert ID 327820):  An uncaught exception during processing for thread Replication DraftCN db  has caused it to terminate abnormally.  The stack trace for that exception is:  java.lang.IllegalStateException: Unclosed Database: draftcnlimitsdb
Unclosed Database: draftcndb

While closing Database draftcndb encountered exception: java.lang.IllegalStateException: Database closed while still referenced by other handles. 1 open cursors.
       at com.sleepycat.je.Database.closeInternalWork(Database.java:646)
       at com.sleepycat.je.Database.closeInternal(Database.java:508)
       at com.sleepycat.je.Database.close(Database.java:438)
       at com.sleepycat.je.Environment.checkOpenDbs(Environment.java:518)
       at com.sleepycat.je.Environment.checkForCloseErrors(Environment.java:414)
       at com.sleepycat.je.Environment.close(Environment.java:375)
       at org.opends.server.replication.server.ReplicationDbEnv.shutdown(ReplicationDbEnv.java:688)
       at org.opends.server.replication.server.ReplicationServer.shutdown(ReplicationServer.java:827)
       at org.opends.server.replication.server.DraftCNDbHandler.run(DraftCNDbHandler.java:358)
       at java.lang.Thread.run(Thread.java:744)

Databases left open: 2

 com.sleepycat.je.Environment.close(Environment.java:390)
 org.opends.server.replication.server.ReplicationDbEnv.shutdown(ReplicationDbEnv.java:688)
 org.opends.server.replication.server.ReplicationServer.shutdown(ReplicationServer.java:827)
 org.opends.server.replication.server.DraftCNDbHandler.run(DraftCNDbHandler.java:358)
 java.lang.Thread.run(Thread.java:744)

 

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms