OVD Hangs, Vde.log Error "LDAP Error 1 : Error Communicating with the SQL Database" "Transmission error : Socket closed" "java.lang.OutOfMemoryError" (Doc ID 739160.1)

Last updated on MARCH 08, 2017

Applies to:

Oracle Virtual Directory - Version 10.1.4.2 and later
Information in this document applies to any platform.

Symptoms

Oracle Access Manager (OAM) using Oracle Virtual Directory (OVD) 10.1.4.2.0 (high availability configuration) with AD and DB adapters.

Scenario:

About 60,000 rows in a join created. OVD brings back about 70,000 records from the join. Initially, when there were only about 7,000 records, the problem did not occur.  Servers have plenty of memory.

Also, a development mirror system has same amount of data and does not have the problem, but does not have the same amount of connections or traffic as the production environment experiencing the problem.


Problem:

Losing contact with the database, while the database is functional.  OVD vde.log errors include:

2008-07-25 23:55:33,501 ERROR - Joiner: Could Not Load Joined Entry Set: LDAP Error 1 : Error Communicating with the SQL Database: No database connection available in cache to process the request-null WorkThread# 4

OVD stops responding and all that can be done to recover is to restart it. This happens about every few days.


Diagnostics:

Tried enabling quota enforcement and then implemented the inactive connections timeout feature (60 mins at first, also tried down to 10 minutes), but then started seeing timeout errors:

2008-07-31 15:36:02,612 ERROR - SearchOperation: Critical error while performing SEARCH
java.lang.OutOfMemoryError: Java heap space WorkThread# 0
2008-07-31 15:36:41,271 ERROR - SearchOperation: Critical error while performing SEARCH
java.lang.OutOfMemoryError: Java heap space WorkThread# 9

Checked against <Document 465627.1> and the vde_start.sh was already with the suggested settings.

Tried the latest merge Patch for OVD 10.1.4.2.0 - Merge Patch #21 <Patch 7272441> (which was the latest merge patch available at the time), but the same errors continued.

Turned on debug level on OVD. OVD stopped working but the log files no longer reported the same SQL connection errors, but did have following socket errors (which followed the SQL error before):

2008-08-18 09:05:30,495 ERROR - MessageHandler: Transmission error : Socket closed WorkThread# 10
2008-08-18 09:05:30,482 ERROR - MessageHandler: Transmission error : Socket closed WorkThread# 5
2008-08-18 09:05:30,502 ERROR - JNDIResultBuffer: Exception in JNDI Buffer thread:null Ldap Proxy
Buffer-158
2008-08-18 09:05:30,468 ERROR - MessageHandler: Transmission error : Socket closed WorkThread# 7

Tried increasing the maximum DB connections in the DB adapter to 100 using OVD Manager. Then monitored the number of active connections used for each DB adapter from OVD Manager -> Status Tab in Adapters textfield. Real-time info of Active connections used, Idle connections in the pool, etc., are displayed in this UI.  After that, not getting the connection error but still running out of memory, and at times had to restart OVD twice in one day so backed the number of DB connections down to 50.

(All the while, trying to collect debug level logs could generated ~40G of logs in less than one day without being able to reproduce the problem, and then the system would run out of disk space.)

 

Finally was able to narrow down the problem to one specific search - the problem happens when searches contain the following filter:

(&(objectclass=inetorgperson)(&(employeeNumber=<# ie 101572>)(|(obUserAccountControl=ACTIVATED)(!(obUserAccountControl=*)))))


 

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms