OID 11g Takes a Long Time (Hours) to Start and then Does Not Respond: Can't Contact LDAP Server (Doc ID 2104886.1)

Last updated on OCTOBER 01, 2019

Applies to:

Oracle Internet Directory - Version 11.1.1 and later
Information in this document applies to any platform.

Symptoms

Oracle Internet Directory (OID) 11g, e.g., 11.1.1.6.0 version.

There is no load balancer or firewall in between OID and its Database (DB).

The groups are not too large, but there are over 135K groups, and over 2000 of them are privilege groups.

The oid is not starting when using opmnctl startall:

$ opmnctl startall
opmnctl startall: starting opmn and all managed processes...
================================================================================
opmn id=<OID_HOSTNAME>:<OPMN_PORT>
Response: 2 of 3 processes started.

ias-instance id=oid_prod_instance
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ias-component/process-type/process-set:
  oid1/oidmon/OID/

Error
--> Process (index=1,uid=<UID>,pid=<PID>)
  time out while waiting for a managed process to start
  Log:
  /u01/products/middleware/11g/oid/diagnostics/logs/OID/oid1/console~OID~1.log

It takes around 1-2 hours to bring up the OID processes, i.e., for it to return the prompt back from an oidctl command. And then OID does not respond to ldapbinds; it returns "can't contact ldap server" error.

OID dispatcher oidldapd01-0000.log shows:

[2016-01-20T11:44:31-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid: <PID>] [tid: <TID>] DispatcherListener: ERROR : gslsflAcceptConnAndSend : No serever process availbale to dispatch TCP Connection

[2016-01-20T11:47:14-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid: <TID>] DispatcherListener: sgslun6xAccept: Accept succedded on new communication endpoint

[2016-01-20T11:47:14-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] DispatcherListener: INFO : gslsflgGetServer : Entry

[2016-01-20T11:47:14-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid: <TID>] DispatcherListener: INFO: gslsflgGetServer : Exit
[2016-01-20T11:47:14-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid: <TID>] DispatcherListener: ERROR : gslsflAcceptConnAndSend : No serever process availbale to dispatch TCP Connection

[2016-01-20T11:47:14-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid: <TID>] DispatcherListener: sgslun6xAccept: Accept succedded on new communication endpoint

[2016-01-20T11:47:14-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid: <TID>] DispatcherListener: INFO : gslsflgGetServer : Entry

OID server logs may show lots of group activities such as:

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Subtree Search Completed with: 100

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: [gsldfd_FetchSubTreeObjs]Re-using prepared Statement

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Bind variable: :0 = "cn=group1,cn=groups,dc=<COMPANY>,dc=com"

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Bind variable: :1 = "cn=group1,cn=groups,dc=<COMPANY>,dc=com"

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Subtree Search Completed with: 100

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: [gsldfd_FetchSubTreeObjs]Re-using prepared Statement

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Bind variable: :0 = "cn=group2,cn=groups,dc=<COMPANY>,dc=com"

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Bind variable: :1 = "cn=group2,cn=groups,dc=<COMPANY>,dc=com"

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: Subtree Search Completed with: 100

[2016-01-20T01:00:44-05:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: <OID_HOSTNAME>] [pid:<PID>] [tid:<TID>] Main:: [gsldfd_FetchSubTreeObjs]Re-using prepared Statement

OID stack files may be generated, for example:

Dumping stack trace for OIDLDAPD (server) ...
calling caller argument values in hex
location (? means dubious value)
--------------------------------------------------------------------
gslusdsDumpStack() __restore_rt() arg#1: ???? value=C1899C38 (ptr)
arg#2: ???? value=C1896010 (ptr)
arg#3: ???? value=C1897528 (ptr)
arg#4: ???? value=C18975F0 (ptr)
arg#5: ???? value=C1899C38 (ptr)
arg#6: ???? value=00000001 (ptr)

__restore_rt() read()+16 arg#1: ???? value=00000014 (ptr)
arg#2: ???? value=0147EA06 (ptr)
arg#3: ???? value=00002010 (ptr)
arg#4: ???? value=FFFFFFFF (ptr)
arg#5: ???? value=00000008 (ptr)
arg#6: ???? value=00000000 (ptr)

read() snttread()+14 arg#1: ???? value=00000014 (ptr)
arg#2: ???? value=0147EA06 (ptr)
arg#3: ???? value=00002010 (ptr)
arg#4: ???? value=FFFFFFFF (ptr)
arg#5: ???? value=00000008 (ptr)
arg#6: ???? value=00000000 (ptr)

snttread() nttfprd()+157 arg#1: ???? value=00000014 (ptr)
arg#2: ???? value=0147EA06 (ptr)
arg#3: ???? value=00002010 (ptr)
arg#4: ???? value=FFFFFFFF (ptr)
arg#5: ???? value=00000008 (ptr)
arg#6: ???? value=00000000 (ptr)

...<etc>...

Tried disabling group cache, and all cache, but the same problem remains.

Cause

	To view full details, sign in with your My Oracle Support account.
	Don't have a My Oracle Support account? Click to get started!

In this Document

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.

OID 11g Takes a Long Time (Hours) to Start and then Does Not Respond: Can't Contact LDAP Server (Doc ID 2104886.1)

Applies to:

Symptoms

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!