OID11g: OID Server / oidldapd Process Crashing And Restarting Automatically. (Doc ID 2191452.1)

Last updated on MARCH 14, 2017

Applies to:

Oracle Internet Directory - Version 11.1.1.5.0 and later
Information in this document applies to any platform.

Symptoms

OID 11.1.1.5 with DB 11.2.0.4

+. Two Node Data Center 1, with each node having two OID instances, with Common RAC DB.
+. Two Node Data Center 2, with each node having two OID instances.with Common RAC DB
+. Replication Enabled between DC-1 and DC-2 via one of OID node.
+. Note that many applicationw around 100+ were using OID, including some custom applications.


Observed that OID process is getting crashed and restarted automatically.

+. Many stack dumps oidldapd_stack*.dmp are getting created.

Example :
oidldapd_stack00_19838.dmp

Dumping stack trace for OIDLDAPD (server) ...
calling caller argument values in hex
location (? means dubious value)
--------------------------------------------------------------------
gslusdsDumpStack() __restore_rt() arg#1: ???? value=8AC6AA48 (ptr)
arg#2: ???? value=8AC66E20 (ptr)
arg#3: ???? value=8AC68338 (ptr)
arg#4: ???? value=8AC68400 (ptr)
arg#5: ???? value=8AC6AA48 (ptr)
arg#6: ???? value=00000001 (ptr)

__restore_rt() pthread_mutex_lock() arg#1: ???? value=00000000 (ptr)
arg#2: ???? value=8AC6F960 (ptr)
arg#3: ???? value=00000048 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

pthread_mutex_lock()SltsPrUnlock()+24 arg#1: ???? value=00000000 (ptr)
arg#2: ???? value=8AC6F960 (ptr)
arg#3: ???? value=00000048 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

SltsPrUnlock() gslfbidbDoBind()+968 arg#1: ???? value=00000000 (ptr)
arg#2: ???? value=8AC6F960 (ptr)
arg#3: ???? value=00000048 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

gslfbidbDoBind() gslarswWorker()+3630 arg#1: ???? value=0677D6A0 (ptr)
arg#2: ???? value=067636B0 (ptr)
arg#3: ???? value=942F9F10 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

gslarswWorker() start_thread()+221 arg#1: ???? value=0E54F120 (ptr)
arg#2: ???? value=067636B0 (ptr)
arg#3: ???? value=06B41510 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

start_thread() clone()+109 arg#1: ???? value=8AC72940 (ptr)
arg#2: ???? value=067636B0 (ptr)
arg#3: ???? value=06B41510 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

clone() 0000000000000000 arg#1: ???? value=8AC72940 (ptr)
arg#2: ???? value=067636B0 (ptr)
arg#3: ???? value=06B41510 (ptr)
arg#4: ???? value=06763AE0 (ptr)
arg#5: ???? value=02020060 (ptr)
arg#6: ???? value=00000000 (ptr)

Dumping stack trace of other threads ...
Signaling thread 1 to dump stack
calling caller argument values in hex
location (? means dubious value)
--------------------------------------------------------------------
gslusdsDumpStack() __restore_rt() arg#1: ???? value=88F91AE8 (ptr)
arg#2: ???? value=88F8DEC0 (ptr)
arg#3: ???? value=88F8F3D8 (ptr)
arg#4: ???? value=88F8F4A0 (ptr)
arg#5: ???? value=88F91AE8 (ptr)
arg#6: ???? value=00000001 (ptr)
....
.....

CONNDN: uid=raju,ou=access seekers,ou=abc,dc=abco,dc=com,dc=in
OPTAG: BIND
METHOD: SIMPLE
BINDDN: uid=vishwa,ou=access
seekers,ou=abc,dc=abco,dc=com,dc=in
ENTRYDN: uid=vishwa,ou=access
seekers,ou=abc,dc=abco,dc=com,dc=in
NORMDN: uid=vishwa,ou=access
seekers,ou=abc,dc=abco,dc=com,dc=in
ENTRYID: 710862
EntrySize: 9184
Dumping attributes ...

ATTRNAME: oblastsuccessfullogin
NUMVALUES: 1
AttrSize: 168
ATTRKIND: u
ATTRVER: 5

ATTRNAME: oblogintrycount
====================

+. And some stack dump has below as part of it...Note that here complete searchfilter is not logged in stack dump file.

....
CONNDN: cn=svc_emm_ldap_prod,ou=service accounts,dc=abco,dc=com,dc=in
OPTAG: SEARCH
SCOPE: SUBTREE
BASEDN: ou=abc,dc=abco,dc=com,dc=in
FILTER:
(&(uid=*)(|(orclguid=de101e6fa42a0657e0409bcc0185778f)(orclguid=c7f9546fd88f59
efe0409bcc01856e72)(orclguid=c7f9546fd8f759efe0409bcc01856e72)(orclguid=23e248
ac4c2b0012e0509bcc01851c43)(orclguid=c96e2f0c571e81a8e0409bcc0185450c)(orclgui
d=0f2fd4486f7935d9e0509bcc01856870)(orclguid=ce75305d53b84ec4e0409bcc01850849)
(orclguid=da2c93dca7584693e0409bcc018545f8)(orclguid=10e58be448ca3527e0509bcc0
185279c)(orclguid=02caa9be14437310e0509bcc018555f1)(orclguid=c7f9546fd77859efe
0409bcc01856e72)(orclguid=ed42b8c1669a782ce0409bcc018518de)(orclguid=c7f9546fd
6f759efe0409bcc01856e72)(orclguid=c7f9546fd73c59efe0409bcc01856e72)(orclguid=f
b90ce4cef477656e0409bcc01851e4f)(orclguid=dd01b3dd4d31c531e0409bcc01854380)(or
clguid=09c9c2e87068bc5ee0509bcc0185368a)(orclguid=c7f9546ff71c59efe0409bcc0185
6e72)(orclguid=c7f9546fe13059efe0409bcc01856e72)(orclguid=c7f9546ff7be59efe040
9bcc01856e72)(orclguid=c7f9546fedff59efe0409bcc01856e72)(orclguid=de101e70945f
4a40e0409bcc01857791)(orclguid=c7f9546ff2a459efe0409bcc01856e72)(orclguid=c7f9
546fe2
ReqdAttrs: orclguid orclisenabled

Dumping connection stat for dispather thread : 3

....

 

OID ldap server log may show, for example:

...<snip>...
[2017-02-25T18:06:36+01:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost] [pid: 3611] [tid: 2] DispatcherListener: sgsluscSendPort: sendmsg failed, OS ERROR = 32

[2017-02-25T18:06:36+01:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost] [pid: 3611] [tid: 1] ProcessDispatcher: sgsluscSendPort: sendmsg failed, OS ERROR = 32

[2017-02-25T18:06:36+01:00] [OID] [ERROR:8] [23124] [OIDLDAPD] [host: oidhost] [pid: 3611] [tid: 1] ProcessDispatcher: The server with Process ID = 3717 is not running.

[2017-02-25T18:06:37+01:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost] [pid: 3611] [tid: 1] ProcessDispatcher: Starting OIDLDAP Server,PID=12169

[2017-02-27T05:26:50+01:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost] [pid: 3611] [tid: 2] DispatcherListener: WARNING : DispatcherListener : Shutting down
...<snip>...

 

Changes

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms