OID 11g Connection Timeouts, Crash or Hang - gsluslldConnect: Failed to connect droid01:3060 / OID Continuously Pings a Deprecated Host Leading to Crashes (Doc ID 1607479.1)

Last updated on DECEMBER 02, 2016

Applies to:

Oracle Internet Directory - Version 11.1.1.7.0 and later
Information in this document applies to any platform.

Symptoms

On : 11.1.1.7.0 version, OID LDAP Server

OID periodically becomes unresponsive , will not accept modifications, hangs or occasionally creates stack dumps and run-away log files. Symptoms would include client applications reporting "Can't contact LDAP server".

The following error occurs repeatedly in the oidmon-0000.log

[2013-10-22T11:18:30-05:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oidhost01] [pid: 7788] [tid: 0] Guardian: Warning: Node=droid01 is not responding .., last update Time=20130511020119Z
[2013-10-22T11:19:43-05:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oidhost01] [pid: 7788] [tid: 0] Guardian: Warning: Node=droid01 is not responding .., last update Time=20130511020119Z
[2013-10-22T11:20:59-05:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oidhost01] [pid: 7788] [tid: 0] Guardian: Warning: Node=droid01 is not responding .., last update Time=20130511020119Z

..
[2013-10-22T12:17:50-05:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oidhost01] [pid: 7788] [tid: 0] Guardian: gslsgldConnect: Failed to connect droid01:3060
[2013-10-22T12:17:50-05:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oidhost01] [pid: 7788] [tid: 0] Guardian: gslsgrnPing: Failed to connect to OID host:droid01 port:3060 ssl:0

 Occasionally OID may crash with a run-away log condition and produce stack dumps. For Example

oidldapd01s7281-0000.log
----------------------------------
gslusrnChkForNewNodes:Adding new node: host=oidhost02.oracle.com, ipaddr=, port=3060
gslusrnChkForNewNodes:Adding new node: host=droid01, ipaddr=, port=3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gsluslldConnect: Failed to connect droid01:3060
gslusrnChkConn2RemNode:Connect to OID on host:droid01 failed

 

oidldapd_stack00_07281.dmp

-----------------------------------------
calling               caller                argument values in hex
location                                     (? means dubious value)
--------------------------------------------------------------------
gslusdsDumpStack()  _ZN2os4Hpux15chained_handlerEiP9__siginfoPv()+560  
_ZN2os4Hpux15chained_handlerEiP9__siginfoPv()_ZN2os4Hpux22JVM_handle_hpux_signalEiP9__siginfoPvi()+960  
_ZN2os4Hpux22JVM_handle_hpux_signalEiP9__siginfoPvi()_ZN2os4Hpux13signalHandlerEiP9__siginfoPv()+128  
_ZN2os4Hpux13signalHandlerEiP9__siginfoPv()<kernel>  
<kernel>            _poll_sys()+48          
_poll_sys()         _poll()+224              
_poll()             sgslufWriteEnabled()+192  
sgslufWriteEnabled()sgslunvWriteEnabled()+384  
sgslunvWriteEnabled()sgslunnNewCtn()+2416    
sgslunnNewCtn()     gslcoic_ConnectToHost()+1168  
gslcoic_ConnectToHost()gslcopc_OpenLdapConnection()+1456  
gslcopc_OpenLdapConnection()gslconn_NewConnection()+528  
gslconn_NewConnection()gslcopd_LdapOpenDefConn()+304  
gslcopd_LdapOpenDefConn()gslcopo_LdapOpen()+208  
gslcopo_LdapOpen()  ldap_open()+176          
ldap_open()         gsluslldConnect()+256                          <<-----Crashing function from OID Server log  
gsluslldConnect()   gslusrnConnect2RemNode()+1120  
gslusrnConnect2RemNode()gslusrnChkConn2RemNode()+144  
gslusrnChkConn2RemNode()gslarscNotifyController()+176  
gslarscNotifyController()__pthread_bound_body()+400

 

or with heavy trace debugging enabled, messages like

oidldapd01sxxxx-0000.log
----------------------------------

[2013-10-22T12:18:06-05:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost01] [pid: 18261] [tid: 17] [ecid: 004uLULULAoCsl0pnwk3yZ0004TL00019G,0] ServerWorker (REG):[[
BEGIN
ConnID:4512 mesgID:475092 OpID:1  OpName:bind ConnIP:192.0.2.73 ConnDN:cn=svc-oid-read,cn=application service accounts,cn=internal,cn=users,dc=oracle,dc=com
gslusldSendExtOp: LDAP result failed rc:81 host:droid01 port:3060
END
]]
[2013-10-22T12:18:06-05:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost01] [pid: 18261] [tid: 17] [ecid: 004uLULULAoCsl0pnwk3yZ0004TL00019G,0] ServerWorker (REG):[[
BEGIN
ConnID:4512 mesgID:475092 OpID:1  OpName:bind ConnIP:192.0.2.73 ConnDN:cn=svc-oid-read,cn=application service accounts,cn=internal,cn=users,dc=oracle,dc=com
gslusrnWriteToRemNodes: Notfy to host:droid01 failed
END
]]
[2013-10-22T12:18:06-05:00] [OID] [NOTIFICATION:16] [] [OIDLDAPD] [host: oidhost01] [pid: 18261] [tid: 17] [ecid: 004uLULULAoCsl0pnwk3yZ0004TL00019G,0] ServerWorker (REG):[[
BEGIN
ConnID:4512 mesgID:475092 OpID:1  OpName:bind ConnIP:192.0.2.73 ConnDN:cn=svc-oid-read,cn=application service accounts,cn=internal,cn=users,dc=oracle,dc=com
sgslufread: Hard error on read, OS error = 32
END
]]

 

This will also cause client-side errors such as connection errors and response read timeouts from applications like OAM eg.
LDAP Error 1 : LDAP response read timed out, timeout used:30000ms.

Changes

 OID had been started with different hostnames at various times. Some of these hostnames were no longer resolveable or not responding.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms