OID 11g Ldapd Server Process Repeatedly Crashing and Restarting After Setting High Level Debug (orcldebugflag=117440511) with Log Error: Recieved signal = 11, shutting down (Doc ID 1984009.1)

Last updated on MARCH 08, 2017

Applies to:

Oracle Internet Directory - Version 11.1.1.7.0 and later
Information in this document applies to any platform.

Symptoms

Clustered Oracle Internet Directory (OID) 11g, e.g. 11.1.1.7.0 version with latest OID Bundle Patch 11.1.1.7.7 applied, and using OID Multimaster (MMR) Replication between the clusters.

OID ldapd server process is crashing and restarting repeatedly.

After enabling logging to orcldebugflag=117440511(*) observed the repeated crashes when checking server logs, every couple minutes:

[2015-01-29T13:10:29-06:00] [OID] [TRACE:16] [] [OIDLDAPD] [host: myoidhost] [pid: 20393] [tid: 9] [ecid: 0053SwxlOsRF^6jCD37i4w0004yd00001I,0] ServerWorker (REG):[[
BEGIN
ConnID:21 mesgID:4 OpID:3  OpName:add ConnIP: ConnDN:cn=replication dn,orclreplicaid=myhost_mysid,cn=replication configuration
gslfadADoAdd:conn=21 op=3 ADD dn="cn=mydb,cn=OracleContext,dc=mycompany,dc=com
"
2015-01-29T13:10:29 * gslbndnWithEscChar: origDN:<cn=mydb,cn=OracleContext,dc=mycompany,dc=com
> NormDn:
2015-01-29T13:10:29 * gslusrReplDecodeVal:bv_val:32628 danlrlb940_oidstl1d1 20150129040342z  0B72C6AAD973D53DE0508FA7298106A8
     chg_no:32628 server:danlrlb940_oidstl1d1 op_time:20150129040342z guid:0B72C6AAD973D53DE0508FA7298106A8 pguid:
2015-01-29T13:10:29 * gsltxpcProcTxnCtl * Entry
2015-01-29T13:10:29 * gsltxpcProcTxnCtl * Exit
2015-01-29T13:10:29 * gsldpSearchStart * BASE = "cn=oraclecontext,dc=mycompany,dc=com" SCOPE = 0
2015-01-29T13:10:29 * gsldfbsFetchBaseObjs * BASE = "cn=oraclecontext,dc=mycompany,dc=com"
2015-01-29T13:10:29 * gsldfbsFetchBaseObjs * RevBaseDN = ""
RevParentDN = "dc=com,dc=mycompany,"
ParentDN = "dc=mycompany,dc=com"
2015-01-29T13:10:29 * gsldfbsFetchBaseObjs * RDN[0] = "cn=oraclecontext"
2015-01-29T13:10:29 * gsldfbsFetchBaseObjs * RDN[1] = "dc=mycompany"
2015-01-29T13:10:29 * gsldfbsFetchBaseObjs * RDN[2] = "dc=com"
2015-01-29T13:10:29 * Base Search Sql: SELECT /*+ USE_NL(store) USE_NL(dn) INDEX(store EI_ATTRSTORE) INDEX(dn RP_DN) ORDERED */ store.entryid,AttrName,NVL(AttrVal,' '),attrkind,NVL(attrstype, ' '),NVL(AttrVer,' ') FROM CT_DN dn, ds_attrStore store WHERE (dn.rdn = :szCommonName AND dn.parentdn = :szBaseDomain) AND store.entryid = dn.entryid
2015-01-29T13:10:29 * Bind variables: :szCommonName = "cn=oraclecontext", :szBaseDomain = "dc=com,dc=mycompany,"
2015-01-29T13:10:29 * Base Search Completed with: 100
2015-01-29T13:10:29 * gslbndnWithEscChar: origDN: NormDn:
2015-01-29T13:10:29 * gslsbaCheckRDNinEntry Entry RDN=cn=mydb
2015-01-29T13:10:29 * gslbadCheckRDNinDN: mydb = valBuf, normdn= mydb
2015-01-29T13:10:29 * gslbadCheckRDNinDN:match = TRUE, attrVal = mydb, normvalue = mydb
2015-01-29T13:10:29 * gslbndnWithEscChar: origDN: NormDn:
2015-01-29T13:10:29 * [gslsbscRetCatalogAlias] Attribute orclnetdescstring is not indexed
2015-01-29T13:10:29 * [gslsbscRetCatalogAlias] Attribute creatorsname is not indexed
2015-01-29T13:10:29 * [gslsbscRetCatalogAlias] Attribute modifytimestamp is not indexed
2015-01-29T13:10:29 * [gslsbscRetCatalogAlias] Attribute createtimestamp is not indexed
2015-01-29T13:10:29 * Updating retry_cnt asr change log (success), sql=update asr_chg_log set retry_cnt = decode(retry_cnt, 1, -1, retry_cnt - 1)     where chg_no = :lpChangeNo AND retry_cnt != -1 AND server = :lpSrvr
2015-01-29T13:10:29 * 2015-01-29T13:10:29 * Recieved signal = 11, shutting down
END
]]

The error occurs on random OID entries (OID is used for tnsnames / database servicename entries only).

All nodes were rebooted after a mandatory OS patching, and the crashing temporarily stopped on one of the nodes only, but returned soon after.

Also cleared out and purged HIQ, as the error occurs immediately after an update to asr_chg_log.

Tried a relink of OID/LDAP client and server, to no help.


After further testing, OID is only crashing with the debug level value set high (117440511) and the replication server running.

It no longer crashes once debug is set to low level (1).


(*)Note: One requirement is to capture log data (including all operations, the client IP, the connection DN, the DN that was operated on, result code, search filters, etc), and this is the reason why the high level debug was applied.

Changes

Set high level debug in OID server.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms