Db Corruption In Oracle 11.1.1.5.0

(Doc ID 1584864.1)

Last updated on OCTOBER 11, 2016

Applies to:

Oracle Directory Server Enterprise Edition - Version 11.1.1.5.0 and later
Information in this document applies to any platform.

Symptoms

------------------------------------------
Loss of functionality
------------------------------------------

11g DS fails to restart after unscheduled restart.

------------------------------------------
The sequence of events
------------------------------------------

During system patching exercise, customer rebooted the server without stopping DS instances. The transaction log in /tmp directory was wiped out due to reboot. Customer would like to know how to recover the db on that.

Unfortunately customer don't have any recent ldif backup and their net backup doesn't backup /tmp file system.


------------------------------------------
Where the issue happened
------------------------------------------

Both Sun-Directory-Server/11.1.1.5.0 B2011.0517.2158 (64-bit) Directory Servers

------------------------------------------
Significance of loss
------------------------------------------

Customer will be losing 2 to 3 months data if they can't recover the existing data.

------------------------------------------------------------------------------------
Specific error messages (error number with syntax)
------------------------------------------------------------------------------------

Please check the errors that are getting when trying to start instance

Waiting for Directory Server instance '/opt/ldap/odsee11g/instances/xxx-xxx' to start...
Directory Server instance '/opt/ldap/odsee11g/instances/xxx-xxx' has detected a disorderly shutdown or a change in cache size
Recovery phase is starting, this may take a while...
Waiting for Directory Server instance '/opt/ldap/odsee11g/instances/xxx-xxx' to start...
Waiting for Directory Server instance '/opt/ldap/odsee11g/instances/xxx-xxx' to start...
Waiting for Directory Server instance '/opt/ldap/odsee11g/instances/xxx-xxx' to start...
Failed to start Directory Server instance '/opt/ldap/odsee11g/instances/xxx-xxx'
ns-slapd wrote the following lines in the error log (/opt/ldap/odsee11g/instances/xxx-xxx/logs/errors):
##[11/Sep/2013:21:55:07 +0000] - Sun-Directory-Server/11.1.1.5.0 B2011.0517.2158 (64-bit) starting up
##[11/Sep/2013:21:55:08 +0000] - WARNING - Backend Database - conn=-1 op=-1 msgId=-1 - Detected Disorderly Shutdown last time Directory Server was running, recovering database.
##[11/Sep/2013:21:55:45 +0000] - Database recovery is 100% complete.
##[11/Sep/2013:21:55:47 +0000] - DEBUG - conn=-1 op=-1 msgId=-1 - libdb: file visa%2ecom_id2entry.db3 has LSN 15728/1859964, past end of log at 1/28
##[11/Sep/2013:21:55:47 +0000] - DEBUG - conn=-1 op=-1 msgId=-1 - libdb: Commonly caused by moving a database from one database environment
##[11/Sep/2013:21:55:47 +0000] - DEBUG - conn=-1 op=-1 msgId=-1 - libdb: to another without clearing the database LSNs, or by removing all of
##[11/Sep/2013:21:55:47 +0000] - DEBUG - conn=-1 op=-1 msgId=-1 - libdb: the log files from a database environment
##[11/Sep/2013:21:55:47 +0000] - DEBUG - conn=-1 op=-1 msgId=-1 - libdb: /opt/ldap/odsee11g/instances/xxx-xxx/db/xxx/xxx_id2entry.db3: unexpected file type or format
##[11/Sep/2013:21:55:47 +0000] - WARNING - Backend Database - conn=-1 op=-1 msgId=-1 - db->open("visa%2ecom_id2entry.db3") failed Invalid argument (22)
##[11/Sep/2013:21:55:47 +0000] - ERROR - Backend Database - conn=-1 op=-1 msgId=-1 - start: Failed to start databases, err=22 Invalid argument
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start database plugin ldbm database
##[11/Sep/2013:21:55:47 +0000] - ERROR - Resource Limit - conn=-1 op=-1 msgId=-1 - nsLookThroughLimit: parameter error (slapi_reslimit_register() already registered)
##[11/Sep/2013:21:55:47 +0000] - ERROR - Backend Database - conn=-1 op=-1 msgId=-1 - start: Resource limit registration failed
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start database plugin ldbm database
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Error: Failed to resolve plugin dependencies
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start accesscontrol plugin ACL Plugin
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start preoperation plugin ACL preoperation
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start postoperation plugin Class of Service
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start object plugin gle
##[11/Sep/2013:21:55:47 +0000] - ERROR - plugin_dependency_startall - conn=-1 op=-1 msgId=-1 - Configuration error Failed to start database plugin ldbm database
##[11/Sep/2013:21

------------------------------------------
What IS working
------------------------------------------

One v5.2 DS is working but replication is one way from 11g to v5.2 DS and all updates are to 11g server.

------------------------------------------------------------------------------------
What is the expected or acceptable result
------------------------------------------------------------------------------------

Customer would like to know if there is any other way to recover the lost data otherwise they will be losing 2 to 3 months data due to this outage.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms