My Oracle Support Banner

OUD11g - How to Shorten Import Time When Performing 'import-ldif' Command on Millions of Entries (Doc ID 2223626.1)

Last updated on MARCH 08, 2017

Applies to:

Oracle Unified Directory - Version 11.1.2.2.3 to 11.1.2.3.161018 [Release 11g]
Information in this document applies to any platform.

Goal

When importing a database that contains millions of entries via the 'import-ldif' command, the import may take hours to complete. In the following example, it takes one hour and 7 minutes to import roughly 37 million entries...
 

- Import number 1
[dsuser@idsync1 bin]$ time ./import-ldif --isCompressed --ldifFile /home/dsuser/Oracle/Middleware/11.1.2.2.example.2016.07.28.ldif --threadCount 2 --includeBranch dc=example,dc=com --backendID userRoot
[01/Sep/2016:10:23:32 -0700] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381717 msg=Installation Directory: /home/dsuser/Oracle/Middleware/Oracle_OUD11.1.2.2
[01/Sep/2016:10:23:32 -0700] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381719 msg=Instance Directory: /home/dsuser/Oracle/Middleware/asinst_11.1.2.2/OUD
[01/Sep/2016:10:23:32 -0700] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381713 msg=JVM Information: 1.7.0_25-b15 by Oracle Corporation, 64-bit architecture, 10603200512 bytes heap size
[01/Sep/2016:10:23:32 -0700] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381714 msg=JVM Host: idsync1.us.oracle.com, running Linux 2.6.32-642.1.1.el6.x86_64 amd64, 33591791616 bytes physical memory size, number of processors available 8
[01/Sep/2016:10:23:32 -0700] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381715 msg=JVM Arguments: "-Xms10240m", "-Xmx10240m", "-XX:+UseCompressedOops", "-Xmn1g", "-XX:+UseParallelGC", "-XX:+UseNUMA", "-Dorg.opends.server.scriptName=import-ldif"
[01/Sep/2016:10:23:37 -0700] category=JEB severity=NOTICE msgID=8847533 msg=Oracle Unified Directory 11.1.2.2.3 starting import (build 20151114172139Z, R1511140808)
<SNIPPED>

[01/Sep/2016:10:56:30 -0700] category=JEB severity=NOTICE msgID=8847517 msg=Migrating excluded entries for base DN dc=example,dc=com
[01/Sep/2016:10:56:31 -0700] category=JEB severity=NOTICE msgID=8847593 msg=Starting phase two (building indexes)
[01/Sep/2016:10:56:41 -0700] category=JEB severity=NOTICE msgID=8847590 msg=Processed 0.84% of indexes
<SNIPPED>
[01/Sep/2016:11:31:01 -0700] category=JEB severity=NOTICE msgID=8847590 msg=Processed 99.99% of indexes
[01/Sep/2016:11:31:07 -0700] category=JEB severity=NOTICE msgID=8847569 msg=Total import time was 4049 seconds. Phase one processing completed in 1973 seconds, phase two processing completed in 2073 seconds
[01/Sep/2016:11:31:07 -0700] category=JEB severity=NOTICE msgID=8847454 msg=Processed 37000002 entries, imported 37000002, skipped 0, rejected 0 and migrated 0 in 4049 seconds (average rate 9136.7/sec)
[01/Sep/2016:11:31:07 -0700] category=JEB severity=NOTICE msgID=8847453 msg=Flushing data to disk, cleaning up and closing the database environment, this may take a while

real 67m38.591s
user 130m26.449s
sys 3m28.941s
[dsuser@idsync1 bin]$

 

Can the import time possibly be improved?

Solution

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Goal
Solution
 --skipDNValidation --skipSchemaValidation --isCompressed
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.