Introduction to ALLIDS Threshold and Modify Method for ODSEE and DSEE (Doc ID 1369615.1)

Last updated on NOVEMBER 19, 2016

Applies to:

Oracle Directory Server Enterprise Edition - Version 6.3.1 SP1 DPS6.3.1.1 and later
Information in this document applies to any platform.
Many Customers want to increase the allidtreshold
but the question is always which value can I use ?


Goal

Definition: all-ids-threshold / allidsthreshold  

 

 all-ids-threshold is the maximum number of values per index key in an index list

 

This property defines the maximum number of values per index key that the server maintains in an index list. It can be set for an entire server instance, for an entire suffix, and for anindividual attribute type. You can also set individual thresholds for equality, presence, and substring indexes.

Classic example  sent  from customers:

 

Example with Sun Directory 6.3.1


Customer  would like to raise the allid threshold and need further information regarding the documentation-recommended soft limit of 50.000.

our Tree-Structure basically is a rather flat structure:
The most important trees are:
dc=root
ou=internal
ou=people ~ 200.000 Entries, flat structure
ou=supplier
ou=people ~ 80.000 Entries, flat structure
ou=dealer
ou=people ~ 70.000 Entries, flat structure
ou=retired
ou=people ~ 20.000 Entries, flat structure
ou=groupofurls ~ 2.000 Entries, flat structure
... some more minor ou's
Our directory is mature, number of entry growth is ~ 1%/year

 

It's very important to found the good sizing of allidsthreshold:

If the index of one of the specified attributes matches too many entries (ALLIDs), the search becomes non-indexed (notes=U in the access log), and If the value of allidsthreshold is too high, you lost the benefit of the index,  the search scan too many entries and the performances are near a search non-indexed.


The Reference Guide says about allidsthreshold:
"You should avoid setting the threshold above 50 000, even for very large deployments.However, you might set all-ids-threshold to a value other than the 5% guideline in the following situations:
set the all-ids-threshold high for parent and ancestor indexes so that all entries below a given branch are indexed."

This magical "50.000" limit has been around since Version 5 and we would like to know, if we can raise the allidsthreshold above 50.000 to a target value of 250.000.
- We have several clients regularly search one of the mentioned subtrees to get a list of the contained DNs (hitting the allid-threshold limit of ancestor/parent index).
- We have several clients regularly search for all members of certain groupOfURLs-groups with ~150.000 members (we have about 50 groupOfURLs with ~100.000 members)

We could really need such a SEARCH speedup and would now like to know if 250.000 would be ok as global allidsthreshold, and only keep all substring-indexes thresholds to 4000.
The "50.000" limit has been around since years (Version 5) and there also has been vast hardware improvements since then.
There is no clear documentation what will happen if this soft limit of 50.000 is exceeded.


Customers Questions



1) - Are there internal issues that need this value to not exceed 50.000 ?



- no 50 000 is not a limit, the limit is near 250 000 but it depends of the size of the db_size_page ...


2) What are the technical side-effect implications?



- normally if the candidate list ( allidsthreshold) > 20 % ~ 50% of the number of entries in your database, probably in majority of cases you will lost the index benefit but it should depend also the size of cache / the performance of disk .....


3) In this case 250 000 is very too high and it better to tune the allidsthreshold per index ( introduce in version 6.x


- if the ldap client can use vlvcontrol and if you execute always the same specifics searches you could use the vlvindex with a search filter which match exactly vlvfilter, see in Admin Guide chapter: Managing Browsing Indexes

-but now if you have very big index you could increase the nsslpad_db_page_size ( 8k by default, valid range 512 bytes to 64 KB ) to have a maximum of candidate list in one dbpage

- there are no good value but a value > 100 000 in the majority of case is already too high, (It is better to reorganize the data structure which is not any more adapt for ldap Directory , difficult operation and not easy with the evolution of the ldap directory )

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms