My Oracle Support Banner

Q & A On Elastic Search With Classic Message Store (Doc ID 2416387.1)

Last updated on MAY 05, 2021

Applies to:

Oracle Communications Messaging Server - Version 8.0.2 and later
Information in this document applies to any platform.

Purpose

Information pertaining to the Elasticsearch options in the documentation for Messaging Server and the various items linked from there and the Searchengine Option, which is identified as a restricted option.

Questions and Answers

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Purpose
Questions and Answers
 1.  Are there more details on the restricted Searchengine option?
 2.  Are there any details about appropriate sizing and elastic search instance for classic store search ? For example, would 1:1 with the size of all idx be appropriate ? Is having the Storesource Option elasticsearch.storesource enabled good, and will this drive more disk space consumption.
 3.  Are there specific details on how the classic store acts as a "client" for Elastic search, in terms of how records are inserted and queried for?
 4.  Are there other items that are appropriate to provide for a successful test deployment?

 5.  Are there any specific Java requirements for ISC on the backends for using Elastic Search?
 6.  Is there a summary of the isc_client options?
 7.  https://docs.oracle.com/communications/F15020_01/doc.803/f15147/stins_planning.htm#MSVMS115
talks about installing ISC on the "message access tier host(s)" and LMTP.
It also says, "Messages that arrive in the store through the IMAP APPEND command or the imsrestore command are to be processed by ISC on the message access tier host(s)."
Does this mean that we cannot use Elastic if we are not using LMTP ?
Can we not run ISC on the backends with ims_master?
 8.  Does "message access tier" mean "backends" rather than "frontends"?
 9.  Regarding: authusername and authpassword on isc, isc_client, and elastic search.
Is it correct that authusername and authpassword on isc and isc_client simply need to match? i.e., these do not reference a user defined in LDAP or anywhere outside of the Messaging Server config.
Meaning isc_client.authusername and isc_client.authpassword simply need to match isc.authusername and isc.authpassword. Is that correct?
And the elasticsearch.authusername and elasticsearch.authpassword needs to match the user/password configured on the Elastic search service. This does not correspond to any user defined in LDAP or anywhere outside of Elastic Search. Is that correct?
 10.  How to initiate a full reindex? Or partial reindex, for that matter?
 The documentation about numshards https://docs.oracle.com/communications/F15020_01/doc.803/f15150.pdf#page=313&zoom=auto,-208,692 says "The number of shards cannot be changed after the index is created."
 So if something like that needs to be changed or if storesource option is disabled and needed to to be rebuilt... How would one go about rebuilding?
 In the upgrade-from-ISS scenario, each folder is converted to Elastic when it is accessed. The fact that it has been indexed in Elastic is stored as a flag in the store.idx (it is assumed).
What would need to be done to cause users/folders to be re-indexed in Elastic if the Elastic service had to be rebuilt?
 What would need to be done to start over?
From the Elastic Search server side, this easy enough: destroy/reinstall the the Elastic Search server.
But on the Messaging Server side, how to tell it to resend everything to Elastic?
 How to decide on the number of shards?
 If a 3-node cluster of Elastic Search servers exists, should all three names be specified in the hostlist parameter?
How does Messaging Server decide which one to use? Or, does it only use one?
 Is information about the number of shards (or which shard a folder is in), stored in store.idx or somewhere on the Messaging Server side?
 11.  Where is the Indexed Search Converter (ISC) going to store 30 days worth of incoming message data?
 From: https://docs.oracle.com/communications/F15020_01/doc.803/f15150.pdf#page=1592&zoom=auto,-208,718
 " ... specifies the time-to-live, measured in seconds, for the converted text in the conversion cache. The converted contents expire after this time, and ISC will have to re-convert any new document with the same content. The default is 30 days. The minimum is one hour."
 Where does data live?
Is there an option to specify the directory/path?
 12.  How can one measure the effectiveness of this cache so one can make informed tuning decisions?
 > ... ISC will have to re-convert any new document with the same content ...
 In the above statement, "document" means email message, correct?
 The same message could be delivered to multiple people. But that would all happen at basically the same time. And messages could be "duplicate" (and therefore benefit from this cache) when being moved (rehostuser: imsbackup->imsrestore) from a different backend. If multiple users were being moved in a short amount of time, they could benefit from this.
 What about moving messages from one folder to another? Does that involve this?
 13.  Part of the answer for question 6:
 Does that actually check with the Elastic Search service to see if the folder is indexed?
 If the need arose to have to change something that cannot be changed, that seems like it would require removing all the data from the Elastic Search servers and rebuilding everything.
It would be the same if some part of the ES service was lost and had to be rebuilt.
If the store thinks the folder has already been indexed, but the ES service has completely been rebuilt, how would one tell the store to do that?
 14.  Perhaps walking through the scenarios and provide examples of relevant parts of the output would be helpful.
 The "imcheck -m" command is a normal command which has existed for a long time.
Does it now also cause some action related to Elastic Search? Exactly when/what/how?
 In the 5th line of the output:
 The above is the same on the system where the new test facility (refer to: Doc ID 2630186.1) is being performed and on a system that has had nothing to do with Elastic.
 Is that the "Folder Flags" that's being talked about?
 And once that folder has been indexed with Elastic, that flag will be set to "I"?
 So in the first part of the answer:
 > 1) Mails sent when ES is down won't get indexed in ES.
 Running imcheck -m on a folder will do what, exactly? And are you saying that if Elastic is ever down, that will need to be performed on every folder?
 > 2) Re-indexing of messages is possible when Elasticsearch host is changed, or in the case the indexes of all the emails of a mailbox are to be rebuilt.
 Yes, is the point of this question.
If the Elastic service needs to be reinitialized, how does one cause folders, which had already been indexed on the previous service, to be indexed on the new service?
 It sounds like the answer is to run "imcheck -m" on every folder.
Two things:
 * That sounds like it would be difficult because "imcheck -m" does not take wildcards. One would need to list every folder and then run "imcheck -m" on each folder.
 How would "imcheck -m" cause the folder index to be rebuilt? What does it check? What does it do? How could one monitor its progress?
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.