Fct_DuplicateCheck Incorrectly Rejecting As Duplicate CDRs When Using Search_key Configuration (Doc ID 2737401.1)

Last updated on APRIL 05, 2023

Applies to:

Goal

The user has introduced duplicate checking and made following observations:

TEST 1:
The user has a unique identifier for every Call Detail Record (CDR) "DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO". In FCT_DuplicateCheck in batch rating pre-processor pipeline, search key is set as DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO and no additional data fields are added. During processing, duplicate check module is rejecting some percent of records even though it is not duplicate. This is noted for CDR files with more than 1K CDR records with the same start time.

TEST 2:
This issue is not happening for the smaller number of duplication records (approx 300/per login) with same start time. Rate of processing – 27K / sec.
1. SearchKey=DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO.
OR
2. SearchKey=DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO and Fields {1 = DETAIL.A_NUMBER}. It gives some more extra gap compared to previous one (approx 1000/login also not having the issue, anything more than that is rejecting few records).

TEST 3:
Above false rejection issue is not happening at all with below setup:
1. SearchKey=DETAIL.A_NUMBER
2. Fields { 1 = DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO }

But slowness is noticed while processing, which varies based on the number of duplicated records with same processing start date. If there is more start date variation, processing time is getting improved.

Questions:
1) Could there a possible explanation for the false duplicate rejection in TEST 1 and 2?
2) Is it correct to go with the configuration in TEST 3, with searchKey as LOGIN (A NUMBER) and additional field as "DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO" (this is unique across CDRs) to identify the duplicate records, while the intention was to just use DETAIL.ASS_VERIF_TRANS_EXT.INTERNAL_REF_NO in searchKey to detect duplicates. This configuration also has a downside of delaying the processing.

(It is to be noted that start time would be same based on business expectation as customer has some customization and perform month end processing based on some aggregation logic or so.)

Solution

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.