Duplicate Records Are Being Extracted Via D2-SPCFX Batch
Last updated on JANUARY 06, 2015
Applies to:Oracle Utilities Meter Data Management - Version 220.127.116.11 and later
Information in this document applies to any platform.
When running the D2-SPCFX to load consumption data into BI, the load failed due to duplicate records being extracted from MDM.
On investigating the issue we found that in the Thread Iteration strategy the low ids and High ids used while filtering the SP query, in the getQueryIteratorForThread() function are nothing but SP ids in this case .
When the processing time of the query iterator exceeds the default 15 minutes, the low id is refreshed. The new low id is an incremented value of the last low id processed.
Since the SP ids are processed in a random fashion, the new low id after 15 minutes could be any random id.
Hence after 15 minutes the SP query is filtered with a new low id and high id 9999999 this causes some of the already extracted ids to be extracted again.
Also this same behaviour is observed in the other extraction batches (D1-SPSFX, D2-SVEFX and D2-SUAFX) as well.
Sign In with your My Oracle Support account
Don't have a My Oracle Support account? Click to get started
My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms