Consideration and Issues of GoldenGate Trails Stored on DBFS (Doc ID 1624327.1)

Last updated on JULY 17, 2017

Applies to:

Oracle GoldenGate - Version 11.2.1.0.6 and later
Information in this document applies to any platform.

Symptoms

We have trails on DBFS, and replicat hangs for several hours before moving again.

example (from ggserr.log):

Switching to next trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675 at 2014-01-22 11:13:05 due to EOF, with current RBA 99999905
Opened trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675 at 2014-01-22 11:13:05

            6917076 records processed as of 2014-01-22 11:31:34 (rate 66,delta 105)

2014-01-22 12:01:40  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:01:49  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:02:05  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:02:38  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:03:44  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:05:56  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:10:19  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:19:05  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 12:36:38  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 13:11:43  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 14:21:54  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 16:42:15  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-22 21:22:59  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

2014-01-23 06:44:27  WARNING OGG-01519  Waiting at EOF on input trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000675, which is not marked as complete; but succeeding trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 exists. If ALTER ETROLLOVER has been performed on source extract, ALTER EXTSEQNO must be performed on each corresponding downstream reader.

Switching to next trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 at 2014-01-23 09:29:29 due to EOF, with current RBA 99999920
Opened trail file /apps/oragg/ggs/11.2.1.0.6/PRDSECF/dirdat/in/pc000676 at 2014-01-23 09:29:29


2014-01-23 09:29:29  INFO    OGG-01521  Scan resumed.
            7109159 records processed as of 2014-01-23 09:29:29 (rate 38,delta 2)

 

 Sometimes, Replicat will also hit this issue and upon restart, it was able to continue run and switch to next trail file OK.

2014-02-24 20:35:20  ERROR   OGG-01733  Trail file header file size value 499999972 for trail file '/data01/dbfs/ggate/dirdat/at000183' differs from the actual size of the file (473601038).

Those issues are indications that the DBFS mount point is using local file systme cache, which causes the reader could not "see" the content from the writer as writer is writing to the DBFS from a different node and the content was still in the local file system cache.  Upon the cache flush, the data will appear in the trail file, then the reader was able to pick it up and continue to run. 

Please check if customer has "direct_io" option enabled for the DBFS mount point.  This option will disable local file system cache and force the I/O get flushed to the disk file directly.  The option could be checked thru the result of "mount" command under shell

shell>mount

Looks for the DBFS mount point and check for the options used.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms