Temporary Disk I/O Stalls; LCP Frag Watchdog : No progress on Table (Doc ID 1582813.1)

Last updated on APRIL 28, 2024

Applies to:

Symptoms

During periods with high load, for example while creating backups or executing bulk updates particularly if these involve large BLOBs, disk I/O activities are temporary stalling.

For performance critical I/O activities this may cause temporary errors and/or the data node to crash. Examples of performance critical I/O in MySQL Cluster are:

• Local Checkpoints (LCPs). If no progress is detected for a period of LcpScanProgressTimeout (defaults to 60 seconds; in 7.2 before 7.2.14 and in 7.3 before 7.3.3 this was hardcoded to 60 seconds), the LCP Frag watchdog shuts down the data node.
• REDO logs. If the REDO logs takes too long to flush, pending transactions will be aborted.

One sign of the issue - although not necessarily required - is 100% disk utilisation while nothing is written to or read from the disk, for example:

Changes

The more memory that is available on the server, the more likely this issue is to happen, so the issue may begin after installing more memory or upgrading to a new server with more memory available.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.