Write Throttling in UFS and ZFS File Systems

(Doc ID 1470681.1)

Last updated on SEPTEMBER 13, 2017

Applies to:

Solaris Operating System - Version 10 3/05 and later
Information in this document applies to any platform.

Symptoms

File system cache is used for storing application data temporarily in physical memory when the system reads/writes data from/to disk.  Considering that accessing data from the disk subsystem is slower (millisecond latency) than physical memory (microsecond latency), data read or written to memory improves application IO performance.  The filesystem cache grows and shrinks on demand.  Pages are consumed from the freelist (list of free pages) as files are read/written into memory from/to disk.  Filesystems work hard to prevent applications from suffering disk IO latency directly, such as by using DRAM to buffer writes and to cache and prefetch reads.  Advantages of filesystem cache include:

When an application writes, the data is buffered in a filesystem cache to improve efficiency.  However, large numbers of dirty (modified) blocks in the filesystem cache can tie up memory and may cause memory shortages that could negatively affect overall system performance.  Filesystems, such as UFS and ZFS, use write throttling to avoid having dirty buffers occupy too much memory.  To prevent a single process dirtying too many pages in the filesystem cache, application processes are frequently put to sleep on write() to slow down the dirty buffer growth until storage catches up.

write() to filesystem cache should be instantaneous (an exception to this is files that are open for synchronous writes, O_SYNC|OD_SYNC).  However, due to filesystem throttling imposed by UFS and ZFS file systems, applications may frequently block on write() or pwrite() system calls.  A typical kernel stack showing an application thread blocking on a pwrite() system call on a ZFS filesytem is shown below:

  

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms