NFSv3 over UDP WRITE requests receive NFS3ERR_JUKEBOX from Solaris 11.4 NFS server; File Writes Cannot Complete.
(Doc ID 2721737.1)
Last updated on APRIL 20, 2021
Applies to:Solaris Operating System - Version 11.4 and later
Information in this document applies to any platform.
An NFS client utilizing NFSv3 with UDP - which is writing to multple files quite often (50-100 times/sec overall) - after about 15 minutes starts seeing NFS3ERR_JUKEBOX responses to NFSv3/UDP WRITE requests.
These NFS3ERR_JUKEBOX responses are the NFS server's way of saying "I can't complete your request right now, please try again later". This is so called due to NFS servers "back in the day" having data stored on CD-ROM jukebox devices, which required a few seconds to access. In this case files on the NFS server are located in a normal ZFS dataset, which are immediately available (and almost certainly cached in the ZFS ARC).
Trying again later is what the NFS client does - it waits a few seconds and tries again - which often ends up with another NFS3ERR_JUKEBOX response.
The NFS3ERR_JUKEBOX responses are not consistent. Not every NFSv3/UDP WRITE requests receives the NFS3ERR_JUKEBOX response - just enough to cause a serious problem for the NFS client. Some files can't be written to for quite a long time.
While this is happening, other NFS operations, READS or GETATTR requests in particular - always receive a response. This is true of any operations that do not alter data on the NFS server (including READDIRPLUS,etc).
During the issue, NFSv3 badcalls stats increment, as does the "full" duplicate request cache counter.
This kstat command illustrates these counters:
The badcalls and rpcmod:0:drc_0_clts:full counter will increment each time a NFS3ERR_JUKEBOX response is generated.
The customer upgraded from Solaris 11.3 to Soiaris 11.4
To view full details, sign in with your My Oracle Support account.
Don't have a My Oracle Support account? Click to get started!
In this Document