Intermittent Long 'log file sync' Waits on Veritas ODM, LGWR Posting Long Write Times, I/O Portion of Wait Minimal
(Doc ID 1278149.1)
Last updated on FEBRUARY 03, 2019
Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.1 and laterInformation in this document applies to any platform.
Symptoms
- The LGWR appears to be encountering regular problems where it hangs for 3 seconds at a time.
During this time, an iostat -x on the redo devices shows zero IO being pushed through. - Veritas ODM (Oracle Disk Manager) is being used
- LGWR trace shows numerous entries such as:
...
*** 2010-11-01 07:33:14.745
Warning: log write elapsed time 3203ms, size 1KB
(set event 10468 level 4 to disable this warning)
*** 2010-11-01 07:52:57.813
Warning: log write elapsed time 3345ms, size 3KB
(set event 10468 level 4 to disable this warning)
... - The average wait times for log file sync do not typically appear to be made up of IO waits (there is no evidence of 3 second IOs):
NAME TOTAL_WAITS TOTAL_WAIT_TIME AVG_TIME_WAITED
----------------------- ----------- ----------------- -----------------
log file sync 84 12684.472 151.006
log file parallel write 17655 28848.331 1.634The affect on any session in the database at this point is to wait with a log file sync event, effectively stalling the system for 3 seconds.
- When dtrace was run to trace the issue, the problem went away. Problem came back as soon as dtrace was stopped.
- Truss on LGWR also avoids the issue.
- At the times when LGWR suffers the slow write, the process is given a priority of 0. So it looks like the operating system is scheduling the LGWR process off CPU and it takes 3 seconds to get back on and catch up.
Dtrace looks to resolve the issue by either keeping the lgwr process on CPU or ensuring if it does come off CPU, it is re-instated immediately. - System is NOT under load when this happens. It even happens more during the quiet periods.
- Renice the LDWR lgwr process makes no difference.
- From 10298 trace, the modules immediately prior to the waits are not the same and are sometimes IO related and are sometimes not which indicates that the delay is not necessarily down to a wait for I/O completion.
Changes
none
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Symptoms |
Changes |
Cause |
Solution |
References |