My Oracle Support Banner

AWR shows high "gc cr grant 2-way" wait and the trace file shows that lms has not moved for more than 60 seconds (Doc ID 1630755.1)

Last updated on JUNE 18, 2023

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.2 to 11.2.0.3 [Release 11.2]
Oracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Cloud Machine - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Oracle Database Backup Service - Version N/A and later
Information in this document applies to any platform.

Symptoms

The database performance is very slow, and there are spikes of slowness.  The database performance eventually may recover after some time (often lasts more than an hour).

Symptoms are

1) The AWR report shows high gc waits including "gc cr/current grant .. " waits. 
The "gc cr/current grant .. " waits usually indicate busy lms processes as the the grant function involves transmission of very small packets, so the network is not likely the cause.

An example AWR report shows

Top 5 Timed Foreground Events

Event Waits Time(s) Avg wait (ms) % DB time Wait Class
gc cr grant 2-way 759,706 54,668 72 18.46 Cluster
DB CPU   40,218   13.58  
gc current grant busy 75,783 27,518 363 9.29 Cluster
gc current grant 2-way 237,492 24,475 103 8.26 Cluster
gc current block busy 51,179 21,462 419 7.25 Cluster

2) The top and ps outputs of OSWatcher show that lms is taking 100% of one CPU.

3) A trace file (usually lmon) shows one or more lms processes are not moved for many seconds although the problem lms is not waiting for any resource.  The trace file shows the the short stack of the lms includes ktrgcm.

An example for a lmon trace file shows
LMS0 (ospid: 29316) has not moved for 76 sec (1384485973.1384485897)
kjfmGCR_HBCheckAll: LMS0 (ospid: 29316) has status 2
 : Not in wait; last wait ended 0 secs ago.
 : last wait_id 238514 at 'gcs remote message'.

Short stack:
...<-ktrgcm()+5936 ...




 
4) in AWR report, the stats "data blocks consistent reads - undo records applied" is much larger (in millions) compare with other stats

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Symptoms
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.