ORA-4031 on RAC Database With 'ges big msg p' Consuming Memory (Doc ID 1433164.1)

Last updated on NOVEMBER 28, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 10.2.0.5 and later
Information in this document applies to any platform.

Symptoms


This incident was observed on a Oracle 10.2.0.5 on a Solaris SPARC 64-bit server, using manual memory management, and on a RAC database.

The database may encounter a short duration hang, maybe 3 to 4 minutes, and there may be no error messages in the alert log.

The buffer cache will be large. In this example, it is 64G.
The shared pool in this example is 10G.
There is most likely many subpools in the shared pool.

You may also notice the presence of a ORA-4031 trace file in the dump directory, and this
ORA-4031 trace file will have the same time stamp as the hang.

A review of the ORA-4031 trace file will show the subpools, and we can get the largest memory
consumers in each subpool.

This is an example with seven subpools and this example shows the top 5 consumers in each subpoool.

subpool 1 major consumers (top 5)
-------------------------

"gcs resources " 205908312
"sql area " 172895856
"gcs shadows " 118403984
"library cache " 78437344
"Checkpoint queue " 74761344



subpool 2 major consumers (top 5)
------------------------

"gcs resources " 205869184
"gcs shadows " 169042816
"sql area " 159811352
"FileOpenBlock " 114943056
"Checkpoint queue " 75017376




subpool 3 major consumers (top 5)
------------------------
"gcs resources " 266302280
"sql area " 199968128
"gcs shadows " 169021384
"library cache " 76432024
"Checkpoint queue " 74761344


subpool 4 major consumers (top 5)
------------------------

"gcs resources " 266281424
"sql area " 199432544
"gcs shadows " 118437168
"library cache " 76502416
"Checkpoint queue " 74761344


subpool 5 major consumers (top 5)
------------------------

"ges big msg p " 1848340560  <----- note largest consumer
"gcs resources " 205907944
"gcs shadows " 118439152
"Checkpoint queue " 75017376
"library cache " 60400992



subpool 6 major consumers (top 5)
------------------------

"sql area " 206921392
"gcs resources " 205888416
"gcs shadows " 118414264
"library cache " 75584432
"Checkpoint queue " 74761344


subpool 7 major consumers (top 5)
------------------------
"gcs resources " 205875496
"sql area " 172248416
"free memory " 146475520
"gcs shadows " 118418472
"Checkpoint queue " 75273408
"library cache " 74915600


If there is a ASH report present for the time of the hang, you may notice the presence of
Dynamic ReMastering (DRM) occurring. It will look like this:

gcs drm freeze in enter server mode

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms