ASM Alert Logs Show Continuously: Attempting Voting File Relocation (Doc ID 1457886.1)

Last updated on MAY 08, 2017

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.3 and later
Information in this document applies to any platform.
.

Symptoms

1) Following messages are observed in ASM alert log:

NOTE: Attempting voting file refresh on diskgroup GRID
NOTE: Voting file relocation is required in diskgroup GRID
NOTE: Attempting voting file relocation on diskgroup GRID
NOTE: Attempting voting file refresh on diskgroup GRID
NOTE: Voting file relocation is required in diskgroup GRID
NOTE: Attempting voting file relocation on diskgroup GRID
NOTE: Attempting voting file refresh on diskgroup GRID

 


2) Also an excessive logging is seen in cssd logs (eg  9 hours of logs are available in all 10 cssd logs under $GRID_HOME/log/<hostname>/cssd)
Most of these logs are related to CLSS_CONFIG_LOCK which is associated with configuration change requests- the most  common use of this is by ASM

2013-11-20 11:01:56.973: [    CSSD][38]clssgmHandleGrockRcfgUpdate: grock(#CLSS_CONFIG_LOCK), updateseq(26543197), status(0), sendresp(1)
2013-11-20 11:01:56.974: [    CSSD][38]clssgmDispatchCMXMSG: received a timed out RPC id(224) from node(2). Discarding.
2013-11-20 11:01:56.979: [    CSSD][5]clssgmExecuteClientRequest: Received group property update request from client (10285fcd0), type 3
........
2013-11-20 20:01:01.011: [    CSSD][38]clssgmFreeRPCIndex: freeing rpc 516
2013-11-20 20:01:01.011: [    CSSD][38]clssgmHandleGrockRcfgUpdate: grock(#CLSS_CONFIG_LOCK), updateseq(27161406), status(0), sendresp(1)
2013-11-20 20:01:01.011: [    CSSD][5]clssgmDiscEndpcl: gipcDestroy 359e018

 

3) We can see memory leak by RBAL process as below

zzz ***Tue Apr 11 15:00:01 BEIST 2017
USER PID PPID PRI %CPU %MEM VSZ RSS WCHAN S ELAPSED TIME COMMAND
grid 6553650 1 65 0.1 31.0 7633820 7660992 - A 183-05:16:42 4-19:59:13 asm_rbal_+ASM2 <<<<
grid 16842828 1 60 0.0 0.0 3584 32508 - A 183-05:16:44 00:09:26 asm_psp0_+ASM2

# svmon -P 6553650 -O segment=category

Unit: page

-------------------------------------------------------------------------------
Pid Command Inuse Pin Pgsp Virtual
6553650 oracle 2065376 27712 1827215 2071364

...............................................................................
SYSTEM segments Inuse Pin Pgsp Virtual
15923 10608 0 15923
...............................................................................
EXCLUSIVE segments Inuse Pin Pgsp Virtual <<<<
1916224 48 1810942 1916674
...............................................................................
SHARED segments Inuse Pin Pgsp Virtual
133229 17056 16273 138767

Rbal trace file records below

2017-04-12 12:01:26.204: [ CSSCLNT]clsssVoteDiskFormat: call clsscfgfmtbegin with leasedata 0, size 0
2017-04-12 12:01:26.204: [ CSSCLNT]clsssVoteDiskFormat: succ-ly format the Voting Disk
src:1107ffec8:7000100afb82a80:3:
/dev/rhdisk8:3f37656050374f2dbffd751d26675dbc:
2017-04-12 12:01:26.213: [ CSSCLNT]clsssVoteDiskFormat: call clsscfgfmtbegin with leasedata 0, size 0
2017-04-12 12:01:26.213: [ CSSCLNT]clsssVoteDiskFormat: succ-ly format the Voting Disk
src:1107fff30:7000100afb82e18:3:
/dev/rhdisk9:0fb332d276284f9abf5a653d4c66d627:
PST-old:1107ffe60:7000100afb82700:47:
/dev/rhdisk7:aa8985a6f6654f91bf876785474fd6dc:

 

Changes

Recent upgrade to 11.2.0.3

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms