ACFS Hang/Freeze During Node Reboot, after several million files were Created in ACFS (Doc ID 1416386.1)

Last updated on FEBRUARY 17, 2012

Applies to:

Oracle Server - Enterprise Edition - Version: 11.2.0.3 and later   [Release: 11.2 and later ]
Information in this document applies to any platform.

Symptoms

It has been observed in a clustered 11.2.0.3 Grid Infrastructure environment that ACFS will hang/freeze for a few minutes (say 4~5 minutes) during node reboot, after several million files (say, 5 million) were created in ACFS.

OKS log:
~~~~~~~~~~
Dec 20 06:20:23 my-node kernel: [Oracle OKS] Cluster Membership change - Current incarn 0x18
Dec 20 06:20:23 my-node kernel: [Oracle OKS] Nodes in cluster:
Dec 20 06:20:24 my-node kernel: [Oracle OKS] Node 1 (IP 0xa9fe5d48) Node 2 (IP 0xa9fe2f00)
Dec 20 06:20:24 my-node kernel: [Oracle OKS] Node count 2, Local node number 1
Dec 20 06:20:24 my-node kernel: ADVMK-00013: Cluster reconfiguration started.
Dec 20 06:20:52 my-node ntpd[19679]: synchronized to xx.x.xx.xx, stratum 1
Dec 20 06:20:52 my-node ntpd[19679]: synchronized to xx.x.xx.xx, stratum 1
Dec 20 06:21:50 my-node hpasmlited[20068]: hpDeferSPDThread: End of Collecting DIMM SPD data.
Dec 20 06:21:50 my-node hpasmlited[20068]: hpDeferSPDThread: End of Collecting DIMM SPD data.
Dec 20 06:24:47 my-node kernel: ADVMK-00014: Cluster reconfiguration completed.
Dec 20 06:24:47 my-node kernel: ADVMK-00014: Cluster reconfiguration completed.      <---- Took over 4 minutes

Note: To get a memory dump of the oks logs on a live system you can exec "cat /proc/oks/log > /tmp/oks.log.<hostname>" on each node.
Starting with 11.2.0.3.0, persistent oks logs are kept on each node. To determine their location, do: /sbin/acfsutil plogconfig -q

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms