Production ODA VMs Went Into UNKNOWN State, Base Nodes Had Load Average Of 130, High I/O Wait and Even df -k Command Can Hang (Doc ID 2413087.1)

Last updated on JULY 24, 2023

Applies to:

Oracle Database Appliance X5-2 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

VM unresponsive going into UNKNOWN state
then all the VM's that were running on the two nodes disappeared.
high load average (> 130) on basenode with high CPU I/O wait utilization
"df -h" where NFS mounts appear to be hung.
servers experienced I/O on loop devices that belong to VMs which causes the hang on VMs
while investigating the oakcli command showed no repo filesystems present -- (athough they were still showing as mounted).
"crsctl check crs" could not communicate to the cluster ready services.

'...Using the srvctl command from ODA_BASE would hang...'

	To view full details, sign in with your My Oracle Support account.
	Don't have a My Oracle Support account? Click to get started!

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.