Alert: Exadata/ODA/RAC - DBFS fails to mount post Linux upgrade to OL7 UEK5 Kernel (Doc ID 2651490.1)

Last updated on JANUARY 21, 2022

Applies to:

Symptoms

After applying Exadata Jan2020 QFSD Patch, the DBFS Filesystem fails to mount on any of the Compute Nodes.

Note: This even applies to RAC/ODA Environment where the Linux kernel is upgraded to UEK5 Kernel.

uname -a
Linux xxxxdb01 4.14.35-1902.9.2.el7uek.x86_64 #2 SMP Mon Dec 23 13:39:16 PST 2019 x86_64 x86_64 x86_64 GNU/Linux

The dbfs_mount Resource is OFFLINE on all Compute Nodes:

$ crsctl stat res dbfs_mount -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ONLINE OFFLINE xxxxdb01 STABLE
ONLINE OFFLINE xxxxdb02 STABLE
ONLINE OFFLINE xxxxdb03 STABLE
ONLINE OFFLINE xxxxdb04 STABLE
--------------------------------------------------------------------------------

Ran the mount-dbfs.sh script with debugging option:

$sh -x mount-dbfs.sh

...
+ nohup <RDBMS_HOME>/bin/dbfs_client  <db_user>@<server> -o allow_other,direct_io,numa,wallet  /dbfs_direct/dbfs
+ /bin/logger -t DBFS_/dbfs_direct/dbfs -p user.info
nohup: redirecting stderr to stdout
+ /bin/rm -f /tmp/.dbfs-xxx.txt.388589
+ <GRID_HOME>/crs/script/./mount-dbfs.sh status
+ '[' 1 -eq 0 ']'
+ logit info 'Start -- OFFLINE'
+ type=info
+ msg='Start -- OFFLINE'
+ '[' info = info ']'
+ /bin/echo Start -- OFFLINE <<<<<<<<<<<

Changes

Exadata JAN2020 QFSD Applied.

Exadata Image version upgraded: From 18.1.21.0.0 to 19.3.4.0.0.200130

Linux O/S upgraded : From OL6 ( UEK4 Kernel: 4.1.12-94.7.8.el6uek.x86_64)  to OL7 ( UEK5 Kernel: 4.14.35-1902.9.2.el7uek.x86_64).

Cause