Accessing Dbfs Using Oracle Wallet Fails With Input/output error (Doc ID 1358194.1)

Last updated on APRIL 09, 2020

Applies to:

Symptoms

After configuring DBFS to use Oracle Wallet, to store the password, via "Configuring DBFS on Oracle Database Machine (Doc ID 1054431.1)" or by following DBFS documentation in Oracle® Database SecureFiles and Large Objects Developer's Guide 11g Release 2 (11.2), Part Number E18294-01 , the /dbfs directory mounts but when trying to access directory in any way you get "Input/output error", i.e.

# df -k /dbfs
df: `/dbfs': Input/output error

# ls -dl /dbfs
ls: /dbfs: Input/output error

Bringing dbfs_mount online, which will mount the file system on all nodes, via "crsctl start resource dbfs_mount" from any cluster node shows no errors.

(oracle)$ crsctl start resource dbfs_mount
CRS-2672: Attempting to start 'dbfs_mount' on 'host05'
CRS-2672: Attempting to start 'dbfs_mount' on 'host06'
CRS-2676: Start of 'dbfs_mount' on 'host06' succeeded
CRS-2676: Start of 'dbfs_mount' on 'host05' succeeded

The "crsctl stat res dbfs_mount -t" confirms that dbfs is online

# oracle)$ crsctl stat res dbfs_mount -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
dbfs_mount
                             ONLINE ONLINE host05
                             ONLINE ONLINE host06

By setting trace parameter as part of the mount command we find that accessing /dbfs directory results in "ORA-01017: invalid username/password; logon denied".

To turn on tracing, modify /etc/fstab to have following mount command and remount /dbfs via "crsctl stop/start resource dbfs_mount" command

/sbin/mount.dbfs#/@dbfs_mount.local /dbfs fuse rw,user,direct_io,allow_other,wallet,trace_file=/tmp/dbfs_mount.trc,trace_level=1,trace_size=0,noauto 0 0

After running "df -k /dbfs" command and getting error you will see following in /tmp/dbfs_mount.trc

[5bbf7940 09/09/11 11:24:42.818800 LcdfLock.cpp:230 ] DEBUG: Global Lock - unlock
[5bbf7940 09/09/11 11:24:42.818806 LcdfLock.cpp:242 ] DEBUG: Global Lock (R) rdwr:0
[45bec940 09/09/11 11:24:42.857258 LcdfDBPool.cpp:398 ] ERROR: ERROR 1017 - ORA-01017: invalid username/password; logon denied

[45bec940 09/09/11 11:24:42.857287 LcdfJob.cpp:304 ] DEBUG: Exit LcdfJob::DoJobInt job:0x2aaaac0bc0e0 op(1)

Cause