[OCI]: Database System Patching Failed With Error "DCS-10001:Internal Error Encountered: Failure : Failed To Apply" And Opatch Log Shows "Prerequisite check "CheckActiveFilesAndExecutables" failed" (Doc ID 2687607.1)

Last updated on JULY 20, 2020

Applies to:

Symptoms

OCI Cloud Database system patching fails with errors as below:
==============================
# dbcli describe-job -i 979e7cfa-c4a3-4924-a72b-c3d1b755b663

Job details
----------------------------------------------------------------
ID: 979e7cfa-c4a3-4924-a72b-c3d1b755b663
Description: Server Patching
Status: Failure
Created: June 26, 2020 3:46:51 PM GMT
Progress: 49%
Message: DCS-10001:Internal error encountered: Failure : failed to apply 30869156 on /u01/app/19.0.0.0/grid.

Task Name Start Time End Time Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
Server Patching June 26, 2020 3:46:51 PM GMT June 26, 2020 4:00:03 PM GMT Failure
Pre-operations for Server Patching June 26, 2020 3:46:51 PM GMT June 26, 2020 3:47:53 PM GMT Success
Server Patching June 26, 2020 3:47:53 PM GMT June 26, 2020 4:00:03 PM GMT Failure
task:TaskSequential_2135 June 26, 2020 3:47:53 PM GMT June 26, 2020 4:00:03 PM GMT Failure
task:TaskSequential_2163 June 26, 2020 3:50:32 PM GMT June 26, 2020 4:00:03 PM GMT Failure
task:TaskSequential_2164 June 26, 2020 3:50:32 PM GMT June 26, 2020 4:00:03 PM GMT Failure
apply clusterware patch[dr0j010ghk] June 26, 2020 3:50:32 PM GMT June 26, 2020 4:00:03 PM GMT Failure
=============================

From /u01/app/19.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch<timestamp>.log
===================
[Jun 25, 2020 12:21:42 PM] [INFO] Finish fuser command /sbin/fuser /u01/app/19.0.0.0/grid/lib/libasmclntsh19.so at Thu Jun 25 12:21:42 GMT 2020
[Jun 25, 2020 12:21:42 PM] [INFO] Files in use by a process: /u01/app/19.0.0.0/grid/lib/libasmclntsh19.so PID( 31295 31321 31323 31361 31419 31440 31546 32002 32007 32011 32015 32022 32028 32036 32043 32047 32051 32055 32059 32063 32065 32071 32080 32084 32088 32092 32096 32100 32104 32108 32112 32116 32120 32124 32129 32133 32137 32146 32310 32314 32318 32323 32354 32424 32678 33092 33102 33139 33166 33214 33243 33551 33565 33572 33642 33648 33652 33670 33684 33688 33692 33696 33701 33714 33718 33727 33731 33739 34019 34041 34042 34075 34085 34091 34093 34098 34099 34106 44773 )

[Jun 25, 2020 12:21:43 PM] [INFO] Following active executables are not used by opatch process :
/u01/app/19.0.0.0/grid/bin/oracle
/u01/app/19.0.0.0/grid/bin/tnslsnr
/u01/app/19.0.0.0/grid/lib/libclntsh.so.19.1
/u01/app/19.0.0.0/grid/lib/libasmclntsh19.so

Following active executables are used by opatch process :
[Jun 25, 2020 12:21:43 PM] [INFO] Prerequisite check "CheckActiveFilesAndExecutables" failed.
The details are:

Following active executables are not used by opatch process :
/u01/app/19.0.0.0/grid/bin/oracle
/u01/app/19.0.0.0/grid/bin/tnslsnr
/u01/app/19.0.0.0/grid/lib/libclntsh.so.19.1
/u01/app/19.0.0.0/grid/lib/libasmclntsh19.so

Following active executables are used by opatch process :
[Jun 25, 2020 12:21:44 PM] [SEVERE] OUI-67073:UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.
[Jun 25, 2020 12:21:44 PM] [INFO] Finishing UtilSession at Thu Jun 25 12:21:44 GMT 2020
[Jun 25, 2020 12:21:44 PM] [INFO] Log file location: /u01/app/19.0.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-06-25_12-19-45PM_1.log
===========================

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.