rootcrs.pl -postpatch/-patch hangs as "acfsroot install" stalls

(Doc ID 2037801.1)

Last updated on JANUARY 30, 2018

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.3 and later
Information in this document applies to any platform.

Symptoms

opatchauto/"opatch auto" hangs when applying Grid Infrasturcture patch including PSU during patch/postpatch phase:

/oracle/crs/12102/grid/crs/install/rootcrs.pl -postpatch <<<<<<<<< is hanging

crspatch_<node>_<time>.log

2015-07-22 14:42:40: acfs is supported
2015-07-22 14:42:40: Executing cmd: /oracle/crs/12102/grid/bin/crsctl stat res ora.drivers.acfs -init
2015-07-22 14:42:40: Command output:
>  CRS-4639: Could not contact Oracle High Availability Services
>  CRS-4000: Command Status failed, or completed with errors.
>End Command output
2015-07-22 14:42:40: Executing '/oracle/crs/12102/grid/bin/acfsroot install'
2015-07-22 14:42:40: Executing cmd: /oracle/crs/12102/grid/bin/acfsroot install

 

--------------After fixing the cause of failure you have two options shown below:
Run 'opatchauto resume'
  or
Manually run the commands listed below
---------------------------------------------------------------------------------

 

Truss of "acfsroot install" shows it's waiting for the following:

waitid(P_PID, 9982, 0xFFFFFFFF7FFFF160, WEXITED|WTRAPPED) (sleeping...)

 

From "ps" output, pid 9982 is modunload:

# ps -ef | grep 9982
root 9982 1 0 08:56:12 ? 0:00 modunload -i 301

# /usr/sbin/modinfo | grep -i oracle
299 108c48c0 567c0 - 1 oracleoks (Oracle Kernel Services module)
300 109168b0 8c6d0 309 1 oracleadvm (ADVM driver)
301 7c002000 367de8 29 1 oracleacfs (Oracle ASM Cluster File System)
301 7c002000 367de8 310 1 oracleacfs (Oracle ASM Cluster File System)

 

From "fuser" command output:

# fuser -d /dev/ksyms
/dev/ksyms: 1737o 1650o


From "ps" output, pid 1737 and 1650 are HPOvGlanc related:

# ps -ef | grep 1737
root 1746 1737 0 Nov 04 ? 17:20 /opt/perf/bin/perf64
root 1737 1 0 Nov 04 ? 56:55 /opt/perf/bin/perfd =====> (from HPOvGlanc, HP Software Performance Glance)

# ps -ef | grep 1650
root 1661 1650 0 Nov 04 ? 34:27 /opt/perf/bin/perf64
root 1650 1 0 Nov 04 ? 86:41 /opt/perf/bin/scopeux ===> (from HPOvPerfAgt,HP Software Performance Agent)

 

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms