My Oracle Support Banner

ExaCS: EPSU update stuck waiting on RHPhelper process (Doc ID 2759510.1)

Last updated on JANUARY 26, 2022

Applies to:

Oracle Cloud Infrastructure - Exadata Cloud Service - Version N/A to N/A [Release N/A]
Information in this document applies to any platform.

Symptoms

EPSU update is hung for hours with no progress and no activity written to the logs.

+ /var/log/cellos/dbnodeupdate.log shows the process is waiting  waiting on RHPhelper.

 

.....
[1613599142][2021-02-17 22:03:27 +0000][INFO][/u01/dbnodeupdate.patchmgr/dbnodeupdate.sh][PrintMsg][]    (ACTION:) Executing RHPhelper to drain sessions and
shutdown instances. (trace: /u01/app/grid/crsdata/oraxyz1-qyrfn2/rhp//executeRHPDrain.170221213839.trc)

+ RHPhelper  trace file shows it has already failed.

...
[pool-1-thread-2] [ 2021-02-17 22:03:30.370 UTC ] [EntityOperations.requestAction:975]  actionName is serviceDrain
[pool-1-thread-2] [ 2021-02-17 22:03:30.371 UTC ] [Utils.getString:259]  ==========Str is ora.gbxyz01_iad392.chubd_high.svc
[pool-1-thread-2] [ 2021-02-17 22:03:30.381 UTC ] [ActionResultImpl.getFinalMessage:81]  no final message received for node oraxyz1-qyrfn2
[pool-1-thread-2] [ 2021-02-17 22:03:30.383 UTC ] [DBServicesSelectionImpl.waitForDrainCompletion:1961]  PRCD-1349 : failed to get the status of draining the
 services ora.gbxyz01_iad392.chubd_high.svc of database ora.gbxyz01_iad392.db
PRCR-1182 : Failed to request action serviceDrain on resource ora.gbxyz01_iad392.db on nodes oraxyz1-qyrfn2
oraxyz1-qyrfn2: PRCR-1187 : No final message for the requested action was received.   <<<<<<<<
oracle.cluster.impl.database.DatabaseAction.drainAction(DatabaseAction.java:717)
oracle.cluster.impl.database.DBServicesSelectionImpl.waitForDrainCompletion(DBServicesSelectionImpl.java:1863)
oracle.cluster.impl.database.DBServicesSelectionImpl.relocateAndStop(DBServicesSelectionImpl.java:1324)
oracle.cluster.gridhome.giprov122.RHPHelper122$SvcRelocationParallelOp$SvcRelocationCommand.execSvcRelocation(RHPHelper122.java:3931)
oracle.cluster.gridhome.giprov122.RHPHelper122$SvcRelocationParallelOp$SvcRelocationCommand.execute(RHPHelper122.java:3884)
oracle.cluster.impl.concurrency.ParallelCommandImpl$CallableCmdExecutor.call(ParallelCommandImpl.java:355)
oracle.cluster.impl.concurrency.ParallelCommandImpl$CallableCmdExecutor.call(ParallelCommandImpl.java:334)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
[pool-1-thread-2] [ 2021-02-17 22:03:30.384 UTC ] [ParallelCommandImpl$CallableCmdExecutor.call:356]  after executing command for node 122 return=false

 

Changes

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.