RHP Server Start Fails With Error: CRS-8503 [_ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_()+30] (Doc ID 2570570.1)

 [+ASM1]-> srvctl start rhpserver -node node1
PRCR-1064 : Failed to start resource ora.rhpserver on node node1
CRS-2674: Start of 'ora.rhpserver' on 'node1' failed
CRS-5804: Communication error with agent process

019-07-18 17:29:31.364 : USRTHRD:322221824: [ INFO] {0:104:2} Thread:[VipRelocate:] isFinished set to true
CLSB:97117952: [ INFO] Oracle Clusterware infrastructure error in ORAROOTAGENT (OS PID 98082): Fatal signal 11 has occurred in program orarootagent thread 97117952; nested signal count is 1
Trace file $ORACLE_BASE/diag/crs/node1/crs/trace/crsd_orarootagent_root.trc
Oracle Database 19c Clusterware Release 19.0.0.0.0 - Production
Version 19.3.1.0.0 Copyright 1996, 2019 Oracle. All rights reserved.
DDE: Flood control is not active
2019-07-18T17:29:32.254578+00:00
Incident 785 created, dump file: $ORACLE_BASE/crs/node1/crs/incident/incdir_785/crsd_orarootagent_root_i785.trc
CRS-8503 [_ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_()+30] [Signal/Exception: 11] [Instruction Addr: 0x55d3bb5783e6] [Memory Addr: (nil)] [] [] [] [] [] [] [] []
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] stop { m_pThnd:0x7fb7b40d25f0 tid:322221824 running:0 alive:0
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread stop:thread is not running:0 or alive:0
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread stop:thread acquire m_pThnd:0x7fb7b40d25f0 m_thndMX:0x7fb79809abf0
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread stop:thread sltstjn
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] stop release m_thndMX:0x7fb79809abf0 m_tid:322221824}
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread::start { acquire thndMX:9809abf0
2019-07-18 17:29:32.333 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread::start spawn pThnd:0x7fb78404d000 thndType:1
2019-07-18 17:29:32.334 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread::start thread spawned tid:322221824
2019-07-18 17:29:32.334 : USRTHRD:2346706688: [ INFO] {0:104:2} Thread:[VipRelocate:] Thread::start spawned release thndMX:9809abf0 }
2019-07-18 17:29:32.357 : USRTHRD:322221824: [ INFO] {0:104:2} Thread:[VipRelocate:] isRunning is reset to false here
2019-07-18 17:29:32.357 : USRTHRD:322221824: [ INFO] {0:104:2} Thread:[VipRelocate:] isFinished set to true

 

Dump file $ORACLE_BASE/crs/<node1>/crs/incident/incdir_713/crsd_orarootagent_root_i713.trc
[TOC00000]
Oracle Database 19c Clusterware Release 19.0.0.0.0 - Production
Version 19.3.1.0.0 Copyright 1996, 2019 Oracle. All rights reserved.
Jump to table of contents
Dump continued from file: $ORACLE_BASE/crs/<node1>/crs/trace/crsd_orarootagent_root.trc
[TOC00001]
CRS-8503 [_ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_()+30] [Signal/Exception: 11] [Instruction Addr: 0x55e8bc7123e6] [Memory Addr: (nil)] [] [] [] [] [] [] [] []
[TOC00001-END]
[TOC00002]
========= Dump for incident 713 (CRS 8503 [_ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_()+30]) ========
Starting a Diag Context default dump (level=3)

..

Number of actions: 9
----- Incident Context Dump -----
Address: 0x7f86b0084db8
Incident ID: 713
Problem Key: CRS 8503 [_ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_()+30]
Error: CRS-8503 [_ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_()+30] [Signal/Exception: 11] [Instruction Addr: 0x55e8bc7123e6] [Memory Addr: (nil)] [] [] [] [] [] [] [] []
[00]: dbgexProcessError [diag_dde]
[01]: dbgePostErrorDirectVaList_int [diag_dde]
[02]: dbgePostErrorDirect [diag_dde]
[03]: clsdAdrPostError []
[04]: clsdadrpr_CreateIncidentCheck []
[05]: clsdadrprAlert []
[06]: clsd_malertprintftSig []
[07]: clsbSigErrCB []
[08]: skgesig_sigactionHandler []
[09]: __sighandler []
[10]: _ZN8cls_agfw4Main12getAttrValueEPK13clsagfw_aectxPKcPS5_ []<-- Signaling
[11]: _ZN10clsn_agent12getResAttribEPK13clsagfw_aectxPKhRSsij []
[12]: _ZN10clsn_agent19getOracleHomeAttribEPK13clsagfw_aectxj []
[13]: _ZN10clsn_agent8UsmUtils7execCmdEPK13clsagfw_aectxPcPNS_5AgentENS_10ActionTypeEPS4_jbbb []
[14]: _ZN10clsn_agent18cleanupOldAcfsLogsESs []
[15]: _ZN10clsn_agent17CaptureLogsThread14captureOKSLogsEPKcSs18clsagfw_res_statusS3_Ss []
[16]: _ZN10clsn_agent17CaptureLogsThread3runEv []
[17]: _ZN10clsn_agent6Thread10threadFuncEPv []
[18]: start_thread []

2019 13:48:11.583 INFO [RMI TCP Connection(978)-30.162.174.133] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["https-jsse-nio-8894"]
18-Jul-2019 13:48:11.590 INFO [RMI TCP Connection(978)-30.162.174.133] org.apache.catalina.core.StandardService.stopInternal Stopping service [Catalina]
18-Jul-2019 13:48:11.599 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [rhp] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
java.util.TimerThread.mainLoop(Timer.java:526)
java.util.TimerThread.run(Timer.java:505)
18-Jul-2019 13:48:11.599 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [rhp] appears to have started a thread named [oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleaser] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
oracle.jdbc.driver.BlockSource$ThreadedCachingBlockSource$BlockReleaser.run(BlockSource.java:331)
18-Jul-2019 13:48:11.600 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [rhp] appears to have started a thread named [InterruptTimer] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:552)
java.util.TimerThread.run(Timer.java:505)