My Oracle Support Banner

ODA:APEX Instance Startup Failed with errors CRS-5011: Check of resource "ora.proxy_advm" And ORA-03113 After GI Upgrade to 18.8 (Doc ID 2694020.1)

Last updated on APRIL 17, 2023

Applies to:

Oracle Database Appliance Software - Version 18.8.0.0.0 Bare Metal and later
Information in this document applies to any platform.

Symptoms

<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/alert.log
============
2020-07-22 05:19:26.765 [ORAAGENT(16839)]CRS-5017: The resource action "ora.proxy_advm start" encountered the following error:
2020-07-22 05:19:26.765+ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc".
2020-07-22 05:19:26.767 [ORAAGENT(16839)]CRS-5011: Check of resource "ora.proxy_advm" failed: details at "(:CLSN00006:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc"
2020-07-22 05:19:26.769 [ORAAGENT(16839)]CRS-5011: Check of resource "ora.proxy_advm" failed: details at "(:CLSN00006:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc"
2020-07-22 05:19:53.152 [ORAAGENT(16839)]CRS-5017: The resource action "ora.proxy_advm start" encountered the following error:
2020-07-22 05:19:53.152+ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc".
2020-07-22 05:19:53.154 [ORAAGENT(16839)]CRS-5011: Check of resource "ora.proxy_advm" failed: details at "(:CLSN00006:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc"
2020-07-22 05:19:53.156 [ORAAGENT(16839)]CRS-5011: Check of resource "ora.proxy_advm" failed: details at "(:CLSN00006:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc"
========
<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc
========
020-07-22 05:19:23.460 :CLSDYNAM:1436346112: [ ora.asm]{1:36177:2} [check] InstAgent::check 180 agent restart start completed continuing check
2020-07-22 05:19:24.465 :CLSDYNAM:1438447360: [ ora.asm]{1:36177:2} [check] InstAgent::check 180 agent restart start completed continuing check
2020-07-22 05:19:25.467 :CLSDYNAM:1438447360: [ ora.asm]{1:36177:2} [check] InstAgent::check 180 agent restart start completed continuing check
2020-07-22 05:19:26.468 :CLSDYNAM:1438447360: [ ora.asm]{1:36177:2} [check] InstAgent::check 180 agent restart start completed continuing check
2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0

2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] InstAgent::startInstance 050 errcode 3113
2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] InstAgent::startInstance 250 untrapped error
2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] InstAgent::startInstance 350 errcode 3113
2020-07-22 05:19:26.765 : USRTHRD:4158605056: {1:36177:12165} InstConnection:~InstConnection destructor this 0x7f79f006af30
2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] InstAgent::startInstance 380 throw excp what:ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0

2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] InstAgent::start 500 exception OCIException
2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0

2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] Agent::commonStart Exception OCIException
2020-07-22 05:19:26.765 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] clsnUtils::error Exception type=2 string=
CRS-5017: The resource action "ora.proxy_advm start" encountered the following error:
ORA-03113: end-of-file on communication channel
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "<ORACLE_BASE>/diag/crs/<HOSTNAME>/crs/trace/crsd_oraagent_grid.trc".

2020-07-22 05:19:26.766 :    AGFW:1451632384: {1:36177:12165} Agent sending reply for: RESOURCE_START[ora.proxy_advm <HOSTNAME> 1] ID 4098:43748
2020-07-22 05:19:26.766 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [start] (:CLSN00107:) clsn_agent::start }
2020-07-22 05:19:26.766 :    AGFW:4158605056: {1:36177:12165} Command: start for resource: ora.proxy_advm <HOSTNAME> 1 completed with status: FAIL
2020-07-22 05:19:26.767 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [check] InstAgent::checkState 030 new gimh oracleSid:+APX1
2020-07-22 05:19:26.767 :    AGFW:1451632384: {1:36177:12165} Agent sending reply for: RESOURCE_START[ora.proxy_advm <HOSTNAME> 1] ID 4098:43748
2020-07-22 05:19:26.767 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [check] ConnectionPool::resetConnection  s_statusOfConnectionMap:0x3dcecf8
2020-07-22 05:19:26.767 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [check] ConnectionPool::resetConnection sid +APX1 status  2
2020-07-22 05:19:26.767 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [check] Gimh::check condition changes to (GIMH_NEXT_NUM) 0(Abnormal Termination) exists
2020-07-22 05:19:26.767 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [check] CLS_DRUID_REF(CLSN00006) AsmAgent::gimhChecks 100 failed gimh state 0
2020-07-22 05:19:26.768 :CLSDYNAM:4158605056: [ora.proxy_advm]{1:36177:12165} [check] ConnectionPool::stopConnection
============

<ORACLE_BASE>//diag/apx/+apx/+APXx/trace/alert_+APXx.log
=======================

2020-07-22T05:19:51.365013+02:00
VBG0 started with pid=16, OS id=75396
Starting background process VDBG
2020-07-22T05:19:51.380539+02:00
Error: ASM Volume driver has incompatible version:
     Driver version - 12.2.0.1.0
     ASM version    - 18.0.0.0.0
2020-07-22T05:19:51.380754+02:00
ERROR: VBG process cannot open VBG control file
2020-07-22T05:19:51.380940+02:00
VDBG started with pid=17, OS id=75401
Starting background process VUBG
Starting background process ACFS
2020-07-22T05:19:51.397400+02:00
VUBG started with pid=18, OS id=75409
2020-07-22T05:19:51.418312+02:00
ACFS started with pid=19, OS id=75413
2020-07-22T05:19:51.426869+02:00
ACFS ERROR: Send message: Send failed err 5
2020-07-22T05:19:53.145484+02:00
PMON (ospid: 75291): terminating the instance due to ORA error 15478
Cause - 'Instance is being terminated due to fatal process death (pid: 19, ospid: 75413, ACFS)'
2020-07-22T05:19:53.146239+02:00
System state dump requested by (instance=1, osid=75291 (PMON)), summary=[abnormal instance termination].
System State dumped to trace file <ORACLE_BASE>/diag/apx/+apx/+APX1/trace/+APX1_diag_75372_20200722051953.trc
2020-07-22T05:19:53.189341+02:00
Dumping diagnostic data in directory=[cdmp_20200722051953], requested by (instance=1, osid=75291 (PMON)), summary=[abnormal instance termination].
2020-07-22T05:19:54.180099+02:00
License high water mark = 1
2020-07-22T05:19:54.249701+02:00
Instance terminated by PMON, pid = 75291
2020-07-22T05:19:54.250336+02:00
Warning: 2 processes are still attacheded to shmid 720900:
 (size: 8192 bytes, creator pid: 75166, last attach/detach pid: 75365)
2020-07-22T05:19:55.180528+02:00
USER (ospid: 75518): terminating the instance
2020-07-22T05:19:55.180773+02:00
Instance terminated by USER, pid = 75518

 

Changes

 Upgraded GI from 12.2.0.1 to 18.8

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

In this Document
Symptoms
Changes
Cause
Solution


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.