ASAP Is Going Down After Switching Oracle RAC Database From One RAC Node To Another (Doc ID 1944319.1)

Last updated on APRIL 18, 2016

Applies to:

Oracle Communications ASAP - Version 7.0.0 to 7.2.0 [Release 7.0.0 to 7.2]
Information in this document applies to any platform.

Symptoms

We have configured RAC for ASAP with two DB Nodes (Ex. Instance_1 and Instance_2). The requirement is one DB node will be always down like active and passive.
The problem we are facing is ASAP is going down after shifting database from one node to another. Every time we need to restart ASAP manually when switching database Nodes.


The following error messages appeared in ASAP diag files:


CTRL diag:

>> 215843.982:39: LOW:release_cda_oci8 :3167:control/libasc/ctlib.c
Purging Stmt: 0x92b3530; deleting CDA_OCI8: 0x9368d00
>> 215843.982:39: LOW:ASC_cprpcexec() :3105:control/libasc/ctlib.c
Successfully established a new connection
>> 215843.982:39: LOW:ASC_cprpcexec() :1244:control/libasc/ctlib.c
Executing RPC [CSP_system_event]. Retry Count [1]
>> 215844.761:39: LOW:ASC_cprpcexec() :1446:control/libasc/ctlib.c
RPC CSP_system_event return status = -1
>> 215844.761:39:PROG:checkerr_returning_errcod:1658:control/libasc/ocilib.c
Error - ORA-03113: end-of-file on communication channel
Process ID: 53625
Session ID: 4403 Serial number: 3
errcode[3113]
>> 215844.762:39:SANE:ASC_cprpcexec() :3088:control/libasc/ctlib.c
Error: DB Connection broken down due to Error[3113]
>> 215844.762:39: LOW:ASC_cprpcexec() :3105:control/libasc/ctlib.c
Successfully established a new connection
Successfully obtained a Session from the Pool
--------------------------------------------

 

SARM diag:

Oct 16 23:16:07 2014: ASAP System Event: UNIX_ERR
Internet Socket Connect: Socket: [78] Can't Connect to Host [xxx] Error: [115 Operation now in progress]
Oct 16 23:16:17 2014: Error - ORA-03113: end-of-file on communication channel
Process ID: 33416
Session ID: 6912 Serial number: 363
errcode[3113]
Oct 16 23:16:17 2014: Error: OCIStmtExecute [ORA-3113] Failure...

Oct 16 23:16:17 2014: RPC SSP_get_wo_srq failed. Error code:[3113]
Oct 16 23:16:18 2014: ASAP System Event: SYS_TERM
Error: SSP_get_wo_srq Failed
Oct 16 23:16:18 2014: ASAP System Event: SYS_TERM
Error: Lost Connection to Control Server 'CTRL2DNZ', Terminating....
Oct 16 23:16:18 2014: ASAP System Event: SYS_TERM
Error: Lost Connection to Control Server 'CTRL2DNZ', Terminating....
Oct 16 23:16:18 2014: SARM2DNZ: Msg 6 Layer 5 Origin 3 Severity 5
'ct_results(): network packet layer: internal net library error: Net-Library operation terminated due to disconnect'
Oct 16 23:16:18 2014: RPC(log_event) Failed in 384ms, First Result in 0ms
Oct 16 23:16:18 2014: SARM2DNZ: Msg 6 Layer 5 Origin 3 Severity 5
'ct_cancel(): network packet layer: internal net library error: Net-Library operation terminated due to disconnect'
Oct 16 23:16:18 2014: Error: RPC 'log_event' Failed
-----------------------------

ASAP.Console

SRP_2DNZ Warning: /asap/asap724/config/ASAP.cfg (line 284) is referencing an undefined variable [$SERVER_IPADDR].  Using NULL instead.
SRP_2DNZ Warning: /asap/asap724/config/ASAP.cfg (line 285) is referencing an undefined variable [$SERVER_PORT].  Using NULL instead.
Application [SRP_2DNZ] is running
Diagnostic.exitHandler() : Terminating ...
Diagnostic.exitHandler() : Terminating ...
JInterp.exitHandler() : Closing DB Pools ...
Listener.exitHandler() : Terminating ...


Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms