Goldengate Integrated Replicat Abends With OGG-00665, ORA-03113 and Database Alert Log Has ORA-7445 [knanrBufProcessHeader()+2684]

(Doc ID 2326729.1)

Last updated on NOVEMBER 09, 2017

Applies to:

Oracle GoldenGate - Version 12.2.0.1.0 and later
Information in this document applies to any platform.

Symptoms

Whenever running Integrated Replicat and the same abends with following errors in the report file:

 

Source Context :
SourceModule : [ggdb.ora.sess]
SourceID : [/scratch/aime/adestore/views/aime_adc4150408/oggcore/OpenSys/src/gglib/ggdbora/ocisess.c]
SourceFunction : [oci_try]
SourceLine : [872]
ThreadBacktrace : [23] elements
: [/u03/app/ogg/product/ogg/12.2.1.0/libgglog.so(CMessageContext::AddThreadContext()+0x1b) [0x7fe49a3800bb]]
: [/u03/app/ogg/product/ogg/12.2.1.0/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x134) [0x7fe49a37a254]]
: [/u03/app/ogg/product/ogg/12.2.1.0/libgglog.so(_MSG_ERR_ORACLE_OCI_ERROR_WITH_DESC_SQL(CSourceContext*, int, char const*, char const*, char const*, CMessageFactory::MessageDisposition)+0x4e) [0x7fe49a36513a]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(OCISESS_context_def::oci_try(int, char const*, ...)+0x531) [0x7392c1]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(IROCI_flush(OCISESS_context_def*, unsigned int)+0x11d) [0x758abd]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(IROCI_flush_call(OCISESS_context_def*)+0x21) [0x758fa1]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(query_handle::select_start(bool, ggs::gglib::ggunicode::UString const&, short, ...)+0x97) [0x73c2b7]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(ORAUTIL_verifyDeferrableConstraint(ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1> const&, int*)+0xdd) [0x74e03d]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(DBUTIL_verifyDeferrableConstraint(ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1> const&, int*)+0x30) [0x659500]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(get_map_entry(ggs::gglib::ggunicode::UString const&, int, wc_def*, int, unsigned int, unsigned int, ggs::gglib::ggmetadata::CTblMetadata*, ggs::gglib::ggmetadata::CMetadataReader*, bool, bool, bool)+0x6d4) [0x5cdd84]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(wc_def::resolve_wc_entry(ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1> const&, int, unsigned int*, ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1>*, int, ggs::gglib::ggmetadata::CTblMetadata*, ggs::gglib::ggmetadata::CMetadataReader*)+0x1ee) [0x6a0f2e]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(WILDCARD_check_table(ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1> const*, int, unsigned int*, int, unsigned int, ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1>*, int, ggs::gglib::ggmetadata::CTblMetadata*, ggs::gglib::ggmetadata::CMetadataReader*, bool)+0x195) [0x6a1545]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(REP_find_source_file_wc(ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1> const&, unsigned int, ggs::gglib::ggapp::CQualDBObjName<(DBObjType)1>*, int, ggs::gglib::ggmetadata::CTblMetadata*, ggs::gglib::ggmetadata::CMetadataReader*, bool)+0x1ad) [0x5cb49d]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(source_file_lookup(ggs::gglib::gglcr::CommonLCR const*, ggs::gglib::ggmetadata::CMetadataReader*)+0x488) [0x5b0528]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(ggs::er::ReplicatContext::processReplicatLoop(ggs::Heartbeat::MapGeneratorParams&)+0xe16) [0x5f45f6]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(ggs::er::ReplicatContext::run(ggs::Heartbeat::MapGeneratorParams&)+0x29) [0x5e8af9]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat() [0x631b0b]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x60) [0x6e9e10]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x14d) [0x6ead9d]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0xb1) [0x6eae81]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat(main+0x3b) [0x634a5b]]
: [/lib64/libc.so.6(__libc_start_main+0xfd) [0x3713a1ed1d]]
: [/u03/app/ogg/product/ogg/12.2.1.0/replicat() [0x56b119]]

2017-11-09 12:07:25 ERROR OGG-00665 OCI Error Flushing database inbound server, 'OGG$REP2' (status = 3113-ORA-03113: end-of-file on communication channel
Process ID: 39813
Session ID: 318 Serial number: 28030), SQL<SELECT status, deferrable FROM dba_constraints WHERE owner =UPPER('XXXXXXXXX') and table_name=UPPER('XXXXXXXXX') and constraint_type = 'P' >.

 

On the database side, you will find following error in the alert log file and traces:

 

*** 2017-11-09 11:44:43.853
*** SESSION ID:(920.9726) 2017-11-09 11:44:43.853
*** CLIENT ID:() 2017-11-09 11:44:43.853
*** SERVICE NAME:(XXXXXXXX) 2017-11-09 11:44:43.853
*** MODULE NAME:(GoldenGate) 2017-11-09 11:44:43.853
*** CLIENT DRIVER:() 2017-11-09 11:44:43.853
*** ACTION NAME:(OGG$REP2- Apply Receiver) 2017-11-09 11:44:43.853

[TOC00000]
Jump to table of contents
Dump continued from file: /u01/app/diag/rdbms/xxxxxxx/XXXXXXXX/trace/XXXXXXXXX_ora_40343.trc
[TOC00001]
ORA-07445: exception encountered: core dump [knanrBufProcessHeader()+2684] [SIGSEGV] [ADDR:0x32] [PC:0x7A289EC] [Address not mapped to object] []

[TOC00001-END]
[TOC00002]
========= Dump for incident 297277 (ORA 7445 [knanrBufProcessHeader]) ========
[TOC00003]
----- Beginning of Customized Incident Dump(s) -----
Dumping swap information
Memory (Avail / Total) = 9350.57M / 258202.23M
Swap (Avail / Total) = 20479.99M / 20479.99M
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x32] [PC:0x7A289EC, knanrBufProcessHeader()+2684] [flags: 0x0, count: 1]
Registers:
%rax: 0x0000000000000000 %rbx: 0x00007fb1a1bc4f50 %rcx: 0x0000000000000001
%rdx: 0x0000000037313032 %rdi: 0x00007ffcb79a7e78 %rsi: 0x00007fb1a1986010
%rsp: 0x00007ffcb79a7e30 %rbp: 0x00007ffcb79a7e90 %r8: 0x4e97000500000000
%r9: 0x00000000000036fb %r10: 0x00000000004e9700 %r11: 0x00000000cf7ab6fb
%r12: 0x0000000927a46b80 %r13: 0x0000000000000000 %r14: 0x0000000a38422040
%r15: 0x00007fb1a2c8d260 %rip: 0x0000000007a289ec %efl: 0x0000000000010282
knanrBufProcessHeader()+2659 (0x7a289d3) addq $4,0x19dc8(%rbx)
knanrBufProcessHeader()+2667 (0x7a289db) call 0xcddbe60
knanrBufProcessHeader()+2672 (0x7a289e0) mov %rax,%r13
knanrBufProcessHeader()+2675 (0x7a289e3) andb $0xf7,0x98(%r12)
> knanrBufProcessHeader()+2684 (0x7a289ec) movzbl 0x32(%r13),%esi
knanrBufProcessHeader()+2689 (0x7a289f1) mov %si,-0xc(%rbp)
knanrBufProcessHeader()+2693 (0x7a289f5) mov 0x88(%r12),%rdi
knanrBufProcessHeader()+2701 (0x7a289fd) test %rdi,%rdi
knanrBufProcessHeader()+2704 (0x7a28a00) je 0x7a28a06

*** 2017-11-09 11:44:43.867
dbkedDefDump(): Starting a non-incident diagnostic dump (flags=0x3, level=3, mask=0x0)
[TOC00004]
----- SQL Statement (None) -----
Current SQL information unavailable - no cursor.
[TOC00004-END]

[TOC00005]
----- Call Stack Trace -----

----- Abridged Call Stack Trace -----
ksedsts()+244<-kjzdssdmp()+321<-kjzduptcctx()+692<-kjzdpcrshnfy()+347<-kstdmp()+352<-dbkedDefDump()+8728<-ksedmp()+261<-ssexhd()+2650<-sslsshandler()+456<-__sighandler()<-knanrBufProcessHeader()+2684<-knanrRcvLCRBuffer()+1729<-knanrHandleSendLCR()+4042<-rpiswu2()+1780
<-knipswu()+299<-knanrdis()+6350<-opiodr()+1165<-ttcpip()+2699<-opitsk()+1740<-opiino()+945
----- End of Abridged Call Stack Trace -----

 



Changes

 This may happen usually when running GG IR 12.2 with Sharded Tables.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms