Pipeline Crashes While Freeing Memory In DAT_AccountBatch Plugin (Doc ID 1221903.1)

Last updated on SEPTEMBER 19, 2016

Applies to:

Oracle Communications Billing and Revenue Management - Version 7.4.0.0.0 to 7.4.0.0.0 [Release 7.4.0]
Information in this document applies to any platform.
***Checked for relevance on 24-Sep-2013***
***Checked for Relevance on 03-April-2015***

Symptoms

While processing traffic in high volumes, the pipeline process terminates unexpectedly with a core dump, characterized by the following stack trace, apparently crashing while freeing memory in DAT_AccountBatch plugin:

Sample pstack output:

----------------- lwp# 231 / thread# 231 --------------------
ffffffff6a245a08 __SLIP.FINAL__S (ffffffff78549cc0, ffffffff785480c0, 1c00, ffffffff7327f6bc, 0, 0)
ffffffff78339dec exit (1, 6, 1, 17c698, ffffffff72c5b21c, ffffffff7853c000) + 4
ffffffff7327f6bc skgdbgcra (2, ffffffff327fd61c, 1, ffffffff73e3b9a0, ffffffff327fd618, ffffffff7410b5a0) + 15c
ffffffff7327b1f4 skgesigCrash (ffffffff327fdf98, ffffffff7410a070, 1a0c00, ffffffff73fdbc40, d60a7c, 1) + 34
ffffffff7327b81c skgesig_sigactionHandler (ffffffff736fb9a0, ffffffff327fedf0, ffffffff327fdf80, ffffffff7410b578, ffffffff327fdf88, ffffffff327fdf98) + dc
ffffffff783d61b8 __sighndlr (6, ffffffff327fedf0, ffffffff327feb10, ffffffff7327b740, 0, 5) + c
ffffffff783c9d1c call_user_handler (ffffffff6db13440, ffffffff6db13440, ffffffff327feb10, 8, 0, 0) + 3e0
ffffffff783c9f14 sigacthandler (0, ffffffff327fedf0, ffffffff327feb10, ffffffff6db13440, 0, ffffffff7853c000) + 54
--- called from signal handler with signal 0 (SIGEXIT) ---
ffffffff783da100 _lwp_kill (6, 0, 112d58, ffffffff7f413e50, ffffffff7f524000, ffffffff7f525b00) + 8
ffffffff7f41134c umem_do_abort (80, ffffffff7f41efd8, 6, 112c7c, 11, 0) + 1c
ffffffff7f4114f0 umem_err_recoverable (ffffffff7f420190, a, 112b74, ffffffff783cecc8, ffffffff7f52848c, a) + 7c
ffffffff7f4138f8 process_free (16c9dde3d0, 16ba7000, 0, ffffffff711a1f3c, 11090c, 0) + 210
ffffffff788078fc __1c2k6Fpv_v_ (16c9dde3d0, 16c9dde9d0, 16ca282138, ffffffff70d1c7e4, 775bd8, 16ca282138) + 4
ffffffff7115a820 __1cHRW_PSeq4nDstdGvector4CpnDDATHAccountSPurchasedProductSD_n0AJallocator4C4____nTRWTPtrOrderedVector4n0D_n0E___n0D__PclearAndDestroy6M_v_ (141400ba8, 16c9dde
9d0, 16ca282100, ffffffff70d1c354, ffffffff71491bb8, 16c9dde3d0) + 3c
ffffffff71155600 __1cDDATHAccountOSearchDataImplRreturnItemsToPool6MrnTRWTPtrOrderedVector4n0BSPurchasedProductSD_nDstdJallocator4Cpn0D______v_ (141400ad0, 141400ba8, 0, 16ca81
7dd0, ffffffff71195f20, 141400e90) + 74
[...]


Also the Dtrace log shows similar information, prividing clear-text function names:

libc.so.1`_lwp_kill+0x4
libumem.so.1`umem_do_abort+0x1c
libumem.so.1`umem_err_recoverable+0x7c
libumem.so.1`process_free+0x210
libCrun.so.1`void operator delete(void*)+0x4
libDAT_AccountBatch64.so`void
RW_PSeq<std::vector<DAT::Account::PurchasedProductSD*>,RWTPtrOrderedVector<DAT::Account::PurchasedProductSD,
std::allocator<DAT::Account::PurchasedProductSD*>>,
DAT::Account::PurchasedProductSD>::clearAndDestroy()+0x3c
libDAT_AccountBatch64.so`void
DAT::Account::SearchDataImpl::returnItemsToPool(RWTPtrOrderedVector<DAT::Account::PurchasedProductSD,
std::allocator<DAT::Account::PurchasedProductSD*>>&)+0x74
libDAT_AccountBatch64.so`void
DAT::Account::SearchDataImpl::release()+0xe8
libDAT_AccountBatch64.so`void
DAT::Account::CustomerSearchData::release()+0x4
libFCT_Account64.so`0xffffffff6d058364

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms