My Oracle Support Banner

BCD Crashes During Connection Check (Doc ID 2537763.1)

Last updated on FEBRUARY 18, 2020

Applies to:

Oracle Communications Network Charging and Control - Version 6.0.1 and later
Information in this document applies to any platform.

Symptoms

On Oracle Communications Network Charging and Control (OCNCC/NCC), brmBeClient process from both SLC (Service Logic Controller) nodes has crashed and created core file, around the same time - 01:48:10 GMT.

As per information received from team supporting Billing and Revenue Management (BRM), there were some problems with Connection Manager(CM) process on BRM side around the time when crash happened.

The following errors were logged in /var/adm/messages log around the crash moment :

Mar 15 01:48:10 <HOSTNAME> brmBeClient: [ID 675952 user.error] brmBeClient(9234) ERROR: {1220518} Error when sending BRM opcode.
Mar 15 01:48:10 <HOSTNAME> brmBeClient: [ID 675952 user.error] brmBeClient(9234) ERROR: {1220534} Already have the maximum allowed number of connections to BRM for all working connection managers.
...
Mar 15 01:48:10 <HOSTNAME> brmBeClient: [ID 675952 user.error] brmBeClient(9234) ERROR: {1220534} Already have the maximum allowed number of connections to BRM for all working connection managers.
Mar 15 01:48:11 <HOSTNAME> brmBeClient: [ID 675952 user.error] brmBeClient(9234) ERROR: {1220522} Failed to create a BcdSleeEvent to return BRM results.
...
Mar 15 01:48:17 <HOSTNAME> genunix: [ID 603404 kern.notice] NOTICE: core_log: brmBeClient[9234] core dumped: /var/crash/core-<HOSTNAME>-9234-brmBeClient

The gdb backtrace of the core file can have different backtraces but are all around connection handling.

Example 1:

Core was generated by `/IN/service_packages/BCD/bin/brmBeClient'.
Program terminated with signal 11, Segmentation fault.
...
#5 <signal handler called>
#6 pini_flist_find (flistp=<optimized out>, fldp=<optimized out>, fld=<optimized out>, rec_id=<optimized out>, rfldp=<optimized out>, ebufp=<optimized out>) at ../pcm/pcmi.c:249
#7 0xfefeb180 in pini_flist_drop (flistp=<optimized out>, fld=<optimized out>, record_id=<optimized out>, ebufp=<optimized out>) at ../pcm/pin_flist.c:4136
#8 0x0002b7b8 in bcd::bcdBillingClient::Connection::getResults (this=0x1999f8) at Connection.cc:837
#9 0x000359b0 in bcd::bcdBillingClient::Controller::processConnection (this=this@entry=0x72c10, fd=77) at Controller.cc:932
#10 0x000361e8 in bcd::bcdBillingClient::DevPollHandler::dataAvailable (this=0x2344d0, fd=...) at Controller.cc:1269
#11 0xff056b88 in cmn::dpFD::readData (this=0x2344d0) at cmndpFD.cc:301
#12 0xff057720 in cmn::dpFDManager::processFdSet (this=this@entry=0x72c18, first=first@entry=0x72c38, last=<optimized out>) at cmndpFD.cc:589
#13 0xff057e6c in cmn::dpFDManager::poll (this=this@entry=0x72c18, timeout=timeout@entry=0xffbfe098) at cmndpFD.cc:755
#14 0x000351c8 in bcd::bcdBillingClient::Controller::run (this=0x72c10, argc=argc@entry=1, argv=argv@entry=0xffbfe21c) at Controller.cc:1202
#15 0x0001ed24 in main (argc=1, argv=0xffbfe21c) at brmBeClient.cc:85

Example 2:

Core was generated by `/IN/service_packages/BCD/bin/brmBeClient'.
Program terminated with signal 10, Bus error.
#0  0xfc2aec60 in __lwp_sigqueue () from /usr/lib/libc.so.1
(gdb) bt
...
#5  <signal handler called>
#6  pcp_op_done (ctxp=<optimized out>, ebufp=<optimized out>) at ../pcp/pcpst.c:3826
#7  0xfefdd1a0 in pcm_op_done (pcm_ctxp=<optimized out>, ebufp=<optimized out>) at ../pcm/pcm.c:1131
#8  0x00029420 in bcd::bcdBillingClient::Connection::checkConnection (this=0xa1f20) at Connection.cc:922
#9  0x000358d4 in bcd::bcdBillingClient::Controller::processConnection (this=this@entry=0x72c10, fd=188) at Controller.cc:926
#10 0x000361e8 in bcd::bcdBillingClient::DevPollHandler::dataAvailable (this=0xa1f60, fd=...) at Controller.cc:1269
#11 0xff056b88 in cmn::dpFD::readData (this=0xa1f60) at cmndpFD.cc:301
#12 0xff057720 in cmn::dpFDManager::processFdSet (this=this@entry=0x72c18, first=first@entry=0x72c38, last=<optimized out>) at cmndpFD.cc:589
#13 0xff057e6c in cmn::dpFDManager::poll (this=this@entry=0x72c18, timeout=timeout@entry=0xffbfdfe8) at cmndpFD.cc:755
#14 0x000351c8 in bcd::bcdBillingClient::Controller::run (this=0x72c10, argc=argc@entry=1, argv=argv@entry=0xffbfe16c) at Controller.cc:1202
#15 0x0001ed24 in main (argc=1, argv=0xffbfe16c) at brmBeClient.cc:85

Changes

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.