Realtime Pipeline Hanged During Billrun (Doc ID 1349459.1)

Last updated on SEPTEMBER 30, 2013

Applies to:

Oracle Communications Billing and Revenue Management - Version 7.3.1.0.0 to 7.3.1.0.0 [Release 7.3.1]
Information in this document applies to any platform.
Checked for relevance on 27-Sep-2013.

Symptoms

When pin_bill_day is running, realtime pipeline is hanging.

CM stack trace at hung state;
======================

484774: /pbmb/opt/portal/ret/7.3.1/bin/cm
0x2001d9b4 read(??, ??, ??) + 0x1a8
0x2031542c _pcp_readwire() + 0x2bc
0x20314c98 pcpxdr_read_buf() + 0x2c0
0x20318f60 pcpxdr_fld_list() + 0x1b48
0x203235d0 pcpxdr_op_decode() + 0x50
0x20322f3c pcp_receive_no_trans_cleanupAF31_22() + 0xac
0x20322df8 pcp_receive() + 0x18
0x20325f48 fwd_op_to_em() + 0x1b0
0x20310108 cm_pre_pcm_op() + 0xa68
0x2030e760 pcm_op_ex() + 0x1a8
0x20329144 pcm_op() + 0x1c
0x307a2240 op_rate_and_discount_event() + 0x790
0x2030fefc cm_pre_pcm_op() + 0x85c
0x2030e760 pcm_op_ex() + 0x1a8
0x20329144 pcm_op() + 0x1c
0x3031822c fm_act_rate_discount_and_tax() + 0x80c
0x3031596c op_act_usage() + 0x1214
0x2030fefc cm_pre_pcm_op() + 0x85c
0x2030e760 pcm_op_ex() + 0x1a8
0x20329144 pcm_op() + 0x1c
0x30480c78 fm_subs_cycle_call_act_usage(??, ??, ??, ??, ??, ??) + 0xaf0
0x3048e57c op_subscription_apply_rate(??, ??, ??, ??, ??, ??) + 0x1574
0x2030fefc cm_pre_pcm_op() + 0x85c
0x2030e760 pcm_op_ex() + 0x1a8
0x20329144 pcm_op() + 0x1c
0x3047f488 fm_subs_call_apply_rate(0x214c36b8, 0x39101838, 0x3b4164a8,
0x3b69a148, 0x3b4a2f68, 0x0, 0x0, 0x2ff1c528) + 0x398
0x304bfb70 op_subscription_cycle_forward(??, ??, ??, ??, ??, ??) + 0x1fd8
0x2030fefc cm_pre_pcm_op() + 0x85c
0x2030e760 pcm_op_ex() + 0x1a8
0x20329144 pcm_op() + 0x1c
0x302159ec fm_bill_mb_apply_cycle_fees(0x360c0a58, 0x66, 0x100, 0x360c3338,
0x2ff1d0d0, 0x2ff1d688) + 0x28c
0x30a91294 fm_bill_mb_partial_billing(??, ??, ??) + 0x16c
0x30a8f3b4 fm_bill_mb_bill_acct(??, ??, ??, ??, ??, ??) + 0x15c
0x30a8de80 fm_bill_mb_make_bill(??, ??, ??, ??, ??) + 0x210
0x30a8c0a4 op_bill_make_bill(??, ??, ??, ??, ??, ??) + 0x424
0x205567a4 cm_custom_ex() + 0x4ac
0x205562a8 cm_custom() + 0x120
0x20552df8 cm_child() + 0xb40
0x2055f178 mainThread() + 0x1e00
0x10001ec4 main() + 0xc
0x100001c0 __start() + 0x98

ifw pstack at hung state;
================== 
;---------- tid# 963315 (pthread ID; 5398) ----------;

0x09000000004862e4 _global_lock_common(??, ??, ??) + 0x40c
0x0900000000064f8c malloc_y(0x360, 0x0, 0x11abcc860, 0x9001000a0719600, 0x0,
0x75700000, 0x736d2f64, 0x61746100) + 0x120
0x0900000000062270 malloc_common_80_64(??) + 0x14
0x0900000000f547c8 pini_flist_grow() + 0x120
0x0900000000f5cbf8 pcpxdr_fld_list() + 0x1688
0x0900000000f68c5c pcpxdr_op_decode() + 0x5c
0x0900000000f6ef74 pcp_receive_no_trans_cleanupAF30_22() + 0xbc
0x0900000000f6ee20 pcp_receive() + 0x18
0x0900000002fa1c28
NET::EM::RequestHandler<ACE_SOCK_Stream,ACE_INET_Addr>::handle_input(int)() +0xa8
0x0900000000757a5c
ACE_TP_Reactor::dispatch_socket_event(ACE_EH_Dispatch_Info&)() + 0xbc
0x0900000000757118
ACE_TP_Reactor::handle_socket_events(int&,ACE_TP_Token_Guard&)() + 0xb0
0x0900000000756810
ACE_TP_Reactor::dispatch_i(ACE_Time_Value*,ACE_TP_Token_Guard&)() + 0x98
0x09000000007566f4 ACE_TP_Reactor::handle_events(ACE_Time_Value*)() + 0x104
0x09000000006c7e44
ACE_Reactor::run_reactor_event_loop(int(*)(ACE_Reactor*))() + 0xe4
0x0900000002f913b0 NET::EM::ThreadPool::run()() + 0x68
0x0900000002fa3118 RWTFunctor0MImp<NET::EM::ThreadPool,void>::run() const()+ 0x20
0x0900000002abeaa8 RWThreadFunctionImp::run()() + 0xe8
0x0900000002a38f64 RWRunnableImp::exec()() + 0x1d0
0x0900000002a80658 RWThreadImp::exec()() + 0xa0
0x0900000002a80590 RWThreadImp_entry() + 0xc
0x09000000004874f4 _pthread_body(??) + 0xdc

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms