Unable To Process Local Calls (Doc ID 1361809.1)

Last updated on AUGUST 02, 2016

Applies to:

Oracle Communications Billing and Revenue Management - Version 7.3.1.0.0 and later
Information in this document applies to any platform.

Symptoms

AAA gateway is unable to process the local calls as dm_oracle is erroring out with the following error:

W Fri Sep 9 16:58:12 2011 testbrm3.south.net dm:27047 dm_front.c(52):2001 1:testbrm.south.net:cm:1906:-146482976:77:1315596969:194575
DMfe #5: dropped connect from 10.16.102.yy:6351, too full, j -1
W Fri Sep 9 16:59:48 2011 testbrm3.south.net dm:27047 dm_front.c(52):2001 1:testbrm.south.net:cm:1906:-146482976:77:1315596969:194575
DMfe #5: dropped connect from 10.16.102.xx:11471, too full, j -1

where dm_oracle cannot take any connections at this point as dm_oracle cannot take more than 96 connections. But when the calls are failing it can be seen that there are more than 103 CMs spawned.

When diameter comes up, there are only 28 CMs but by the time the issues comes there are more than 103 CMs.

Also many Signal 11 messages are seen when the authorize opcode is being called from AAA gateway.
Below is the stack trace from the cm.pinlog :


Bad flist ptr in pini_flist_get
#0  0xffffe410 in __kernel_vsyscall ()
#1  0x003bcdc3 in __waitpid_nocancel () from /lib/libc.so.6
#2  0x0036203b in do_system () from /lib/libc.so.6
#3  0x004ae87d in system () from /lib/libpthread.so.0
#4  0xf7db57ea in printExceptionReportprintExceptionReport ()
#5  0xf7dae3e2 in sigsegv_handler () from
/usr/local/brm/7.3/lib/libcm_main.so
#6  <signal handler called>
#7  0x003958d8 in _int_malloc () from /lib/libc.so.6
#8  0x003973ab in malloc () from /lib/libc.so.6
#9  0xf7e55786 in pini_flist_grow () from /usr/local/brm/7.3/lib/libcmpin.so
#10 0xf7e76bf8 in pin_flist_copy_exclude ()
#11 0xf7e72f21 in pin_flist_fldval_copy ()
#12 0xf7e76d64 in pin_flist_copy_exclude ()
#13 0xf7e72f21 in pin_flist_fldval_copy ()
#14 0xf7e741de in pini_flist_set () from /usr/local/brm/7.3/lib/libcmpin.so
#15 0xf7e3a29d in cm_fm_set_named_trans_flist ()
#16 0xf6dac2b4 in fm_bill_utils_set_results_to_cache ()
#17 0xf669b6e3 in op_bal_get_balances () from
/usr/local/brm/7.3/lib/fm_bal.so
#18 0xf7e37970 in cm_pre_pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#19 0xf7e4aca7 in pcm_op_ex () from /usr/local/brm/7.3/lib/libcmpin.so
#20 0xf7e4a850 in pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#21 0xf605e3b6 in fm_reserve_utils_get_balances ()
#22 0xf60531c3 in op_reserve_create ()
#23 0xf7e37970 in cm_pre_pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#24 0xf7e4aca7 in pcm_op_ex () from /usr/local/brm/7.3/lib/libcmpin.so
#25 0xf7e4a850 in pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#26 0xf6cea3ce in fm_act_auth_call_create_reservation ()
#27 0xf6ce78ef in fm_act_authorize () from /usr/local/brm/7.3/lib/fm_act.so
#28 0xf6ce72e3 in op_act_authorize () from /usr/local/brm/7.3/lib/fm_act.so
#29 0xf7e37970 in cm_pre_pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#30 0xf7e4aca7 in pcm_op_ex () from /usr/local/brm/7.3/lib/libcmpin.so
#31 0xf7e4a850 in pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#32 0xf5d15cc3 in fm_tcf_aaa_authorize ()
#33 0xf5d15a76 in op_tcf_aaa_authorize ()
#34 0xf7e37970 in cm_pre_pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#35 0xf7e4aca7 in pcm_op_ex () from /usr/local/brm/7.3/lib/libcmpin.so
#36 0xf7e4a850 in pcm_op () from /usr/local/brm/7.3/lib/libcmpin.so
#37 0xf5c8abb5 in cs_op_tcf_custom_aaa_authorize ()
#38 0xf5c878c6 in op_tcf_custom_aaa_authorize ()
#39 0xf7da94b6 in cm_custom_ex () from /usr/local/brm/7.3/lib/libcm_main.so
#40 0xf7da899e in cm_custom () from /usr/local/brm/7.3/lib/libcm_main.so
#41 0xf7da6cfb in cm_child () from /usr/local/brm/7.3/lib/libcm_main.so
#42 0xf7db5160 in mainThread () from /usr/local/brm/7.3/lib/libcm_main.so
#43 0x0804878e in main ()

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms