Real-time Discounting Pipeline Is Crashing Under Heavy CDR Loading (Doc ID 1963151.1)

Last updated on AUGUST 24, 2016

Applies to:

Oracle Communications Billing and Revenue Management - Version 7.5.0.0.0 to 7.5.0.0.0 [Release 7.5.0]
Information in this document applies to any platform.

Symptoms

For our implementation we have OCMC load balancing real-time charging requests to 2 BRM servers.
Each BRM server is running 4 CMs for data service, 4 CMs for SMS service and 8 real-time discounting pipelines.
Each CM is configured to connect to the 8 real-time discounting pipelines in load balancing mode.
Each real-time discounting pipeline has 24 threads.

While testing a relatively high load (1320 real-time charging requests per second), we experienced several times one real-time discounting pipeline crashing.
We can see in the real-time discounting pipeline log ERR_EXECUTING_OPCODE errors:

Once we start seeing this in the real-time discounting pipeline log, the pipeline usually keeps logging these errors until we stop the test or the pipeline stops responding or it crashes.
When this happens we sometimes are not able to stop the pipeline properly using a semaphore and we have to kill the pipeline manager process.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms