Large Batch Hangs at Random Tasks in OFSAA After Router Crashes (Doc ID 1907750.1)

Last updated on MARCH 03, 2023

Applies to:

Symptoms

In Oracle Financial Services Analytical Applications (OFSAA), when you run a large batch with over 1,000 Profitability Management (PFT) Allocations, the batch hangs in an 'ongoing' status at different stages.  For example, when you run the batch three times, the 1st time the batch hangs at Task 1020.  The 2nd time the batch hangs at Task 1016.  And the 3rd time the batch hangs at Task 374.

After the Batch hangs, you see the router process no longer exists on the server (when you run "ps -ef|grep router").  The router process crashed while running the batch.

You many see errors like the following in the $FIC_APP_HOME/icc/log/iccserver.log:

[18:52:46,221] - Request to connect to router.
[18:52:46,221] - Stream to router not null. Not reconnecting.
[18:52:46,221] - Finished writing to router.
[18:52:46,222] - Waiting for task completions.
[18:52:52,564] - Error reading from router socket.
java.lang.NumberFormatException: For input string: " "
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
    at java.lang.Integer.parseInt(Integer.java:449)
    at java.lang.Integer.parseInt(Integer.java:499)
    at com.iflex.fic.icc.iccal.iccexecution.jobscheduler.JobManager.read(JobManager.java:247)
    at com.iflex.fic.icc.iccal.iccexecution.jobscheduler.Reader.run(JobManager.java:342)

and

[18:52:52,565] - Connecting to Router at 0.0.0.0:8600
[18:52:52,565] - Error connecting to router
java.net.ConnectException: Connection refused

If you run the Allocations outside of the batch, they run successfully.

Note: You verified you do not have the issue described in <Document 1505024.1> "Large Batches of Allocations Hanging During Execution".  You either have OFSAAI 7.3.3 or higher or, when you start the agent processes (router, am, and messageserver), you start them individually instead of using the agentstartup.sh script.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.