JVM Controllers Cause Hang Intermittently in Forms 12c (Doc ID 2761810.1)

Last updated on MAY 06, 2023

Applies to:

Symptoms

Environment
=========
Forms 12.2.1.3
Linux-64

Description
=========
JVM Controllers cause hang intermittently.
 
Randomly, the JVMController will fail/crash and the Oracle Forms will no longer be able to submit RUN_REPORT_OBJECT requests to Oracle Reports.

It appears that the JVM Controller is not releasing the threads/eventpolls causing it to stack up and exceed the Linux open files limit.

The concern is that these eventpolls should not be piling up. They should disappear after closing form/browser.

Tried tuning the following parameters, but it does not help:

In gathering the information on Linux Operating System Requirements, the dump of lsof command shows a pattern that leads to the cause of the crash.

For example:

The ulimit max number of open files is 65536  (ulimit -n).
Pulling these statistics revealed this:

    Some of the servers were over the 65K mark (70k)
    The remaining were close to it....
The problem happens once over the 65K mark.

In testing, found that the eventpoll goes away if properly close the Forms session.
But if kill browser session, the eventpoll does not go away although forms session goes away. In this case, all these hung processes/open files take up resources. However, this is not always consistent. Sometimes even if properly close the Forms session, the eventpoll does not go away, but other times it goes away.

To Recover
========
Restarting JVMController does clear out the problem eventpolls, but this is not feasible. Need to restart Forms Managed Server to recover.
The problem with the JVM restart is that the user's call to Reports will fail due to existing session to JVM is lost, and then get a different error, "Unable to communicate <jvm>" errors. So the cleaner way to recover is to restart WLS_FORMS.  

To Reproduce
=========
 Reproduce by seeing the eventpolls increase.

Steps:
======

Given Reports and Forms are set up and have a form that calls reports.
 1. Configure a jvmcontroller (call it: repjvmc)
 2. Open 10 browser windows at once
 3. In each window, call a report from form using RRO (any simple report)
 4. Forcefully close all 10 browser windows after successfully running  reports.
 5. On O/S level, check:

The ones hanging around are the problem ones. In this case, the dejvm is the top consuming processes for open files. They keep piling up until hit open files limit.

 Again, this is not consistent.

Changes

 There were no issues with this JVM Controller until upgraded to Forms 12c.  This was not a problem in Forms 11g.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.