OHS httpd.worker Processes Consume CPU and Slows Down System During CGI Application Processing (Doc ID 1560799.1)

Last updated on MARCH 18, 2024

Applies to:

Symptoms

Using Oracle HTTP Server 11g, there are two visible symptoms:

1) Upon a spike in heavier load, httpd.worker processes consume more CPU then they should until system reaches a CPU limit. Even if load subsides without hitting the limit, some httpd.worker processes still consume more CPU than others, but apparently not processing requests. Restarting OHS is the only way to reset.

2) If system is configured to automatically allocate more CPU resources, the system slows down every once in a while in a random fashion. It is during a heavier load time and appears as if tuning is required.

The two issues are related because over time the CPU gradually increases until it reaches a limit, either way. It does not completely correct itself until a restart. This will most likely not be seen in normal runtime, it must be with a higher load to force a specific condition:

• Depending on tuning parameters and processes initiated at startup, OHS will determines that it needs to create another httpd.worker process to handle more concurrent connections.
   
• OHS creates another httpd.worker process the thread creation process hangs and only a few threads are created. This is not always noticable but the new httpd.worker process begins to use 5% of cpu. (see pstack results below). Note the already existing httpd.worker processes will still be processing incoming requests consuming <0.5% of cpu.
   
• OHS's main program loop has an unfinished task to increase the number of threads so it creates another the httpd.worker process with a few threads and hangs again.
   
• This sequence of events continues until 10 to 20 new httpd.worker processes have been created each consuming 5% of cpu.
   
• None of these new httpd.worker processes have sufficent threads to match the httpd.conf tuning parameters.
   
• Now there are no spare cpu cycles to service the clients and they experience a slow down. Response time increases in a linear manner from 80ms to 1 sec or more. A core dump may occur and the OHS error log will reveal time outs for the CGI program and all other requests. The machine itself will become slower in all its tasks.
   

The following is seen on the OHS error log, snippets from during a slowdown and then at a core dump time:

Changes

Because of 10g and 11g differences, this may be popular to see when using a CGI application with Oracle Application Server 10g, then migrating/upgrading to Oracle Fusion Middleware 11g. This issue shows a good reason to require additional 10g-to-11g migration/upgrade steps before going production with the new version.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.