WebTier DMS Showing Incorrect Metric Values On Connection.active And BusyThreads.Value (Doc ID 2160409.1)

Last updated on FEBRUARY 16, 2017

Applies to:

Oracle HTTP Server - Version 12.1.3.0.0 and later
Enterprise Manager Base Platform - Version 12.1.0.1.0 and later
Oracle Fusion Middleware - Version 12.1.3.0.0 and later
Information in this document applies to any platform.

Symptoms

On a standalone WebTier 12.1.3 installation the metrics obtained with the dumpMetrics WLST command shows that there is a incoherent metric value for connection.active and busyThreads.value.

Please take a look on the output:

...
busyThreads.value: 4173 threads
busyProcesses.value: 0 processes
numThreads.value: 256 threads
numProcesses.value: 3 processes
mpm.value: Worker
connection.active: 44991 threads
...

 

Verified:

Checked current connections using netstat command and the output was around 252, which is not close to 44991 (connection.active).
OHS was being monitored with Cloud Control and such variation on metric output triggers false alerts.

At the same time, it has been observed within the OHS logs the following:

[2016-06-23T00:10:49.0879+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: GEOSPRPWEB01] [host_addr: 10.9.64.1] [pid: 891] [tid: 139880718501680] [user: ora_share] [VirtualHost: main] Oracle-HTTP-Server-12c/12.1.3.0.0 (Unix) mod_ssl/12.1.3.0.0 OtherSSL/0.0.0 mod_plsql/11.1.1.0.0 configured -- resuming normal operations

[2016-06-23T00:10:51.0895+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: GEOSPRPWEB01] [host_addr: 10.9.64.1] [pid: 891] [tid: 139880718501680] [user: ora_share] [VirtualHost: main] child pid 21567 exit signal Segmentation fault (11)

 

coredump was parsed with stackx tool (refer to KM <Note:362791.1>), which given the following stack:

(...)
Loaded symbols for /lib64/libgcc_s.so.1
Core was generated by '/opt/oracle/webe01/webe01/12.1/fmw/ohs/wlserver/../ohs/bin/httpd.worker -k star'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f4b1e75c005 in ssl_scache_remove () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/ohs/modules/mod_ossl.so
(gdb) (gdb) </status>
<stackinfo>
(...)
Thread 1 (Thread 0x7f4b18cfa940 (LWP 28500)):
#0 0x00007f4b1e75c005 in ssl_scache_remove () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/ohs/modules/mod_ossl.so
#1 0x00007f4b1e7524b7 in ssl_callback_DelSessionCacheEntry () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/ohs/modules/mod_ossl.so
#2 0x00007f4b2511392f in nzossc_Delete () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#3 0x00007f4b252526af in ssl_ctx_remove_session_nolock () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#4 0x00007f4b25252639 in R_SSL_CTX_remove_session () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#5 0x00007f4b25298a6a in ssl3_send_alert () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#6 0x00007f4b25299d5c in ssl3_read_bytes () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#7 0x00007f4b252579d6 in ssl3_read () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#8 0x00007f4b2510e6cb in nzos_Read () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/lib/libnnz11.so
#9 0x00007f4b1e7515a9 in ssl_io_input_read () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/ohs/modules/mod_ossl.so
#10 0x00007f4b1e751025 in ssl_io_filter_input () from /opt/oracle/webe01/webe01/12.1/fmw/ohs/ohs/modules/mod_ossl.so
(...)

 

Changes

 None, just upgraded the WebTier to 12.1.3.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms