Attempting Administration Configuration of OHS in FMW Control Console Results in Error:Load Failed With IOExceptionAdministrator Connection is Down (Doc ID 1527922.1)

Last updated on JULY 27, 2016

Applies to:

Oracle HTTP Server - Version 11.1.1.2.0 to 11.1.2.0.0 [Release Oracle11g]
Microsoft Windows (32-bit)
This issue can be reproduced with a WebTier install on any type.
The following were the steps to reproduce the issue internally.

1. Use Option #1 of Document ID:1074140.1 & Install Guide Option "1.3.3 Tasks for
Associating Oracle Web Tier With Fusion Middleware (EM) Control Post-Installation"

- Installed WLS 10.3.6 and did not create Domain.
- Installed, but did NOT configure 11.1.1.6.0 WebTier
- Ran config.cmd from $WLS_HOME/common/bin
- created new Domain and selected EM and JRF
- Ran config.bat from WebTier $OH/bin and configured WebTier using existing Domain.
- Ensured http://localhost:7001/console worked
- Ensured http://localhost:7001/em worked

2. Logged into http://localhost:7001/em as weblogic

3. Expanded WebTier, and clicked on ohs1

4. Clicked on Oracle HTTP Server drop-down, then Administration.

5. Select Log Configuration. Error stack below is now seen.

Symptoms

 When one attempts to edit/configure Administration settings of the Oracle HTTP Server(OHS), the following errors are raised in the browser

Failed to invoke operation load on MBean
oracle.as.management.mbeans.register:type=component,name=ohs1,instance=instance1,Location=AdminServer
Load failed with IOException

There is a large stack trace that begins with the following, but not all is listed for brevity sake.

oracle.sysman.emSDK.app.exception.EMSystemException at
oracle.sysman.emas.sdk.model.jmx.JMXUtil.invokeOperation(JMXUtil.java:100) at
oracle.sysman.emas.model.ohs.LogManagementModel.initializeData(LogManagementModel.java:134) at
oracle.sysman.core.model.util.ModelUtil$EMObjectResetter.initOrResetEMObject(ModelUtil.java:319) at
oracle.sysman.core.model.util.ModelUtil$EMObjectResetter.access$000(ModelUtil.
java:280)

 

Further troubleshooting and debugging using the method in <Note:1463919.1> produces the following:


1. Check the <domain home>/servers/AdminServer/logs/AdminServer-diagnostic.log and found the following.

[2013-01-28T03:11:30.859+00:00] [AdminServer] [ERROR] [] [oracle.ohs] [tid:
[ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default
(self-tuning)'] [userId: <WLS Kernel>] [ecid:
b0a556564f7b0ddb:6b1e60de:13c7f1ec3cb:-8000-000000000000002d,0] [APP:
NonJ2EEManagement#11.1.1]  ReadFile failed  Connection reset


In addition to:

[2013-01-28T03:11:40.078+00:00] [AdminServer] [WARNING] []
[oracle.ohs.mbeans] [tid: [ACTIVE].ExecuteThread: '0' for queue:
'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid:
b0a556564f7b0ddb:6b1e60de:13c7f1ec3cb:-8000-000000000000002d,0] [APP:
NonJ2EEManagement#11.1.1] Load failed with IOExceptionAdministrator
connection is down, Please check component ohs1 is up

 

2. Check the MIDDLEWARE_HOME/Oracle_WT1/instance/instance1/diagnostics/logs/OHS and find no errors.

3. Check the MIDDLEWARE_HOME/Oracle_WT1/instance/instance1/diagnostics/logs/OPMN/opmn/opmn.log and see the following.

[2013-01-28T03:09:13+00:00] [opmn] [NOTIFICATION:1] [90] [ons-internal] ONS
server initiated
[2013-01-28T03:09:13+00:00] [opmn] [TRACE:1] [522] [pm-internal] PM state
directory exists:
C:\Oracle\Middleware\Oracle_WT1\instances\instance1\config\OPMN\opmn\states
[2013-01-28T03:09:13+00:00] [opmn] [NOTIFICATION:1] [675] [pm-internal] OPMN
server ready. Request handling enabled
[2013-01-28T03:09:16+00:00] [opmn] [NOTIFICATION:1] [667] [pm-requests]
Request 2 Started. Command: /start
[2013-01-28T03:09:16+00:00] [opmn] [NOTIFICATION:1] [662] [pm-process]
Starting Process: ohs1~OHS~OHS~1 (1619397654:0)
[2013-01-28T03:09:27+00:00] [opmn] [NOTIFICATION:1] [665] [pm-process]
Process Alive: ohs1~OHS~OHS~1 (1619397654:4196)
[2013-01-28T03:09:27+00:00] [opmn] [NOTIFICATION:1] [668] [pm-requests]
Request 2 Completed. Command: /start
[2013-01-28T03:11:30+00:00] [opmn] [ERROR:1] [] [libopmnohs] Process Ping
Failed: ohs1~OHS~OHS~1 (1619397654:4196) [The connection receive failed]

 

Lastly, by further OS tracing it was found that the child Apache process is getting killed by the parent Apache process.
Crash dump shows that the child process is getting crashed in mod_ossl.so in the function "ssl_date2string()".

This crash is happening while reading the END date of the OHS certificate which is used for handshaking between OHS admin port and FMWC console.

By default the OHS generates a self-signed certificate for the newly created OHS component. This default certificate is used for handshaking between
weblogic FMWC and OHS admin port. This default certificate END date is calculated based on the OHS installation date.

For example,

If the OHS instance is created on "29 Jan 2013", then the default self-signed certificate validity start date is "29 Jan 2013" and end date validity is "23
Jan 2038".

The data type "time_t"  variable doesn't have the capability of storing the date value more than "19 jan 2038" in 32 bit m/c's.

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms