My Oracle Support Banner

12c OMS Failed to Start or takes long time to start or Restarts Frequently when the OMS is installed on NFS Shared Location (Doc ID 1474015.1)

Last updated on DECEMBER 06, 2017

Applies to:

Enterprise Manager Base Platform - Version 12.1.0.1.0 to 12.1.0.5.0 [Release 12.1]
Information in this document applies to any platform.
Checked for relevance on 23-Dec-2013

Symptoms

Multiple performance issues are hit when Enterprise Manger Cloud Control 12c is installed on a NFS Shared Location.

1. OMS startup may take a long time:

Refer: Note 1385539.1 12c Cloud Control Performance: OMS Installed on a NetApp Filer with NFS Takes a Long Time to Startup.

 

2. OMS goes down and restarts frequently with the following error in the file [...]gc_inst/Middleware/EMGC_OMS1/sysman/log/emctl.msg file

  OMS Heartbeat Recorder:  error: Zero rows updated for this oms during heartbeat
  Critical error err=3 detected in module OMS Heartbeat Recorder:
  OMS will be restarted. A full thread dump will be generated in the log file
  /apps/OPOMSA/oracle/oracle-OPOMSA-OPOMSA-p/product/Middleware/gc_inst/user_pr
  ojects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out

 Additionally, all the agents are showing Unreachable status alerts and Unreachable clear notifications frequently.

 

 3. OMS start up fails intermittently.

.../gc_inst/WebTierIH1/diagnostics/logs/OPMN/opmn/opmn.log file has the following entries repeatedly.

[2012-05-29T15:54:35][opmn][TRACE:1][][OPMN][code:libopmnohs]Process Ping Failed: ohs1~OHS~OHS~1 (749666542:15335490) [The connection receive timed out]
[2012-05-29T15:55:25][opmn][TRACE:1][][OPMN][code:libopmnohs]Process Ping Failed: ohs1~OHS~OHS~1 (749666542:15335490) [The connection receive timed out]
[2012-05-29T15:56:15][opmn][TRACE:1][][OPMN][code:libopmnohs]Process Ping Failed: ohs1~OHS~OHS~1 (749666542:15335490) [The connection receive timed out]
[2012-05-29T15:56:15][opmn][TRACE:1][][OPMN][code:libopmnohs]Process Unreachable: ohs1~OHS~OHS~1 (749666542:15335490)

 

.../gc_inst/WebTierIH1/diagnostics/logs/OHS/ohs1/ohs1.log has this:

[2012-05-29T15:56:25.4496+02:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: am3hh068] [host_addr: 32.88.14.68] [pid: 15335490] [tid: 1] [user: oradba] [VirtualHost: main]  child process 10879076 still did not exit, sending a SIGTERM
[2012-05-29T15:56:25.4497+02:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: am3hh068] [host_addr: 32.88.14.68] [pid: 15335490] [tid: 1] [user: oradba] [VirtualHost: main]  child process 18743544 still did not exit, sending a SIGTERM

 or this:

[2014-02-08T17:17:17.7998-10:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: em12c.test.com] [host_addr: 10.10.10.10] [pid: 17093] [tid: 140384688568064] [user: oracle] [VirtualHost: main] (116)Stale file handle:  apr_proc_mutex_lock failed. Attempting to shutdown process gracefully.

[2014-02-08T17:17:17.8048-10:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: em12c.test.com] [host_addr: 10.10.10.10] [pid: 17123] [tid: 140384688568064] [user: oracle] [VirtualHost: main] (116)Stale file handle:  apr_proc_mutex_lock failed. Attempting to shutdown process gracefully.

  

4. Console Access may be slow:

Refer: Note 1404077.1 12c Cloud Control Performance: Rendering of Cloud Console Pages is Very Slow when OMS is Installed on NFS.



Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.