12c Cloud Control: OMS is Re-started Frequently by the HealthMonitor Due to Problems in the Repository Database
(Doc ID 1491412.1)
Last updated on OCTOBER 13, 2019
Applies to:
Enterprise Manager Base Platform - Version 12.1.0.1.0 and laterGeneric Linux
Symptoms
The 12c OMS is crashing frequently and the following error messages are recorded in log files:
<ORACLE_BASE>/gc_inst/em/EMGC_OMS1/sysman/log/emctl.msg shows:
HealthMonitor Aug 27, 2012 8:23:07 PM OMS Heartbeat Recorder error: Heartbeat recorder timed out. OMS id=77
Critical error err=3 detected in module OMS Heartbeat Recorder
OMS will be restarted. A full thread dump will be generated in the log file
/pg1as/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out
to help Oracle Support analyse the problem.
Please consult My Oracle Support Note 964469.1 for additional information.
HealthMonitor Aug 27, 2012 11:37:13 PM OMS Heartbeat Recorder: error: Zero rows updated for this oms during heartbeat
Critical error err=3 detected in module OMS Heartbeat Recorder:
OMS will be restarted. A full thread dump will be generated in the log file
/u01/app/oracle/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out
to help Oracle Support analyse the problem.
Please consult My Oracle Support Note 964469.1 for additional information.
HealthMonitor Aug 29, 2012 7:22:39 PM Job Dispatcher error: Job Dispatcher timed out. Job Dispatcher id=80
Critical error err=3 detected in module Job Dispatcher
OMS will be restarted. A full thread dump will be generated in the log file
/pg1as/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out
to help Oracle Support analyse the problem.
Please consult My Oracle Support Note 964469.1 for additional information.
At each OMS re-start, the Repository Database's alert.log shows the below entries:
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.3.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.3.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.3.0 - Production
Time: 30-AUG-2012 09:20:59
Tracing not turned on.
Tns error struct:
ns main err code: 12537
TNS-12537: TNS:connection closed
ns secondary err code: 12560
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
opiodr aborting process unknown ospid (28979) as a result of ORA-609
Wed Aug 29 01:14:46 2012
Errors in file /u01/app/oracle/diag/rdbms/omr/OMR/trace/OMR_cjq0_16843.trc (incident=45073):
ORA-00445: background process "J011" did not start after 120 seconds
Incident details in: /u01/app/oracle/diag/rdbms/omr/OMR/incident/incdir_45073/OMR_cjq0_16843_i45073.trc
Wed Aug 29 01:14:46 2012
Dumping diagnostic data in directory=[cdmp_20120829011446], requested by (instance=1, osid=16843 (CJQ0)), summary=[incident=45073].
kkjcre1p: unable to spawn jobq slave process
Errors in file /u01/app/oracle/diag/rdbms/omr/OMR/trace/OMR_cjq0_16843.trc:
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Symptoms |
Cause |
Solution |
References |