Critical Error Err=3 Detected In Module Oms Heartbeat Recorder:Oms Will Be Restarted.

(Doc ID 1359693.1)

Last updated on MARCH 26, 2016

Applies to:

Enterprise Manager Base Platform - Version to [Release 11.1]
Information in this document applies to any platform.
Checked for relevance on 26th March' 16

Symptoms Grid Control Crashes intermittently.

Errors observed from emctl.msg:

HealthMonitor Sep 16, 2011 12:48:18 PM OMS Heartbeat Recorder:  error: Zero rows updated for this oms during heartbeat
Critical error err=3 detected in module OMS Heartbeat Recorder: 
OMS will be restarted. A full thread dump will be generated in the log file 
to help Oracle Support analyse the problem. 
Please consult My Oracle Support Note 964469.1 for additional information.
Errors observed from EMGC_OMS*.out* file:

<Sep 27, 2011 5:52:04 AM EDT> <Error> <WebLogicServer> <BEA-000337> <[STUCK]
ExecuteThread: '153' for queue: 'weblogic.kernel.Default (self-tuning)' has
been busy for "647" seconds working on the request
 4%2e0 HTTP/1.1
 X-ORCL-EMUR: https://hostname:3872/emd/main
 X-ORCL-EMAK: F5672E35F5D39E0F96A920FFB13A88F3
 Content-Length: 665716
 ECID-Context: 1.0000itPG^NSBLAq_4LnZ6i00068T000JHK;kXjE1ZDLIPHGj5PRj5SSgVPO^G
 Connection: Keep-Alive
 X-WebLogic-KeepAliveSecs: 30
 X-WebLogic-Force-JVMID: 1636863620
 ]", which is more than the configured time (StuckThreadMaxTime) of "600"
 seconds. Stack trace:
 "OMSHeartbeatThread" daemon prio=3 tid=0x00000001055e6800 nid=0x5f waiting on
 condition [0xffffffff07afe000]
 java.lang.Thread.State: TIMED_WAITING (sleeping)
 at java.lang.Thread.sleep(Native Method)

Also as per the ThreadDump generated in EMGC_OMS1.out file

"PingHeartBeatRecorder" daemon prio=3 tid=0x0000000103105800 nid=0x65
 runnable [0xffffffff040fe000]
 java.lang.Thread.State: RUNNABLE
 at Method)
 at Source)



Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms