The Enterprise Manager Agent Status (emctl status agent) shows Agent Unreachable with Possible Errors: Collection Status: Disabled By Upload Manager; Exceeded max. amount of upload data; Collections disabled due to DISK_FULL (Doc ID 396238.1)

Last updated on APRIL 16, 2016

Applies to:

Enterprise Manager Base Platform - Version 10.2.0.1 to 12.1.0.2.0 [Release 10.2 to 12.1]
Information in this document applies to any platform.
Checked for relevance on 02-Dec-2013

Symptoms

12c Agent

$AGENT_INST/bin/emctl status agent

Collection Status Shows '[DISK_FULL]' message

Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version : 12.1.0.1.0
OMS Version : 12.1.0.1.0
Protocol Version : 12.1.0.1.0
Agent Home : /int1/oracle/oem/agent/agent_inst
Agent Binaries : /int1/oracle/oem/agent/core/12.1.0.1.0
Agent Process ID : 6635
Parent Process ID : 6531
Agent URL : https://myagent.oracle.com:3872/emd/main/
Repository URL : https://myoms.oracle.com:1159/empbs/upload
Started at : 2013-05-23 14:13:38
Started by user : emuser
Last Reload : (none)
Last successful upload : (none)
Last attempted upload : (none)
Total Megabytes of XML files uploaded so far : 0
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0
Available disk space on upload filesystem : 98.72%
Collection Status : [DISK_FULL]
Last attempted heartbeat to OMS : 2013-05-23 14:20:42
Last successful heartbeat to OMS : 2013-05-23 14:20:42

 
agent_inst/sysman/log/gcagent.log reports the following message

    2013-05-29 11:25:31,984 [1:3305B9] DEBUG - Upload breadcrumb=Starting a purge  for overage/fileOverage = -2437.0/0
    2013-05-29 11:25:31,984 [1:3305B9] DEBUG - Upload breadcrumb=Purge unable to  purge requested overages
    2013-05-29 11:25:31,984 [1:3305B9] WARN - Upload system can't accept inbound  message; can't purge
    2013-05-29 11:25:31,985 [1:3305B9] DEBUG - Collections disabled due to  DISK_FULL
 
A notification may also have triggered with  "Message=WARNING:Agent File system is almost FULL. Metric Collection is  DISABLED."

$AGENT_INSt/bin/emctl upload agent

Oracle Enterprise Manager Cloud Control 12c Release 2
Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload error:full upload has failed: Disabling the upload system due to protocol internal programming exception.

10g/11g Agent

- In the Agent machine, the output of the 'emctl status agent' command shows:

cd <AGENT_HOME>/bin
emctl status agent
Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version : 10.2.0.5.0
OMS Version : 10.2.0.5.0
Protocol Version : 10.2.0.5.0
Agent Home : /u01/app/oracle/product/10.2.0/agent10g
Agent binaries : /u01/app/oracle/product/10.2.0/agent10g
Agent Process ID : 69560
Parent Process ID : 68602
Agent URL : https://agentmachine.domain:3872/emd/main
Repository URL : https://omsmachine.domain:1159/em/upload
Started at : 2010-02-03 10:21:16
Started by user : oracle
Last Reload : 2010-02-03 10:26:19
Last successful upload : (none)
Last attempted upload : (none)
Total Megabytes of XML files uploaded so far : 0.00
Number of XML files pending upload : 1968
Size of XML files pending upload(MB) : 60.65
Available disk space on upload filesystem : 30.87%
Collection Status : Disabled by Upload Manager
Last attempted heartbeat to OMS : 2010-02-05 13:29:39
Last successful heartbeat to OMS : unknown
---------------------------------------------------------------
Agent is Running and Ready


The 'Collection Status : Disabled by Upload Manager' indicates that the Agent has disabled all the metric collections for its targets as it has already crossed the limits set for the number of files pending or their total size. When the Metric Collections are disabled:

- The <AGENT_HOME>/sysman/log/emagent.trc shows:

2006-10-19 10:41:23 Thread-19 ERROR upload: Exceeded max. amount of upload data: 12187 files, 24.995519 MB Data. 34.06% of disk used. Disabling collections.
2006-10-19 10:41:23 Thread-19 WARN collector: Disable collector


- Forcing a manual upload using:

cd <AGENT_HOME>/bin
emctl upload

can return:

upload failed with ret=-10

OR

EMD upload error: Upload was successful but collections currently disabled - disk full


1. The 'upload failed with ret=-10' error indicates that the file uploads to the OMS are failing.
2. The output 'EMD upload error: Upload was successful but collections currently disabled - disk full' indicates that the file uploads to the OMS are going through and the number of pending files may reduce slowly.

-  If notifications are enabled for 'Agent Unreachable' Availability state, then you could receive notifications of the form:

Target Name=<AGENT_HOST>:3872
Target Type=Agent
Host=oracle.com
Metric=Status
Timestamp=<timestamp>
Severity=Unreachable Start
Message=Agent has stopped monitoring. The following errors are reported : COLL_DISABLED.
Notification Rule Name=Agents Unreachable
Notification Rule Owner=SYSMAN
Notification Count=1

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms