My Oracle Support Banner

[OMC]: Gateway Agent is Crashing Frequently, gcagent.log Shows " java.net.SocketTimeoutException: Read timed out" (Doc ID 2423658.1)

Last updated on JULY 16, 2018

Applies to:

OMC Application Performance Monitoring Cloud Service - Version N/A and later
Information in this document applies to any platform.

Symptoms

OMC Gateway agent crashing/going in abort status after sometime of startup:
================
[oracle@omc bin]$ ./omcli status agent
Oracle Management Cloud Gateway
Copyright (c) 1996, 2018 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Version : 1.29.0
State Home : /u01/omc_gateway/agent_inst
Log Directory : /u01/omc_gateway/agent_inst/sysman/log
Binaries Location : /u01/omc_gateway/GATEWAY_LINUX.X64_180426.1847/core/1.29.0
Process ID : 14917
Parent Process ID : 20967
URL : https://agenthostname:3872/emd/main/
Started at : 2018-06-27 00:47:41
Started by user : oracle
Operating System : Linux version 2.6.39-400.214.5.1.el6uek.x86_64 (amd64)
Registered entities : 104
Sender Status : FUNCTIONAL
Data Receiver Upload Status : UNKNOWN
Last successful upload : 2018-06-27 00:56:26
Last attempted upload : 2018-06-27 00:56:24
Gateway Pending Files (MB) : 9.91
Gateway Pending Files : 102
Backoff Expiration : (none)

---------------------------------------------------------------
Agent is Running and Ready
====================

From <AGENT_INST_HOME>/sysman/log/emagent.nohup
================
----- 2018-06-28 13:25:10,982::20371::Time elapsed between Launch of Watchdog process and execing EMAgent is 809 secs -----
----- 2018-06-28 13:50:12,795::11007::Checking status of EMAgent : 20371 -----
----- 2018-06-28 13:50:12,795::11007::Abnormality reported for EMAgent : 20371 -----
----- 2018-06-28 13:50:12,795::11007::Debugging component EMAgent -----
----- 2018-06-28 13:50:12,795::generate first thread dump file for diagnosis -----
----- 2018-06-28 13:50:23,241::generate second thread dump file for diagnosis -----
----- 2018-06-28 13:50:23,473::generate Threads.20371lsof.1 for diagnosis -----
----- Attempting to kill EMAgent : 20371 -----
----- 2018-06-28 13:50:34,683::11007::EMAgent exited at 2018-06-28 13:50:34,683 with signal 9 -----
----- 2018-06-28 13:50:34,683::11007::EMAgent either hung or in abnormal state. -----
----- 2018-06-28 13:50:34,683::11007::EMAgent will be restarted/thrashed. -----
----- 2018-06-28 13:50:34,683::11007::writeAbnormalExitTimestampToAgntStmp: exitCause=ABNORMAL : restartRequired=1 -----
----- 2018-06-28 13:50:34,685::11007::Restarting EMAgent. -----
=====================

 

From <AGENT_INST_HOME>/sysman/log/gcagent.log:
=======================
018-06-28 06:47:39,616 [4261:1CF5D51A] INFO - GetNextWorkItemInvocation -->req--> GET https://xxxxxxxxxxxxxxxxxx.oraclecloud.com/static/workdepot/queue/B1B08A6FE920BE23196EB7CC314701DD
2018-06-28 06:47:40,723 [4261:1CF5D51A] INFO - GetNextWorkItemInvocation <--rsp<-- GET https://xxxxxxxxxxxxxxxxxx.oraclecloud.com/static/workdepot/queue/B1B08A6FE920BE23196EB7CC314701DD: [204]
2018-06-28 06:48:01,471 [3612:65630385:HTTP Listener-3612] WARN - *jetty*: /static/regmanager/agents
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
=======================

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.