EM 12c: Enterprise Manager 12.1.0.4 Cloud Control OMS Restarts Due to Error: PbsAdminMsgListener thread timed out (Doc ID 1996844.1)

Last updated on OCTOBER 03, 2016

Applies to:

Enterprise Manager Base Platform - Version 12.1.0.4.0 to 12.1.0.5.0 [Release 12.1]
Information in this document applies to any platform.

Symptoms

The 12.1.0.4 Enterprise Manager (EM) 12.1.0.4 Cloud Control OMS restarts or crashes with the following error in $EM_INSTANCE_BASE/gc_inst/em/EMGC_OMS1/sysman/log/emctl.msg

PbsAdminMsgListener error: PbsAdminMsgListener thread timed out.
Critical error err=3 detected in module PbsAdminMsgListener
OMS will be restarted. A full thread dump will be generated in the log file
/u01/app/oracle/em12c/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out

 

$EM_INSTANCE_BASE/gc_inst/em/EMGC_OMS1/sysman/log/oms_diag_info_TimeStamp.msg file reports the following errors:

PbsAdminMsgListener prio=10 id=127 state=BLOCKED
oracle.sysman.core.pbs.blackout.BlackoutCache.refreshBlackoutWindowCache(BlackoutCache.java:229)
oracle.sysman.core.pbs.blackout.BlackoutCache.refreshCache(BlackoutCache.java:195)
oracle.sysman.core.pbs.blackout.BlackoutAdminQueueListener.handleAdminMsg(BlackoutAdminQueueListener.java:49)
oracle.sysman.emdrep.adminmsg.AdminMsgListener.processAdminMessage(AdminMsgListener.java:659)
oracle.sysman.core.pbs.adminmsg.PbsAdminMsgListener.run(PbsAdminMsgListener.java:180)

 or

the OMS hangs with the following errors in $EM_INSTANCE_BASE/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out  

 DEADLOCK DETECTED:
==================

[deadlocked thread] Thread-690033:
---------------------------------

Thread 'Thread-690033' is waiting to acquire lock 'java.lang.Class@2dd068ce' that is held by thread 'GCLoader[data] -'
Stack trace:
------------
oracle.sysman.core.pbs.blackout.BlackoutCache.setCacheData(BlackoutCache.java:156)
oracle.sysman.core.pbs.blackout.BlackoutCache.refreshBlackoutWindowCache(BlackoutCache.java:244)
oracle.sysman.core.pbs.blackout.BlackoutCachePeriodicThread.run(BlackoutCachePeriodicThread.java:40)

  

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms