My Oracle Support Banner

AGENT UNABLE TO START ORA.ASM DUE TO PROXYAGENT ERROR (Doc ID 2425931.1)

Last updated on FEBRUARY 03, 2019

Applies to:

Oracle Database - Enterprise Edition - Version 12.1.0.2 to 12.2.0.1 [Release 12.1 to 12.2]
Information in this document applies to any platform.

Symptoms

ora.asm is failed to start and status shows offline or intermediate.

$crsctl stat res -t -init

--------------------------------------------------------------------------------

Name Target State Server State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm  1   ONLINE  INTERMEDIATE

While starting the ora.asm getting below errors.

$ srvctl start asm 

CRS-2674: Start of 'ora.asm' on 'rwsbf06' failed 

CRS-5804: Communication error with agent process 

CRS-2674: Start of 'ora.asm' on 'rwsbi03' failed 

CRS-2528: Unable to place an instance of 'ora.asm' as all possible servers are occupied by the resource

Alert.log:

2016-09-07 14:13:30.012 [ORAAGENT(498)]CRS-5011: Check of resource "ora.asm" failed: details at "(:CLSN00006:)" in /scratch/oracle/base/diag/crs/rwsbf06/crs/trace/ohasd_oraagent_grid.trc"

2016-09-07 14:13:30.162 [ORAAGENT(498)]CRS-5011: Check of resource "ora.asm" failed: details at "(:CLSN00006:)" in "/scratch/oracle/base/diag/crs/rwsbf06/crs/trace/ohasd_oraagent_grid.trc"

2016-09-07 14:13:30.166 [ORAAGENT(498)]CRS-5011: Check of resource "ora.asm" failed: details at "(:CLSN00006:)" in "/scratch/oracle/base/diag/crs/rwsbf06/crs/trace/ohasd_oraagent_grid.trc"

ohasd_oraagent_grid_2.trc: 

2016-09-07 14:13:29.990 : AGFW:3877193472: {1:23503:371} Agent received the message: CLSAGFW_RESOURCE_ACTION[ora.asm 1 1] ID 4102:1636

2016-09-07 14:13:29.991 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] (:CLSN00114:) clsn_agent::action {

2016-09-07 14:13:29.991 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Agent::getResAgent resname:ora.asm version:12.1.0.1.0

2016-09-07 14:13:29.991 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Agent::getAgent __IS_HASD_AGENT = TRUE

2016-09-07 14:13:29.993 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Agent::Agent pAgent:0x7f01bc0ea8a0 &m_resName:0x7f01bc0ea920 resname:ora.asm

2016-09-07 14:13:29.993 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Agent::Agent pAgent:0x7f01bc0c5680 &m_resName:0x7f01bc0c5700 resname:ora.dummyOcrDg.dg

2016-09-07 14:13:29.993 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] DgpAgent::DgpAgent: enter { this:0x7f01bc0c5680

2016-09-07 14:13:29.993 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] DgpAgent::DgpAgent: resname ora.dummyOcrDg.dg compId dummyOcrDg asmCommonAgent 0x7f01bc0eaab0

2016-09-07 14:13:29.993 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] DgpAgent::DgpAgent: exit }

2016-09-07 14:13:29.993 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmCommonAgent::AsmCommonAgent m_pDgpAgent:bc0c5680

2016-09-07 14:13:29.994 : USRTHRD:3487557376: {1:23503:371} Thread:InstStartThread: thread constructor exit this:bc0ea2f0 m_pThnd:0 m_thndMX:bc0ea310, m_stopMX:bc0ea380 m_tintMX:bc0ea360 m_postMX:bc0ea340

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent constructor

2016-09-07 14:13:29.994 : USRTHRD:3487557376: {1:23503:371} Thread:ASM DedicatedThread thread constructor exit this:bc0ea510 m_pThnd:0 m_thndMX:bc0ea530, m_stopMX:bc0ea5a0 m_tintMX:bc0ea580 m_postMX:bc0ea560

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmAgent::AsmAgent Constructor resName ora.asm compId +ASM

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmAgent::refresh

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmAgent::refresh ORACLE_HOME = /scratch/oracle/crshome

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmAgent:getOracleSidAttrib 3 getResAttrib GEN_USR_ORA_INST_NAME oracle_sid:+ASM1

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmAgent:getOracleSidAttrib oracleSid:+ASM1

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmAgent::refresh ORACLE_SID = +ASM1

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::init oracleSid +ASM1

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::init resVersion 12.1.0.1.0

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] ConnectionPool::ConnectionPool 2 constructor this:bc119020 m_oracleHome:/scratch/oracle/crshome, m_oracleSid:+ASM1, m_usrOraEnv: m_instanceType:2 m_instanceVersion:12.1.0.1.0

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] ConnectionPool::ConnectionPool 2 constructor m_pResState:0x7f01bc119120

2016-09-07 14:13:29.994 : USRTHRD:3487557376: {1:23503:371} Thread:CssOsmGroupThread thread constructor exit this:bc0ea778 m_pThnd:0 m_thndMX:bc0ea798, m_stopMX:bc0ea808 m_tintMX:bc0ea7e8 m_postMX:bc0ea7c8

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmOhasdAgent constructor{

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmOhasdAgent constructor}

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Agent::setResType setResType type:ora.asm.type

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] AsmOhasdAgent Action {

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::check entry { m_connectionLock:0x7f01bc0ea4f0

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::check m_comment:

2016-09-07 14:13:29.994 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::check call GEN_RESTART

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::getGenRestart entry logLevel:1

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::getGenRestart getting attributeName:GEN_RESTART

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::getGenRestart exit GEN_RESTART:StartOption[1] }

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::check GEN_RESTART:StartOption[1]

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::checkstate: 000 entry s_SvcDbXLock:0x138ed28

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::checkState 010 type:ffff reason THA:0 oracleSid:+ASM1

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::checkState 030 new gimh oracleSid:+ASM1

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Gimh::constructor ohome:/scratch/oracle/crshome sid:+ASM1

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::checkState 040 getStatusOfConnection

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::checkState 050 m_prevState 3 poolState 0

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] InstAgent::checkState 060 call checkCbk

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] Gimh::check condition changes to (GIMH_NEXT_NUM) 0(Abnormal Termination) exists

2016-09-07 14:13:29.995 :CLSDYNAM:3487557376: [ ora.asm]{1:23503:371} [action] CLS_DRUID_REF(CLSN00006) AsmAgent::gimhChecks 100 failed gimh state 0

 

crsd_oraagent_grid.trc:

2016-09-07 14:13:29.943 : AGFW:1368995584: {1:23501:371} Agent received the message: RESOURCE_START[ora.asm 2 1] ID 4098:1416

2016-09-07 14:13:29.943 : AGFW:1368995584: {1:23501:371} Preparing START command for: ora.asm 2 1

2016-09-07 14:13:29.943 : AGFW:1368995584: {1:23501:371} ora.asm 2 1 state changed from: PLANNED_OFFLINE to: STARTING

2016-09-07 14:13:29.945 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] (:CLSN00107:) clsn_agent::start {

2016-09-07 14:13:29.945 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] AsmClusterAgent::start {

2016-09-07 14:13:29.945 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] AsmProxyAgent:start entry {

2016-09-07 14:13:29.945 : CLSCEVT:1361217280: (:CLSCE0028:)clsce_unsubscribe 0x7f7f40080430 successfully unsubscribed : 9

2016-09-07 14:13:29.945 : USRTHRD:1361217280: {1:23501:371} WorkerThread::removeWorker total workers: 4

2016-09-07 14:13:29.945 : USRTHRD:1361217280: {1:23501:371} Worker::schedule adding worker 'ReconnSub-AsmAgentSub+ASM' to thread 'ReconnectingSubscribers'

2016-09-07 14:13:29.946 : USRTHRD:1361217280: {1:23501:371} WorkerThread::addWorker total workers: 5

2016-09-07 14:13:29.946 : CLSCEVT:1365419776: clsce_subscribe 0x7f7f1846df00 filter='(^CRS_RESOURCE_PROFILE_CHANGE.*RESOURCE_CLASS='(scan_listener|listener|scan_vip|vip)')|(^CRS_RESOURCE_PROFILE_CHANGE.*TYPE='ora\.network\.type')|(^CRS_RESOURCE_STATE_CHANGE.*RESOURCE_CLASS='gns')', flags=1, handler=0xc69a38, arg=0x7f7f180bd040

2016-09-07 14:13:29.946 : USRTHRD:1361217280: {1:23501:371} Thread:ASM DedicatedThread stop { m_pThnd:(nil)

2016-09-07 14:13:29.946 : USRTHRD:1361217280: {1:23501:371} Thread:ASM DedicatedThread Thread stop:thread acquire m_pThnd:(nil) m_thndMX:0x7f7f18312a90

2016-09-07 14:13:29.946 : USRTHRD:1361217280: {1:23501:371} Thread:ASM DedicatedThread stop release m_thndMX:0x7f7f18312a90 }

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ProxyAgent::clearEnv

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ProxyAgent::setEnv

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] Utils::getResAttrib entry attribName:USR_ORA_OPI required:0 loglevel:1

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] Utils::getResAttrib: attribname USR_ORA_OPI value false len 5

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] Utils::getResAttrib attribname:USR_ORA_OPI value:false exit

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] Agent::valueOfAttribIs attrib: REASON compare value: user attribute value: user

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] Agent::valueOfAttribIs returns 1

2016-09-07 14:13:29.946 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] proxyAgent start with flag 64

2016-09-07 14:13:29.949 : CLSCEVT:1365419776: (:CLSCE0022:)clsce_subscribe 0x7f7f1846df00 successfully subscribed : 12

2016-09-07 14:13:29.969 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] clsnUtils::cmdIdIsStart CmdId:257

2016-09-07 14:13:29.969 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] AsmProxyAgent::setGenUsrOraInst updating GEN_USR_ORA_INST_NAME@SERVERNAME(rwsbf06) to +ASM1

2016-09-07 14:13:29.969 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] clsnUtils::setResAttrib nonPerX current value GEN_USR_ORA_INST_NAME value +ASM1

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] setResAttrib: attr GEN_USR_ORA_INST_NAME@SERVERNAME(rwsbf06) clsagfw attribVal +ASM1 clscrs tmpAttrValue +ASM1

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] Utils::setResAttrib clsagfw_modify_attribute attr GEN_USR_ORA_INST_NAME@SERVERNAME(rwsbf06) value +ASM1 retCode 0

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::stopConnection

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::broadcastEvent 000 entry { OHSid:/scratch/oracle/crshome+ASM1 s_ohSidEventMapLock:0x139d830 action:2

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::removeConnection connection count 0

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::removeConnection freed 0

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::stopConnection sid +ASM1 status 1

2016-09-07 14:13:29.974 : USRTHRD:1361217280: {1:23501:371} ConnectionPool::~ConnectionPool destructor this:400822a0 m_oracleHome:/scratch/oracle/crshome, m_oracleSid:+ASM1, m_usrOraEnv: m_pResState:0x7f7f401cdbe0

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::ConnectionPool 2 constructor this:18321160 m_oracleHome:/scratch/oracle/crshome, m_oracleSid:+ASM1, m_usrOraEnv: m_instanceType:2 m_instanceVersion:12.2.0.2.0

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::ConnectionPool 2 constructor m_pResState:0x7f7f180bd490

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] AsmProxyAgent:start new ConnectionPool 18321160

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::startConnection s_statusOfConnectionMap:0x139d920

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ConnectionPool::startConnection sid +ASM1 status 2

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] AsmProxyAgent:start exit }

2016-09-07 14:13:29.974 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] AsmClusterAgent::start ProxyAgent::action

2016-09-07 14:13:30.157 : AGFW:1368995584: {1:23505:371} Agent received the message: RESOURCE_PROBE[ora.asm 2 1] ID 4097:1435

2016-09-07 14:13:30.157 : AGFW:1368995584: {1:23505:371} Preparing CHECK command for: ora.asm 2 1

2016-09-07 14:13:34.381 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] ProxyAgent::ProxyAgentClscrsCB Propogate Error

2016-09-07 14:13:34.381 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] clsnUtils::error Exception type=2 string=

CRS-5804: Communication error with agent process

2016-09-07 14:13:34.382 : AGFW:1368995584: {1:23501:371} Agent sending reply for: RESOURCE_START[ora.asm 2 1] ID 4098:1416

2016-09-07 14:13:34.383 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] checkCrsStat 2 CLSCRS_STAT ret: 200

2016-09-07 14:13:34.383 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] clsnUtils::error Exception type=2 string=

2016-09-07 14:13:34.383 :CLSDYNAM:1361217280: [ ora.asm]{1:23501:371} [start] checkCrsStat 2 clscrs_res_get_op_status CLSCRS_STAT status 200 err_msg CRS-0200: CRS general failure. No reason is provided.

crsd.trc:

2016-09-07 14:13:29.936 :UiServer:2157872896: {1:23501:371} Container [ Name: UI_START

API_HDR_VER:

TextMessage[3]

CLIENT:

TextMessage[]

CLIENT_NAME:

TextMessage[/usr/bin/java]

CLIENT_PID:

TextMessage[12841]

CLIENT_PRIMARY_GROUP:

TextMessage[oinstall]

FORCE_TAG:

TextMessage[1]

LOCALE:

TextMessage[AMERICAN_AMERICA.US7ASCII]

QUEUE_TAG:

TextMessage[1]

RESOURCE:

TextMessage[ora.asm]

]

2016-09-07 14:13:29.936 :UiServer:2157872896: {1:23501:371} Sending to PE. ctx= 0x7fede407eb00, ClientPID=12841 set Properties (grid,38257)

2016-09-07 14:13:29.942 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server received the message: RESOURCE_START[ora.asm 2 1] ID 4098:4817

2016-09-07 14:13:29.942 :UiServer:2157872896: {1:23501:371} Response: c4|5!ORDERk7|MESSAGEt52|CRS-2672: Attempting to start 'ora.asm' on 'rwsbf06'k7|MSGTYPEt1|3k5|OBJIDt11|ora.asm 2 1k4|WAITt1|0

2016-09-07 14:13:29.942 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server forwarding the message: RESOURCE_START[ora.asm 2 1] ID 4098:4817 to the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:29.970 :UiServer:2157872896: {1:23503:371} Sending to PE. ctx= 0x7fede407fe80, ClientPID=28072 set Properties (grid,21858), tint: {1:23501:371}

2016-09-07 14:13:29.973 :UiServer:2157872896: {1:23503:371} Done for ctx=0x7fede407fe80

2016-09-07 14:13:30.029 :UiServer:2157872896: {1:23501:374} Container [ Name: UI_STOP

API_HDR_VER:

TextMessage[3]

ASYNC_TAG:

TextMessage[1]

CLIENT:

TextMessage[]

CLIENT_NAME:

TextMessage[/scratch/oracle/crshome/bin/oraagent.bin]

CLIENT_PID:

TextMessage[498]

CLIENT_PRIMARY_GROUP:

TextMessage[oinstall]

FILTER:

TextMessage[(((((STATE==ONLINE)||(STATE==INTERMEDIATE))||(STATE==UNKNOWN))&&((NAME==ora.asm)&&(LAST_SERVER==rwsbf06)))&&(STATE_DETAILScoUnder THA))]

FILTER_TAG:

TextMessage[1]

LOCALE:

TextMessage[AMERICAN_AMERICA.AL32UTF8]

TINT:

TextMessage[{1:23503:371}]

T_HA_FINISH_TAG:

TextMessage[1]

]

2016-09-07 14:13:30.029 :UiServer:2157872896: {1:23505:371} Sending to PE. ctx= 0x7fede407ff40, ClientPID=498 set Properties (grid,38386), tint: {1:23503:371}

2016-09-07 14:13:30.081 :UiServer:2157872896: {1:23505:371} Sending to PE. ctx= 0x7fede40809a0, ClientPID=498 set Properties (grid,38707), tint: {1:23503:371}

2016-09-07 14:13:30.085 :UiServer:2157872896: {1:23505:371} Done for ctx=0x7fede40809a0

2016-09-07 14:13:30.133 :UiServer:2157872896: {1:23505:371} Sending to PE. ctx= 0x7fede4080a40, ClientPID=498 set Properties (grid,38782), tint: {1:23503:371}

2016-09-07 14:13:30.136 :UiServer:2157872896: {1:23505:371} Done for ctx=0x7fede4080a40

2016-09-07 14:13:30.153 :UiServer:2157872896: {1:23501:379} Container [ Name: UI_CHECK

API_HDR_VER:

TextMessage[3]

CLIENT:

TextMessage[]

CLIENT_NAME:

TextMessage[/scratch/oracle/crshome/bin/oraagent.bin]

CLIENT_PID:

TextMessage[498]

DELAY_SECS:

TextMessage[0]

HOST:

TextMessage[rwsbf06]

HOST_TAG:

TextMessage[1]

LOCALE:

TextMessage[AMERICAN_AMERICA.AL32UTF8]

RESOURCE:

TextMessage[ora.asm]

TINT:

TextMessage[{1:23503:371}]

]

2016-09-07 14:13:30.153 :UiServer:2157872896: {1:23505:371} Sending to PE. ctx= 0x7fede4080710, ClientPID=498 set Properties (grid,38707), tint: {1:23503:371}

2016-09-07 14:13:30.156 : AGFW:2174682880: {1:23505:371} Agfw Proxy Server received the message: RESOURCE_PROBE[ora.asm 2 1] ID 4097:4823

2016-09-07 14:13:30.156 : AGFW:2174682880: {1:23505:371} Agfw Proxy Server forwarding the message: RESOURCE_PROBE[ora.asm 2 1] ID 4097:4823 to the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:30.156 :UiServer:2157872896: {1:23505:371} Response: c1|7!UI_DATAk7|ora.asmt1|0

2016-09-07 14:13:30.156 :UiServer:2157872896: {1:23505:371} Done for ctx=0x7fede4080710

2016-09-07 14:13:34.383 : AGFW:2174682880: {1:23501:371} Received the reply to the message: RESOURCE_START[ora.asm 2 1] ID 4098:1416 from the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:34.383 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server sending the reply to PE for message:RESOURCE_START[ora.asm 2 1] ID 4098:4817

2016-09-07 14:13:34.385 : AGFW:2174682880: {1:23501:371} Received the reply to the message: RESOURCE_START[ora.asm 2 1] ID 4098:1416 from the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:34.385 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server sending the reply to PE for message:RESOURCE_START[ora.asm 2 1] ID 4098:4817

2016-09-07 14:13:34.386 :UiServer:2157872896: {1:23501:371} Response: c4|5!ORDERk7|MESSAGEt48|CRS-5804: Communication error with agent processk7|MSGTYPEt1|1k5|OBJIDt11|ora.asm 2 1k4|WAITt1|0

2016-09-07 14:13:34.409 : AGFW:2174682880: {1:23501:371} Received the reply to the message: RESOURCE_START[ora.asm 2 1] ID 4098:1416 from the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:34.409 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server sending the last reply to PE for message:RESOURCE_START[ora.asm 2 1] ID 4098:4817

2016-09-07 14:13:34.409 :UiServer:2157872896: {1:23503:371} Sending to PE. ctx= 0x7fede40808f0, ClientPID=28072 set Properties (grid,17423), tint: {1:23501:371}

2016-09-07 14:13:34.412 :UiServer:2157872896: {1:23503:371} Done for ctx=0x7fede40808f0

2016-09-07 14:13:34.412 :UiServer:2157872896: {1:23501:371} Response: c4|5!ORDERk7|MESSAGEt48|CRS-2674: Start of 'ora.asm' on 'rwsbf06' failedk7|MSGTYPEt1|1k5|OBJIDt11|ora.asm 2 1k4|WAITt1|0

2016-09-07 14:13:34.413 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server received the message: RESOURCE_CLEAN[ora.asm 2 1] ID 4100:4854

2016-09-07 14:13:34.413 : AGFW:2174682880: {1:23501:371} Agfw Proxy Server forwarding the message: RESOURCE_CLEAN[ora.asm 2 1] ID 4100:4854 to the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:34.413 :UiServer:2157872896: {1:23501:371} Response: c4|5!ORDERk7|MESSAGEt52|CRS-2679: Attempting to clean 'ora.asm' on 'rwsbf06'k7|MSGTYPEt1|3k5|OBJIDt11|ora.asm 2 1k4|WAITt1|0

2016-09-07 14:13:34.428 : AGFW:2174682880: {0:7:9} Agfw Proxy Server received the message: RESOURCE_STATUS[Proxy] ID 20481:475

2016-09-07 14:13:34.428 : AGFW:2174682880: {0:7:9} Verifying msg rid = ora.asm 2 1

2016-09-07 14:13:34.428 : AGFW:2174682880: {0:7:9} Agfw Proxy Server replying to the message: RESOURCE_STATUS[Proxy] ID 20481:475

2016-09-07 14:13:34.428 : AGFW:2174682880: {1:23505:371} Received the reply to the message: RESOURCE_PROBE[ora.asm 2 1] ID 4097:1435 from the agent /scratch/oracle/crshome/bin/oraagent_grid

2016-09-07 14:13:34.428 : AGFW:2174682880: {1:23505:371} ora.asm 2 1 received state from probe request. Old state = OFFLINE, New state = OFFLINE

2016-09-07 14:13:34.429 : AGFW:2174682880: {1:23505:371} Agfw Proxy Server sending the last reply to PE for message:RESOURCE_PROBE[ora.asm 2 1] ID 4097:4823

2016-09-07 14:13:34.450 :UiServer:2157872896: {1:23503:371} Sending to PE. ctx= 0x7fede40808f0, ClientPID=28072 set Properties (grid,17423), tint: {1:23501:371}

2016-09-07 14:13:34.453 :UiServer:2157872896: {1:23503:371} Done for ctx=0x7fede40808f0

 Stack trace:

gdb /scratch/app/12.2.0/grid/bin/oraagent.bin /scratch/app/oracle/crsdata/rws1270138/core/core.oraagent.bin.13783 

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fffba106000 
Core was generated by `/scratch/app/12.2.0/grid/bin/oraagent.bin'. Program terminated with signal 11, Segmentation fault. 

#0  0x00007f2db06fda36 in SlfFopen () from 
/scratch/app/12.2.0/grid/lib/libclntshcore.so.12.1 Missing separate debuginfos, use: debuginfo-install 
glibc-2.12-1.166.el6_7.7.x86_64 libaio-0.3.107-10.el6.x86_64 
libattr-2.4.44-7.el6.x86_64 libcap-2.16-5.5.el6.x86_64 
libgcc-4.4.7-4.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64 
numactl-devel-2.0.7-8.el6.x86_64 (gdb) bt 
.... 
#4  0x00007f2db2444a4e in sskgmsmr_op () from  /scratch/app/12.2.0/grid/lib/libclntsh.so.12.1 
#5  0x0000000000d61513 in gimh_init_inst_ctx () 
#6  0x00000000005354df in clsn_agent::Gimh::check (this=0x7f2d986c8080, 
actx=0x7f2d88085e80, gimh_states=0x7f2dab485172) at clsnGimh.cpp:159 
#7  0x000000000048a72e in clsn_agent::AsmAgent::gimhChecks  (this=0x7f2d840c9610, actx=0x7f2d88085e80, pGimh=0x7f2d986c8080, druid= 
    "", agfwStates=..., stateStrs=..., instanceRestarted=0x0) at clsnAsmAgent.cpp:511 
#8  0x000000000048b6fc in clsn_agent::AsmAgent::checkCbk 
(this=0x7f2d840c9610, actx=0x7f2d88085e80, pGimh=0x7f2d986c8080, druid="") at clsnAsmAgent.cpp:782 
#9  0x00000000005c88fc in clsn_agent::InstAgent::checkState 
(this=0x7f2d840c9610, actx=0x7f2d88085e80) at clsnInstAgent.cpp:1617 
#10 0x00000000005bfa2a in clsn_agent::InstAgent::check (this=0x7f2d840c9610, 
actx=0x7f2d88085e80) at clsnInstAgent.cpp:1449 
#11 0x00000000007324b4 in clsn_agent::AsmOhasdAgent::stop 
(this=0x7f2d840c9610, actx=0x7f2d88085e80) at clsnAsmOhasdAgent.cpp:197 
#12 0x00000000005bfd83 in clsn_agent::InstAgent::check (this=0x7f2d840c9610, 
actx=0x7f2d88085e80) at clsnInstAgent.cpp:1473 
#13 0x00000000007324b4 in clsn_agent::AsmOhasdAgent::stop 
(this=0x7f2d840c9610, actx=0x7f2d88085e80) at clsnAsmOhasdAgent.cpp:197 

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
References

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.