'IPC Send Timeout Detected' errors between QMON Processes after RAC reconfiguration (Doc ID 458912.1)

Last updated on APRIL 16, 2017

Applies to:

Oracle Database - Enterprise Edition - Version 10.2.0.1 to 10.2.0.4 [Release 10.2]
Information in this document applies to any platform.
Oracle Server Enterprise Edition - Version: 10.2.0.1 to 10.2.0.4


Symptoms

After RAC reconfiguration, intermittently the QMNC processes on different nodes cannot communicate with each other in order to allow queue table repartitioning to occur.  This results in node affinity not changing and propagation schedules remaining on a non-primary node and messages not being propagated.

You may see the qmc process on node 3 send a message to the qmnc process on node 1 presumably because node 1 owns schedules which should be owned by node 3 so we see

alert_<dbname>.log

IPC Send timeout detected.Sender: ospid 22795
Receiver: inst 1 binc 4 ospid 17307

In node 1 alert_<dbname>.log we see messages like

IPC Send timeout detected.Sender: ospid 17307
Receiver: inst 2 binc 2 ospid 13661

and also
.
IPC Send timeout detected. Receiver ospid 17307
Wed Jul 11 07:42:01 2007
Errors in file /haclu/64bit/app/oracle/admin/H102/bdump/h1021_qmnc_17307.trc:
IPC Send timeout detected. Receiver ospid 17307
Wed Jul 11 07:42:04 2007
Errors in file /haclu/64bit/app/oracle/admin/H102/bdump/h1021_qmnc_17307.trc:
Wed Jul 11 07:42:04 2007

and

ORA-4021: timeout occurred while waiting to lock object D1.D115Q_EXAMPLE

In node 2 alert_<dbname>.log we see messages of the form

Wed Jul 11 07:41:42 2007
IPC Send timeout detected. Receiver ospid 13661
Wed Jul 11 07:41:42 2007
Errors in file /haclu/64bit/app/oracle/admin/H102/bdump/h1022_qmnc_13661.trc:

and in /haclu/64bit/app/oracle/admin/H102/bdump/h1022_qmnc_13661.trc:

kwqmnstslv: less than 10s since last failed ksvcreate.
Couldn't start a new slave

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms