My Oracle Support Banner

Exadata: Instance eviction with IPC sendtimeout messages. (Doc ID 1404576.1)

Last updated on AUGUST 04, 2018

Applies to:

Linux OS - Version Oracle Linux 5.5 to Oracle Linux 5.5 [Release OL5U5]
Oracle Exadata Storage Server Software - Version to [Release 11.2]
Information in this document applies to any platform.


Intermittent database/ASM instance evictions after the Exadata storage software version has been upgraded to

Fri Dec 23 05:58:29 2011
Errors in file /u01/app/oracle/diag/rdbms/ctra/ctra2/trace/ctra2_lmd0_1285.trc:
IPC Send timeout to 1.2 inc 16 for msg type 36 from opid 177
IPC Send timeout: Terminating pid 177 osid 30942
Fri Dec 23 06:02:45 2011
minact-scn master exiting with err:12751
Fri Dec 23 06:03:40 2011
LMS1 (ospid: 1291) has detected no messaging activity from instance 1
LMS1 (ospid: 1291) issues an IMR to resolve the situation
Please check LMS1 trace file for more detail.
Fri Dec 23 06:03:40 2011
Suppressed nested communications reconfiguration: instance_number 1
Detected an inconsistent instance membership by instance 1
Fri Dec 23 06:03:41 2011
Received an instance abort message from instance 1
Please check instance 1 alert and LMON trace files for detail.
LMS0 (ospid: 1287): terminating the instance due to error 481

The LMS/LMON/LMD trace files indicate that there were heartbeats  time outs across the instances and that resulted in the eviction.

*** 2011-12-23 06:02:12.550
===== Idle Connection Overview =====
Idle connections [0x307c3]: 2
IdleConn List: 2[r:0.23926378,t:0x307c3]
GSIPC:IKILL: ping to inst 2 start 198595 now 198878 icktm 280 psm 0
Timeout after 280 seconds
Reporting Communication error with instance 2

At the same time, the rds-ping statistics collected in the OS Watcher show that the RDS connectivity issues between some or all of the compute nodes.

[INFO:2011-12-23-05:52:35] To database hosts: rds-ping -c 1 -i 5 -I <<<<



Exadata cell software upraded to which contain the OFA version 1.5.1-4.0.53.


To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.