CRS root.sh Script Failing on Second Node When MTU Larger than 1500 (Doc ID 1085885.1)

Last updated on MAY 22, 2017

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.1.0 and later
Oracle VM - Version 2.1 to 2.2 [Release OVM21 to OVM22]
Linux x86-64
Linux x86

Symptoms

Installing the 11gR2 RAC Grid Infrastructure running the post install configuration root.sh script it's failing reporting the following errors:

(...)
CRS-2676: Start of 'ora.diskmon' on 'rc-vrac2' succeeded
CRS-2674: Start of 'ora.cssd' on 'rc-vrac2' failed
CRS-2679: Attempting to clean 'ora.cssd' on 'rc-vrac2'
CRS-2681: Clean of 'ora.cssd' on 'rc-vrac2' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'rc-vrac2'
CRS-2677: Stop of 'ora.diskmon' on 'rc-vrac2' succeeded
CRS-4000: Command Start failed, or completed with errors.
CRS-2672: Attempting to start 'ora.cssd' on 'rc-vrac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rc-vrac2'
CRS-2674: Start of 'ora.diskmon' on 'rc-vrac2' failed
CRS-2679: Attempting to clean 'ora.diskmon' on 'rc-vrac2'
CRS-5016: Process "/u01/grid/11.2.0/bin/diskmon" spawned by agent "/u01/grid/11.2.0/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/log/rc-vrac2/agent/ohasd/orarootagent_root/orarootagent_root.log"
CRS-2681: Clean of 'ora.diskmon' on 'rc-vrac2' succeeded
CRS-2674: Start of 'ora.cssd' on 'rc-vrac2' failed
CRS-2679: Attempting to clean 'ora.cssd' on 'rc-vrac2'
CRS-2681: Clean of 'ora.cssd' on 'rc-vrac2' succeeded
CRS-4000: Command Start failed, or completed with errors.
Command return code of 1 (256) from command: /u01/grid/11.2.0/bin/crsctl start resource ora.ctssd -init -env USR_ORA_ENV=CTSS_REBOOT=TRUE
Start of resource "ora.ctssd -init -env USR_ORA_ENV=CTSS_REBOOT=TRUE" failed
Failed to start CTSS
Failed to start Oracle Clusterware stack



The diskmon log is reporting the following:

(...)
2010-04-16 11:43:01.405: [ DISKMON][22352:4054047056] dskm_clss_ini22: clssgsgrpstat failed with clssret 8
2010-04-16 11:43:01.405: [ DISKMON][22352:4054047056] dskm_rac_ini1: dskm_clss_ini failed with error 56830 ... exiting
2010-04-16 11:43:01.405: [ DISKMON][22352:4054047056] dskm_nfy_kgzf1: notified thread kgzf disabled
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_proc_kgzf_handshake5: got the post from the hb thread
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_send_slave1: error 56822 from skgznp_write_msg to slave id 1, client kgzf/22368 reid cid=383214bf5ae1dfd9ff8c14ececc45501,icin=166875107,nmn=2,lnid=166875108,gid=0,gin=0,gmn=0,umemid=0,opid=0,opsn=0,lvl=node hdr=0xfece0100
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_send_slave2: error 56824 at location skgznpwm2 - send() - Broken pipe
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_proc_kgzf_handshake8: error 56822 from dskm_send_slave
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_process_msg7: processed msg 0 type KGZM_KGZF_HANDSHAKE (0x0010), retcode 56822
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_process_msg8: error 56822 processing msg type KGZM_KGZF_HANDSHAKE (0x0010) from client kgzf/22368, reid cid=383214bf5ae1dfd9ff8c14ececc45501,icin=166875107,nmn=2,lnid=166875108,gid=0,gin=0,gmn=0,umemid=0,opid=0,opsn=0,lvl=node hdr=0xfece0100
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_slave_thrd_main3: peer disconnected
2010-04-16 11:43:01.405: [ DISKMON][22352:4028868944] dskm_slave_thrd_main5: client kgzf/22368 disconnected, reid cid=383214bf5ae1dfd9ff8c14ececc45501,icin=166875107,nmn=2,lnid=166875108,gid=0,gin=0,gmn=0,umemid=0,opid=0,opsn=0,lvl=node hdr=0xfece0100
2010-04-16 11:43:01.406: [ DISKMON][22352:4054047056] dskm_rac_thrd_main3: dskm_rac_ini failed with error 56830 ... exiting
2010-04-16 11:43:01.406: [ DISKMON][22352:4054047056] SHUTDOWN ABORT due to error 56830
2010-04-16 11:43:01.406: [ DISKMON][22352:4054047056] dskm_rac_thrd_main: exiting
2010-04-16 11:43:03.244: [ DISKMON][22352:4139341552] dskm_cleanup_thrds: cleaning up the rac event handling thread tid 4054047056
2010-04-16 11:43:03.244: [ DISKMON][22352:4139341552] dskm_cleanup_thrds: cleaning up slave 1 thread tid 4028868944
2010-04-16 11:43:04.176: [ DISKMON][22352:4037261648] dskm_slave_thrd_main5: client orarootagent/22336 disconnected, reid cid=DUMMY,icin=-1,nmn=-1,lnid=-1,gid=-1,gin=-1,gmn=-1,umemid=-1,opid=-1,opsn=-1,lvl=process hdr=0xfece0100
2010-04-16 11:43:04.252: [ DISKMON][22352:4139341552] dskm_cleanup_thrds: cleaning up slave 0 thread tid 4037261648
2010-04-16 11:43:04.756: [ DISKMON][22352:4045654352] dskm_oss_thrd_main2: posted
2010-04-16 11:43:04.756: [ DISKMON][22352:4045654352] dskm_oss_thrd_main: exiting
2010-04-16 11:43:05.260: [GIPCXCPT] gipcInternalSend: connection not valid for send operation endp 0x684840 [000000000000007b] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=ipc)(KEY=)(GIPCID=db410257-085aee83-22352))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rc-vrac2_)(GIPCID=085aee83-db410257-22368))', numPend 0, numReady 0, numDone 0, numDead 0, numTransfer 0, objFlags 0x5760, pidPeer 0, flags 0x3861e, usrFlags 0x20010 }, ret gipcretConnectionLost (12)
2010-04-16 11:43:05.260: [GIPCXCPT] gipcSendSyncF [clsssServerRPC : clsss.c : 5922]: EXCEPTION[ ret gipcretConnectionLost (12) ]  failed to send on endp 0x684840 [000000000000007b] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=ipc)(KEY=)(GIPCID=db410257-085aee83-22352))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=ipc)(KEY=OCSSD_LL_rc-vrac2_)(GIPCID=085aee83-db410257-22368))', numPend 0, numReady 0, numDone 0, numDead 0, numTransfer 0, objFlags 0x5760, pidPeer 0, flags 0x3861e, usrFlags 0x20010 }, addr 0000000000000000, buf 0x7ffffed24710, len 80, flags 0x0
2010-04-16 11:43:05.260: [ CSSCLNT]clsssServerRPC: send failed with err 12, msg type 7
2010-04-16 11:43:05.260: [ CSSCLNT]clsssCommonClientExit: RPC failure, rc 3

 

Changes

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms