gipcd not coming up
(Doc ID 2034669.1)
Last updated on JULY 24, 2023
Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.4 and laterOracle Database Exadata Cloud Machine - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Database Cloud Exadata Service - Version N/A and later
Information in this document applies to any platform.
Symptoms
Cluster node restarted but GI is not coming up as ora.gipcd fails to start:
ora.gpnpd
1 ONLINE ONLINE racnode2
ora.gipcd
1 ONLINE OFFLINE
1 ONLINE ONLINE racnode2
ora.gipcd
1 ONLINE OFFLINE
GI alert.log
2015-07-17 21:30:01.462:
[ohasd(14353)]CRS-2765:Resource 'ora.gipcd' has failed on server 'racnode2'.
2015-07-17 21:30:03.480:
[ohasd(14353)]CRS-2765:Resource 'ora.gipcd' has failed on server 'racnode2'.
2015-07-17 21:30:05.499:
[ohasd(14353)]CRS-2765:Resource 'ora.gipcd' has failed on server 'racnode2'.
2015-07-17 21:30:03.480:
[ohasd(14353)]CRS-2765:Resource 'ora.gipcd' has failed on server 'racnode2'.
2015-07-17 21:30:05.499:
gipcd.log
2015-07-17 21:30:01.335: [ GIPCLIB][4144572176] gipclibSetTraceLevel: to set level to 0
2015-07-17 21:30:01.337: [GIPCDCLT][4152977168] gipcdClientThread: gipcListen() failed (1) on endp 000000000000024c
2015-07-17 21:30:01.337: [ GIPCD][4152977168] gipcdSetThreadState: changing the status of clientThread. current status gipcdThreadStatusInit desired status gipcdThreadStatusOffline
2015-07-17 21:30:01.337: [ GIPCLIB][4152977168] gipclibMapSearch: gipcMapSearch() -> gipcMapGetNodeAddr() failed: ret:gipcretKeyNotFound (36), ht:0x5a0c00, idxPtr:0x7f98f84bc8c0, key:0x7f98f78945d0, flags:0x0
2015-07-17 21:30:01.337: [GIPCXCPT][4152977168] gipcObjectLookupF [gipcPostF : gipc.c : 2022]: search found no matching oid 0000000000000000, ret gipcretKeyNotFound (36), ret gipcretInvalidObject (3)
2015-07-17 21:30:01.337: [GIPCXCPT][4152977168] gipcPostF [gipcdClientThread : gipcdClientThread.c : 3485]: EXCEPTION[ ret gipcretInvalidObject (3) ] failed to post obj 0000000000000000, flags 0x0
2015-07-17 21:30:01.337: [GIPCDCLT][4152977168] gipcdClientThread: Client thread has exited
2015-07-17 21:30:01.338: [ GIPCD][4253378304] gipcdMain: invalid status of gipcd Threads. Status of clientThread gipcdThreadStatusOffline Status of nodeThread gipcdThreadStatusInit Status of monitorThread gipcdThreadStatusReady
2015-07-17 21:30:01.338: [ GIPCD][4148774672] gipcdThreadWait: GIPCD received a shutdown msg from agent framework or client/node/monitor thread died
2015-07-17 21:30:01.338: [GIPCDMON][4148774672] gipcdMonitorThread: clientThread/nodeThread did not came into 'READY' state
2015-07-17 21:30:01.338: [ GIPCD][4148774672] gipcdSetThreadState: changing the status of monitorThread. current status gipcdThreadStatusReady desired status gipcdThreadStatusOffline
2015-07-17 21:30:01.338: [GIPCDMON][4148774672] gipcdMonitorThread: Monitor thread is exiting..
2015-07-17 21:30:01.356: [ CLSINET][4144572176] Returning NETDATA: 1 interfaces
2015-07-17 21:30:01.356: [ CLSINET][4144572176] # 0 Interface 'eth12',ip='172.xx.x.x',mac='f8-4a-bf-69-a9-0e',mask='255.xxx.xxx.0',net='172.xx.x.0',use='cluster_interconnect'
2015-07-17 21:30:01.356: [GIPCHGEN][4144572176] gipchaNodeAddInterface: adding interface information for inf 0x7f98f01089a0 { host '', haName 'gipcd_ha_name', local (nil), ip '172.xx.x.x', subnet '172.xx.x.0', mask '255.xxx.xxx.0', mac 'f8-4a-bf-69-a9-0e', ifname 'eth12', numRef 0, numFail 0, idxBoot 0, flags 0x1841 }
2015-07-17 21:30:01.356: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local interface for node 'racnode2', haName 'gipcd_ha_name', inf 'udp://172.xx.x.x:31179'
2015-07-17 21:30:01.356: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local bootstrap multicast interface for node 'racnode2', haName 'gipcd_ha_name', inf 'mcast://224.0.0.251:42424/172.xx.x.x'
2015-07-17 21:30:01.357: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local bootstrap multicast interface for node 'racnode2', haName 'gipcd_ha_name', inf 'mcast://230.0.1.0:42424/172.xx.x.x'
2015-07-17 21:30:01.357: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local bootstrap broadcast interface for node 'racnode2', haName 'gipcd_ha_name', inf 'udp://172.xx.x.xxx:42424'
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThread: gipcdNodeThread started
2015-07-17 21:30:01.357: [ GIPCD][4150875920] gipcdSetThreadState: changing the status of nodeThread. current status gipcdThreadStatusInit desired status gipcdThreadStatusReady
2015-07-17 21:30:01.357: [ GIPCD][4150875920] gipcdThreadWait: GIPCD received a shutdown msg from agent framework or client/node/monitor thread died
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThread: clientThread/monitorThread did not came into 'READY' state
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThreadShutdown: deleting all the peer connections
2015-07-17 21:30:01.357: [ GIPCD][4150875920] gipcdSetThreadState: changing the status of nodeThread. current status gipcdThreadStatusReady desired status gipcdThreadStatusOffline
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThread: Node thread has exited
2015-07-17 21:30:01.358: [ GIPCD][4253378304] gipcdMain: All threads terminated
2015-07-17 21:30:01.458: [ GIPCD][4253378304] gipcdMain: GIPCD terminated
2015-07-17 21:30:01.483: [ default][3990066944]
2015-07-17 21:30:01.337: [GIPCDCLT][4152977168] gipcdClientThread: gipcListen() failed (1) on endp 000000000000024c
2015-07-17 21:30:01.337: [ GIPCD][4152977168] gipcdSetThreadState: changing the status of clientThread. current status gipcdThreadStatusInit desired status gipcdThreadStatusOffline
2015-07-17 21:30:01.337: [ GIPCLIB][4152977168] gipclibMapSearch: gipcMapSearch() -> gipcMapGetNodeAddr() failed: ret:gipcretKeyNotFound (36), ht:0x5a0c00, idxPtr:0x7f98f84bc8c0, key:0x7f98f78945d0, flags:0x0
2015-07-17 21:30:01.337: [GIPCXCPT][4152977168] gipcObjectLookupF [gipcPostF : gipc.c : 2022]: search found no matching oid 0000000000000000, ret gipcretKeyNotFound (36), ret gipcretInvalidObject (3)
2015-07-17 21:30:01.337: [GIPCXCPT][4152977168] gipcPostF [gipcdClientThread : gipcdClientThread.c : 3485]: EXCEPTION[ ret gipcretInvalidObject (3) ] failed to post obj 0000000000000000, flags 0x0
2015-07-17 21:30:01.337: [GIPCDCLT][4152977168] gipcdClientThread: Client thread has exited
2015-07-17 21:30:01.338: [ GIPCD][4253378304] gipcdMain: invalid status of gipcd Threads. Status of clientThread gipcdThreadStatusOffline Status of nodeThread gipcdThreadStatusInit Status of monitorThread gipcdThreadStatusReady
2015-07-17 21:30:01.338: [ GIPCD][4148774672] gipcdThreadWait: GIPCD received a shutdown msg from agent framework or client/node/monitor thread died
2015-07-17 21:30:01.338: [GIPCDMON][4148774672] gipcdMonitorThread: clientThread/nodeThread did not came into 'READY' state
2015-07-17 21:30:01.338: [ GIPCD][4148774672] gipcdSetThreadState: changing the status of monitorThread. current status gipcdThreadStatusReady desired status gipcdThreadStatusOffline
2015-07-17 21:30:01.338: [GIPCDMON][4148774672] gipcdMonitorThread: Monitor thread is exiting..
2015-07-17 21:30:01.356: [ CLSINET][4144572176] Returning NETDATA: 1 interfaces
2015-07-17 21:30:01.356: [ CLSINET][4144572176] # 0 Interface 'eth12',ip='172.xx.x.x',mac='f8-4a-bf-69-a9-0e',mask='255.xxx.xxx.0',net='172.xx.x.0',use='cluster_interconnect'
2015-07-17 21:30:01.356: [GIPCHGEN][4144572176] gipchaNodeAddInterface: adding interface information for inf 0x7f98f01089a0 { host '', haName 'gipcd_ha_name', local (nil), ip '172.xx.x.x', subnet '172.xx.x.0', mask '255.xxx.xxx.0', mac 'f8-4a-bf-69-a9-0e', ifname 'eth12', numRef 0, numFail 0, idxBoot 0, flags 0x1841 }
2015-07-17 21:30:01.356: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local interface for node 'racnode2', haName 'gipcd_ha_name', inf 'udp://172.xx.x.x:31179'
2015-07-17 21:30:01.356: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local bootstrap multicast interface for node 'racnode2', haName 'gipcd_ha_name', inf 'mcast://224.0.0.251:42424/172.xx.x.x'
2015-07-17 21:30:01.357: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local bootstrap multicast interface for node 'racnode2', haName 'gipcd_ha_name', inf 'mcast://230.0.1.0:42424/172.xx.x.x'
2015-07-17 21:30:01.357: [GIPCHTHR][4146673424] gipchaWorkerCreateInterface: created local bootstrap broadcast interface for node 'racnode2', haName 'gipcd_ha_name', inf 'udp://172.xx.x.xxx:42424'
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThread: gipcdNodeThread started
2015-07-17 21:30:01.357: [ GIPCD][4150875920] gipcdSetThreadState: changing the status of nodeThread. current status gipcdThreadStatusInit desired status gipcdThreadStatusReady
2015-07-17 21:30:01.357: [ GIPCD][4150875920] gipcdThreadWait: GIPCD received a shutdown msg from agent framework or client/node/monitor thread died
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThread: clientThread/monitorThread did not came into 'READY' state
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThreadShutdown: deleting all the peer connections
2015-07-17 21:30:01.357: [ GIPCD][4150875920] gipcdSetThreadState: changing the status of nodeThread. current status gipcdThreadStatusReady desired status gipcdThreadStatusOffline
2015-07-17 21:30:01.357: [GIPCDNDE][4150875920] gipcdNodeThread: Node thread has exited
2015-07-17 21:30:01.358: [ GIPCD][4253378304] gipcdMain: All threads terminated
2015-07-17 21:30:01.458: [ GIPCD][4253378304] gipcdMain: GIPCD terminated
2015-07-17 21:30:01.483: [ default][3990066944]
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |