My Oracle Support Banner

DBaaS: High CPU usage by gipcd.bin process (Doc ID 2348610.1)

Last updated on APRIL 09, 2020

Applies to:

Oracle Database Cloud Service - Version N/A and later
Information in this document applies to any platform.

Symptoms

"gipcd.bin" process found to be consuming high CPU in both the RAC nodes. The system was having the below merge patch already installed:

BUG 25817230 - MERGE REQUEST ON TOP OF OCW PSU 12.1.0.2.170418 FOR BUGS 21210985 21692274

 

The pstack collected for the problematic gipcd.bin process is below:

pstack_output.txt
=================
Thread 7 (Thread 0x7f9f2f297700 (LWP 16169)):
#0  0x00007f9f301b9383 in poll () from /lib64/libc.so.6
#1  0x00007f9f36cd6a60 in sgipcwWaitHelper () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#2  0x00007f9f36cd3d91 in sgipcwWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#3  0x00007f9f36a302bd in gipcWaitOsd () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#4  0x00007f9f36a16508 in gipcInternalWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#5  0x00007f9f369a7541 in gipcWaitF () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#6  0x00007f9f367a6776 in clsdms_thdmai () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#7  0x00007f9f32052aa1 in start_thread () from /lib64/libpthread.so.0
#8  0x00007f9f301c2bcd in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7f9f2dc94700 (LWP 16259)):
#0  0x00007f9f301b9383 in poll () from /lib64/libc.so.6
#1  0x00007f9f36cd6a60 in sgipcwWaitHelper () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#2  0x00007f9f36cd3d91 in sgipcwWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#3  0x00007f9f36a302bd in gipcWaitOsd () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#4  0x00007f9f36a16508 in gipcInternalWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#5  0x00007f9f369a7541 in gipcWaitF () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#6  0x000000000047d9d1 in gipcdClientThread ()
#7  0x00007f9f32052aa1 in start_thread () from /lib64/libpthread.so.0
#8  0x00007f9f301c2bcd in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7f9f2da93700 (LWP 16260)):
#0  0x00007f9f301b9383 in poll () from /lib64/libc.so.6
#1  0x00007f9f36cd6a60 in sgipcwWaitHelper () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#2  0x00007f9f36cd3d91 in sgipcwWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#3  0x00007f9f36a302bd in gipcWaitOsd () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#4  0x00007f9f36a16508 in gipcInternalWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#5  0x00007f9f369a7541 in gipcWaitF () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#6  0x000000000044de4e in gipcdNodeThread ()
#7  0x00007f9f32052aa1 in start_thread () from /lib64/libpthread.so.0
#8  0x00007f9f301c2bcd in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7f9f2d892700 (LWP 16261)):
#0  0x00007f9f301b9383 in poll () from /lib64/libc.so.6
#1  0x00007f9f36cd6a60 in sgipcwWaitHelper () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#2  0x00007f9f36cd3d91 in sgipcwWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#3  0x00007f9f36a302bd in gipcWaitOsd () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#4  0x00007f9f36a16508 in gipcInternalWait () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#5  0x00007f9f369a7541 in gipcWaitF () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#6  0x00000000004906a6 in gipcdMonitorThread ()
#7  0x00007f9f32052aa1 in start_thread () from /lib64/libpthread.so.0
#8  0x00007f9f301c2bcd in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7f9f2d691700 (LWP 16266)):
#0  0x00007f9f3018671a in times () from /lib64/libc.so.6
#1  0x00007f9f32b86a06 in sltrgatime64 () from /u01/app/12.1.0.2/grid/lib/libclntshcore.so.12.1
#2  0x00007f9f36ae917c in gipchaWorkerWork () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#3  0x00007f9f36aebec9 in gipchaWorkerThread () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#4  0x00007f9f36acef4d in gipchaWorkerThreadEntry () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#5  0x00007f9f32052aa1 in start_thread () from /lib64/libpthread.so.0
#6  0x00007f9f301c2bcd in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f9f2d490700 (LWP 16267)):
#0  0x00007f9f301b9383 in poll () from /lib64/libc.so.6
#1  0x00007f9f2ce7c086 in __libc_res_nsend () from /lib64/libresolv.so.2
#2  0x00007f9f2ce78811 in __libc_res_nquery () from /lib64/libresolv.so.2
#3  0x00007f9f2ce78dd0 in __libc_res_nquerydomain () from /lib64/libresolv.so.2
#4  0x00007f9f2ce798f8 in __libc_res_nsearch () from /lib64/libresolv.so.2
#5  0x00007f9f2d08b7d7 in _nss_dns_gethostbyname4_r () from /lib64/libnss_dns.so.2
#6  0x00007f9f301a9f7b in gaih_inet () from /lib64/libc.so.6
#7  0x00007f9f301acfbf in getaddrinfo () from /lib64/libc.so.6
#8  0x00007f9f369221b6 in sclsinetaiInitialize () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#9  0x00007f9f368c1124 in clsinet_GetOSDNodeData () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#10 0x00007f9f368bf4b7 in clsinet_GetNodeData () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#11 0x00007f9f36a7a477 in gipchaDaemonGetPeerNodeData () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#12 0x00007f9f36a7c36b in gipchaDaemonProcessClientLocMon () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#13 0x00007f9f36a834bf in gipchaDaemonProcessClientReq () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#14 0x00007f9f36a895f3 in gipchaDaemonCheckClient () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#15 0x00007f9f36ac8fbf in gipchaDaemonWork () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#16 0x00007f9f36ace9a5 in gipchaDaemonThread () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#17 0x00007f9f36a6d155 in gipchaDaemonThreadEntry () from /u01/app/12.1.0.2/grid/lib/libhasgen12.so
#18 0x00007f9f32052aa1 in start_thread () from /lib64/libpthread.so.0
#19 0x00007f9f301c2bcd in clone () from /lib64/libc.so.6

Changes

 No change was done in the system.

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Changes
Cause
Solution
References

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.