kmalloc-96 leak causing memory leak at X7 cells after upgrade to 21.2.0.0.0EXARU
(Doc ID 2792688.1)
Last updated on FEBRUARY 14, 2022
Applies to:
Oracle Exadata Storage Server Software - Version 21.1.0.0.0 and laterInformation in this document applies to any platform.
Symptoms
After upgrading X7 cells to 21.2.0.0.0EXARU,within 9 days offload servers restarting due to memory leak on kernel
level slab .It was taking place as there was a leak taking place on kmalloc-96.
Event Time 2021-06-24T08:00:01-05:00
Description ORA-700 [Offload issue job timed out] [Group startup failed due to too many restarts for the group]
[AAAAAA] [384707] [xxxx.cccc.xxx] [OSS_IOCTL_SETIORMPLAN] [iorm_kuty]
and
2021-06-24T14:00:09.158266-05:00
[RS] Starting offload server with pid 23621 for group SYS_212000_210429, package cellofl-21.2.0.0.0_LINUX.X64_210429.1
2021-06-24T14:00:09.406421-05:00
Memory consumption of offload server is high. Terminating all offload servers
Current memory consumption in MB: CELLSRV: 117497, OFLSRV: 3 KERNEL: 81715, cell max memory limit: 188813 MB
2021-06-24T14:00:10.467817-05:00
Memory consumption of offload server is high. Terminating all offload servers
Current memory consumption in MB: CELLSRV: 117497, OFLSRV: 47 KERNEL: 81753, cell max memory limit: 188813 MB
2021-06-24T14:00:12.030633-05:00
[RS] Restarting CELLSRV, Running out of Swap Free: 1548796 Swap Total: 2097148 Swap base value: 1791740
Errors in file /opt/oracle/cell/log/diag/asm/cell/xxxx/trace/rstrc_35859_omt.trc (incident=849):
RS-7445 [No more free swap space] [CELLSRV will be restarted] [] [] [] [] [] [] [] [] [] []
Incident details in: /opt/oracle/cell/log/diag/asm/cell/xxxx/incident/incdir_849/rstrc_35859_omt_i849.trc
Active / Total Caches (% used) : 99 / 129 (76.7%)
Active / Total Size (% used) : 79788714.05K / 79810466.12K (100.0%)
Minimum / Average / Maximum Object : 0.01K / 0.09K / 16.56K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
845338998 845338247 99% 0.09K 20127119 42 80508476K kmalloc-96
^^^^^^^^^^^^^^^^^^^^^
362817 298568 82% 0.10K 9303 39 37212K buffer_head
173344 173344 100% 0.25K 5417 32 43336K rds_ib_inco
zzz <06/24/2021 10:00:51> Count:16
TIMESTAMP %OS_AVAIL OS_AVAIL =( +OS_TOT -OS_USR -OS_KNL =(
+SLAB +RDS +RDMA +PGST +OTHER ) ) %CL_AVAIL CL_AVAIL =( +CL_MAX =(
OS_TOT -CL_RVD ) -CL_USD =( +C +O +K ) )
2021-06-24T10:00:52-05:00 .80 1.5 =( 187.2 107.5 78.2 =(
73.5 2.6 0.0 0 2.1 ) ) -6.56 -11.9 =( 181.3 =(
187.2 5.9 ) 193.2 =( 114.7 0.3 78.2 ) )
zzz <06/24/2021 10:01:52> Count:17
zzz <06/24/2021 17:12:28> Count:22
TIMESTAMP %OS_AVAIL OS_AVAIL =( +OS_TOT -OS_USR -OS_KNL =(
+SLAB +RDS +RDMA +PGST +OTHER ) ) %CL_AVAIL CL_AVAIL =( +CL_MAX =(
OS_TOT -CL_RVD ) -CL_USD =( +C +O +K ) )
2021-06-24T17:12:28-05:00 3.51 6.5 =( 187.2 99.4 81.3 =(
76.8 2.6 0.0 0 1.9 ) ) 0 0 =( 0 =(
187.2 0 ) 0 =( 0 0 0 ) )
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |
| References |