Exadata - ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] On Cell Storage Server (Doc ID 2579267.1)

Last updated on AUGUST 22, 2023

Applies to:

Symptoms

Cell node's alert history output shows many of following internal errors :

        62      2018-08-27T15:36:36-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"
        63      2018-09-10T16:33:21-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"
        64      2018-09-10T17:42:18-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"
        65      2018-09-10T17:44:36-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"
        66      2018-09-20T15:31:23-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"
        67      2018-09-20T16:11:06-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"
        68      2018-09-20T16:19:57-05:00       critical        "ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000] [] [] []"

More details in the incident alert 62 :

------------------------------------------

.....
ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000 on cell server xyzceladm01 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Critical: Incident Alert 62
Event Time 2018-08-27T15:36:36-05:00
Description ORA-07445: exception encountered: core dump [kdzcbuffer_basic()+136] [11] [0x000000000]
Affected Cell Name xyzceladm01
Server Model Oracle Corporation SUN SERVER X4-2L High Capacity
Chassis Serial Number 1443NM508L
Release Version 18.1.5.0.0.180506
RPM Version 18.1.5.0.0_LINUX.X64_180506-1
Cell offload version SYS_121240_180227

Recommended Action Errors in file /opt/oracle/cell/log/diag/asm/cell/SYS_121240_180227/trace/cellofltrc_22185_21.trc (incident=25).
Diagnostic package is attached. It is also accessible at https://xyzceladm01.local.com/diagpack/download?name=xyzceladm01_2018_08_27T15_36_36_62.tar.bz2
It will be retained on the storage server for 28 days, after which it may be automatically purged by MS during accelerated space reclamation.
Diagnostic packages for critical alerts can be downloaded and/or re-created at https://xyzceladm01.local.com/diagpack

In affected cell node's alert.log file, the offload server was restarted as per below shown :

.....
2018-08-27T15:36:36.703923-05:00
[RS] Terminating process pid 22185 with pkg cellofl-12.1.2.4.0_LINUX.X64_180227 path /opt/oracle/cell/cellofl-12.1.2.4.0_LINUX.X64_180506 gid=3 incid=4         
2018-08-27T15:36:36.704120-05:00
[RS] CELLOFLSRV crash detected for process (pid: 22185), group SYS_121240_180227, reason: 2           
2018-08-27T15:36:41.749417-05:00
Dumping diagnostics information, in file /opt/oracle/cell/log/diag/asm/cell/xyzceladm01/trace/svtrc_12405_10.trc, for smart IO session: '/box/predicate25308450' due to celloflsrv failure           
2018-08-27T15:36:42.627047-05:00
QuarantineMgr: adding imcQM for diskNum:1068604908, offset:124137766912 dbid:2613679030, cdbid:3972938940              
QuarantineMgr: Disk Region {DATA_CD_01_xyzceladm01[1068604908] 124137766912 1M} is quarantined, QuarantinePlanName=SYSTEM, QuarantineMode=FULL_Quarantine,  [reason=Crash ID=172 SQLID=87d03cka7k8g5 DBUName=CD
BTEST.PDBTEST
dbid=2613679030 cdbid=3972938940]
2018-08-27T15:36:46.760698-05:00
Offload group SYS_121240_180227 (id: 000000040003) recovery has been completed.          
2018-08-27T15:36:48.715290-05:00
[RS] Starting offload server with pid 30478 for group SYS_121240_180227, package cellofl-12.1.2.4.0_LINUX.X64_180227
2018-08-27T15:36:52.712985-05:00
[RS] Offload server with pid 30478 for group SYS_121240_180227, package cellofl-12.1.2.4.0_LINUX.X64_180227 successfully started
MS_ALERT OFFLOADGROUP_STATEFUL CLEAR SYS_121240_180227
.....

Changes

 N/A

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.