Exadata Storage Server Crashed With RS-7445 Error (Doc ID 1500288.1)

Last updated on JANUARY 30, 2013

Applies to:

Oracle Exadata Storage Server Software - Version 11.1.0.3.0 to 11.2.3.1.1 [Release 11.1 to 11.2]
Information in this document applies to any platform.

Symptoms

We can see the ORA-07445 error followed by cellsrv crash in a Exadata Storage Server alert.log.


Sun Sep 23 20:12:27 2012
Smart scan resource message - informational. Disk: 922589
Non critical error DIA-48913 caught while writing to trace file "/opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/log/diag/asm/cell/zzipcel01/trace/svtrc_8984_59.trc"
Error message: DIA-48913: Writing into trace file failed, file size limit [43962814] reached
Writing to the above trace file is disabled for now on...
Sun Sep 23 20:18:18 2012
[RS] Process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt (pid: 8980) received exception [signal num: 14] [ADDR:0x0]
Sun Sep 23 20:20:04 2012
[RS] Monitoring process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt (pid: 8980) timed out while trying a 64K/1MB ioctl to Cellsrv. Will retry without killing Cellsrv
[RS] Started monitoring process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt with pid 21568
Sun Sep 23 20:21:43 2012
[RS] Process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt (pid: 21568) received exception [signal num: 14] [ADDR:0x0]
Sun Sep 23 20:21:43 2012
[RS] Monitoring process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt (pid: 21568) timed out while trying a 64K/1MB ioctl to Cellsrv. Will retry without killing Cellsrv
[RS] Started monitoring process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt with pid 21966
Sun Sep 23 20:22:05 2012
[RS] Process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt (pid: 21966) received exception [signal num: 14] [ADDR:0x0]
Sun Sep 23 20:22:05 2012
[RS] monitoring process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt (pid: 0) returned with error: 123
Sun Sep 23 20:22:05 2012
State dump signal delivered to Cellsrv<8984>
State dump signal delivered to Cellsrv<8984> by RS.
Sun Sep 23 20:22:10 2012
State dump interrupted for Cellsrv<8984> by RS.  It did not complete in 5 seconds.
Sun Sep 23 20:22:14 2012
Sun Sep 23 20:22:11 2012
State dump completed for Cellsrv<8984>
Sun Sep 23 20:22:16 2012
[RS] Stopped Service CELLSRV
Errors in file /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/log/diag/asm/cell/zzipcel01/trace/rstrc_8970_4.trc  (incident=1):
RS-7445 [Serv CELLSRV hang detected] [It will be restarted] [] [] [] [] [] [] [] [] [] []
Incident details in: /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/log/diag/asm/cell/zzipcel01/incident/incdir_1/rstrc_8970_4_i1.trc
Sweep [inc][1]: completed
[RS] Started monitoring process /opt/oracle/cell11.2.2.4.2_LINUX.X64_111221/cellsrv/bin/cellrsomt with pid 22375
Sun Sep 23 20:22:16 2012

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms