Flashcache missing, in status critical after multiple "Flash disk removed" alerts
Last updated on JULY 08, 2014
Applies to:Oracle Exadata Storage Server Software - Version 188.8.131.52.2 to 184.108.40.206.0 [Release 11.2]
Information in this document applies to any platform.
Versions affected: 220.127.116.11.0 to 18.104.22.168.0
One or more storage cells will generate multiple critical alerts of Flash disk removed events, which are also logged in the "cellcli -e list alerthistory", one for each of the a cell's 16 flash disks. Sample:
Size : 23G Serial Number : 1048M052XE Firmware : D20Y Slot Number : PCI
Slot: 5; FDOM: 2 Cell Disk : FD_14_dmorlx8cel01 Grid Disk : No grid disks exist
Flash Cache : Present Error Count : 0 Last Failure : unknown"
Running cellcli -e list flashcache detail on the affected cell will reveal a flashcache whose status is critical, with an effectiveCacheSize: 0 and 16 degradedCelldisks. Sample:
While the above symptoms could be caused by other issues, the presence of the following error message in /var/log/oracle/diag/asm/cell/`hostname -s`/trace/ms-odl.trc will uniquely identify the issue to be the one described here:
[ms.hwadapter.util.ExecCmdStream] [tid: 550] [ecid:
10.31.220.74:73419:1322591493192:6147,0] Error java.lang.Exception: Failed to
parse the command. Output of the command is not as expected while parsing line
00.0-sas-0x5080020000c88aa0:1 in the output of command type ls -l /dev/disk
/by-path/pci-0000*sas* | cut -d: -f3,4,7.
The storage cell's uptime has exceeded 180 days. This is the key precursor to the issue described in this document. Uptime can be obtained by running w or uptime commands.
Sign In with your My Oracle Support account
Don't have a My Oracle Support account? Click to get started
My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms