[OCI] Oracle Datafile went offline / recover mode with following error message ORA-15025/ORA-27041/ORA-15081 in alert log file
(Doc ID 2559930.1)
Last updated on DECEMBER 01, 2020
Applies to:
Oracle Cloud Infrastructure - Database Service - Version N/A to N/A [Release 1.0]Information in this document applies to any platform.
Symptoms
The oracle datafile went offline / corruption reported in database alert log file
The database alert log file will have below error message
2019-01-25T20:32:23.211048-08:00
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_41163.trc:
ORA-15025: could not open disk "/dev/DATADISK1"
ORA-27041: unable to open file
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
2019-01-25T20:32:23.211178-08:00
WARNING: Read Failed. group:1 disk:2 AU:645 offset:0 size:4096
path:Unknown disk
incarnation:0xf0f0dc3e synchronous result:'I/O error'
subsys:Unknown library krq:0x7f954eb308b0 bufp:0x7f954eb23000 osderr1:0x434c5344 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
WARNING: failed to read mirror side 1 of virtual extent 0 logical extent 0 of file 292 in group [1.3781176515] from disk DATA_0002 allocation unit 645 reason error; if possible, will try another mirror side
2019-01-25T20:32:23.215297-08:00
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_41163.trc:
ORA-15081: failed to submit an I/O operation to a disk
2019-01-29T03:51:53.393580-08:00
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_20886.trc:
...
ORA-15080: synchronous I/O operation failed to read block 0 of disk 2 in disk group DATA
The /var/log/messages iscsi related error message
Jan 25 20:13:52 iddb1 kernel: [1598946.592254] connection3:0: detected conn error (1020)
Jan 25 20:13:52 iddb1 iscsid: Kernel reported iSCSI connection 3:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Jan 25 20:13:55 iddb1 iscsid: connection3:0 is operational after recovery (1 attempts)
Jan 25 20:20:56 iddb1 dhclient[2622]: DHCPREQUEST on eth0 to 169.254.169.254 port 67 (xid=0x61be28b3)
Jan 25 20:20:56 iddb1 dhclient[2622]: DHCPACK from 169.254.169.254 (xid=0x61be28b3)
Jan 25 20:20:58 iddb1 dhclient[2622]: bound to 192.168.0.70 -- renewal in 40031 seconds.
Jan 25 20:23:00 iddb1 kernel: [1599494.513867] connection4:0: detected conn error (1020)
Jan 25 20:23:00 iddb1 iscsid: Kernel reported iSCSI connection 4:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Jan 25 20:23:03 iddb1 iscsid: connection4:0 is operational after recovery (1 attempts)
Jan 25 20:32:17 iddb1 kernel: [1600052.151026] connection9:0: detected conn error (1020)
Jan 25 20:32:18 iddb1 iscsid: Kernel reported iSCSI connection 9:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Jan 25 20:32:20 iddb1 iscsid: connection9:0 is operational after recovery (1 attempts)
Jan 25 20:33:09 iddb1 kernel: [1600103.572194] connection7:0: detected conn error (1020)
Jan 25 20:33:09 iddb1 iscsid: Kernel reported iSCSI connection 7:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Jan 25 20:33:12 iddb1 iscsid: connection7:0 is operational after recovery (1 attempts)
Jan 25 20:35:10 iddb1 kernel: [1600224.964700] connection1:0: detected conn error (1020)
Jan 25 20:35:11 iddb1 iscsid: Kernel reported iSCSI connection 1:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (3)
Jan 25 20:35:13 iddb1 iscsid: connection1:0 is operational after recovery (1 attempts)
Jan 25 20:47:11 iddb1 kernel: [1600946.321690] connection4:0: detected conn error (1020)
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
| Symptoms |
| Cause |
| Solution |
| References |