Last updated on FEBRUARY 07, 2014
Applies to:
Oracle Database - Enterprise Edition - Version 10.1.0.2 to 12.1.0.1 [Release 10.1 to 12.1]Information in this document applies to any platform.
Symptoms
Database alert log reports errors such as the following:
=========================================
Sun Dec 29 09:07:11 2013
Dumping diagnostic data in directory=[cdmp_20131229090710], requested by (instance=1, osid=8486 (IF01)), summary=[incident=1335033].
Sun Dec 29 09:07:12 2013
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x2B0258054B30] [PC:0x4750C51, ksfq_del()+93] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/rdbms/dwhrbp/DWHRBP2/trace/DWHRBP2_i901_14201.trc (incident=1526781):
ORA-07445: exception encountered: core dump [ksfq_del()+93] [SIGSEGV] [ADDR:0x2B0258054B30] [PC:0x4750C51] [Address not mapped to object] []
Incident details in: /u01/app/oracle/diag/rdbms/dwhrbp/DWHRBP2/incident/incdir_1526781/DWHRBP2_i901_14201_i1526781.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Dumping diagnostic data in directory=[cdmp_20131229090713], requested by (instance=2, osid=14201 (I901)), summary=[incident=1526781].
Sun Dec 29 09:07:16 2013
Dumping diagnostic data in directory=[cdmp_20131229090710], requested by (instance=1, osid=8486 (IF01)), summary=[incident=1335033].
Sun Dec 29 09:07:12 2013
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x2B0258054B30] [PC:0x4750C51, ksfq_del()+93] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/rdbms/dwhrbp/DWHRBP2/trace/DWHRBP2_i901_14201.trc (incident=1526781):
ORA-07445: exception encountered: core dump [ksfq_del()+93] [SIGSEGV] [ADDR:0x2B0258054B30] [PC:0x4750C51] [Address not mapped to object] []
Incident details in: /u01/app/oracle/diag/rdbms/dwhrbp/DWHRBP2/incident/incdir_1526781/DWHRBP2_i901_14201_i1526781.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Dumping diagnostic data in directory=[cdmp_20131229090713], requested by (instance=2, osid=14201 (I901)), summary=[incident=1526781].
Sun Dec 29 09:07:16 2013
The trace file reports the messages similar to the following:
==========================================
ORA-07445: exception encountered: core dump [ksfq_del()+93] [SIGSEGV] [ADDR:0x2B0258054B30] [PC:0x4750C51] [Address not mapped to object] []
========= Dump for incident 1526781 (ORA 7445 [ksfq_del()+93]) ========
----- Beginning of Customized Incident Dump(s) -----
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x2B0258054B30] [PC:0x4750C51, ksfq_del()+93] [flags: 0x0, count: 1]
Registers:
%rax: 0x0000000000000000 %rbx: 0x00000007ca25c888 %rcx: 0x00000007d5dff9e2
%rdx: 0x0000000000000000 %rdi: 0x00000007ca25c888 %rsi: 0x00002b0258054b28
%rsp: 0x00007fff86886de0 %rbp: 0x00007fff86886e00 %r8: 0x00000000ffffffff
%r9: 0x0004eea7d084d3e6 %r10: 0x00000007b9f33f80 %r11: 0x0000000000000001
%r12: 0x00002b0258054b28 %r13: 0x00000007d5e00548 %r14: 0x00000007d5dff9e2
%r15: 0x0000000000000001 %rip: 0x0000000004750c51 %efl: 0x0000000000010246
ksfq_del()+77 (0x4750c41) jnz 0x4750cf7
ksfq_del()+83 (0x4750c47) xor %eax,%eax
ksfq_del()+85 (0x4750c49) test $0x2,%al
ksfq_del()+87 (0x4750c4b) jnz 0x4750d4d
> ksfq_del()+93 (0x4750c51) mov 0x8(%r12),%rax
ksfq_del()+98 (0x4750c56) test %rax,%rax
ksfq_del()+101 (0x4750c59) jz 0x4750d06
ksfq_del()+107 (0x4750c5f) mov $0x92cfce8,%edi
ksfq_del()+112 (0x4750c64) mov $0x1,%esi
dbkedDefDump(): Starting a non-incident diagnostic dump (flags=0x3, level=3, mask=0x0)
----- SQL Statement (None) -----
Current SQL information unavailable - no cursor.
========= Dump for incident 1526781 (ORA 7445 [ksfq_del()+93]) ========
----- Beginning of Customized Incident Dump(s) -----
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x2B0258054B30] [PC:0x4750C51, ksfq_del()+93] [flags: 0x0, count: 1]
Registers:
%rax: 0x0000000000000000 %rbx: 0x00000007ca25c888 %rcx: 0x00000007d5dff9e2
%rdx: 0x0000000000000000 %rdi: 0x00000007ca25c888 %rsi: 0x00002b0258054b28
%rsp: 0x00007fff86886de0 %rbp: 0x00007fff86886e00 %r8: 0x00000000ffffffff
%r9: 0x0004eea7d084d3e6 %r10: 0x00000007b9f33f80 %r11: 0x0000000000000001
%r12: 0x00002b0258054b28 %r13: 0x00000007d5e00548 %r14: 0x00000007d5dff9e2
%r15: 0x0000000000000001 %rip: 0x0000000004750c51 %efl: 0x0000000000010246
ksfq_del()+77 (0x4750c41) jnz 0x4750cf7
ksfq_del()+83 (0x4750c47) xor %eax,%eax
ksfq_del()+85 (0x4750c49) test $0x2,%al
ksfq_del()+87 (0x4750c4b) jnz 0x4750d4d
> ksfq_del()+93 (0x4750c51) mov 0x8(%r12),%rax
ksfq_del()+98 (0x4750c56) test %rax,%rax
ksfq_del()+101 (0x4750c59) jz 0x4750d06
ksfq_del()+107 (0x4750c5f) mov $0x92cfce8,%edi
ksfq_del()+112 (0x4750c64) mov $0x1,%esi
dbkedDefDump(): Starting a non-incident diagnostic dump (flags=0x3, level=3, mask=0x0)
----- SQL Statement (None) -----
Current SQL information unavailable - no cursor.
Some or all of the following function codes may be found in the 'Call Stack' section of the trace file:
ksfq_del <- ksfqsubmit <- ksfvrdp <- opirip <- opidrv <- sou2o <- opimai_real <- ssthrdmain
The Wait Stack indicates RMAN processes:
==============================
Current Wait Stack:
0: waiting for 'RMAN Tape slave I/O'
tape operation=0x6c, operation flags=0x0, timeout=0xffffffff
wait_id=2425 seq_num=3121 snap_id=1
wait times: snap=0.458402 sec, exc=0.458402 sec, total=0.458402 sec
wait times: max=infinite, heur=0.458402 sec
wait counts: calls=0 os=0
in_wait=1 iflags=0x5a0
Wait State:
fixed_waits=0 flags=0x22 boundary=(nil)/-1
Session Wait History:
elapsed time of 0.000006 sec since current wait
0: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2424 seq_num=3120 snap_id=1
wait times: snap=1.000994 sec, exc=1.000994 sec, total=1.000994 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000014 sec of elapsed time
1: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2423 seq_num=3119 snap_id=1
wait times: snap=1.001005 sec, exc=1.001005 sec, total=1.001005 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000005 sec of elapsed time
2: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2422 seq_num=3118 snap_id=1
wait times: snap=1.001934 sec, exc=1.001934 sec, total=1.001934 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000011 sec of elapsed time
3: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2421 seq_num=3117 snap_id=1
wait times: snap=1.001004 sec, exc=1.001004 sec, total=1.001004 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000007 sec of elapsed time
4: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2420 seq_num=3116 snap_id=1
wait times: snap=1.001013 sec, exc=1.001013 sec, total=1.001013 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000013 sec of elapsed time
5: waited for 'RMAN Tape slave I/O'
tape operation=0x6c, operation flags=0x0, timeout=0xffffffff
wait_id=2418 seq_num=3115 snap_id=2
wait times: snap=0.000003 sec, exc=0.000007 sec, total=0.063029 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000000 sec of elapsed time
6: waited for 'Backup: MML delete backup piece'
=0x0, =0x0, =0x0
wait_id=2419 seq_num=3114 snap_id=1
wait times: snap=0.063022 sec, exc=0.063022 sec, total=0.063022 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000000 sec of elapsed time
7: waited for 'RMAN Tape slave I/O'
tape operation=0x6c, operation flags=0x0, timeout=0xffffffff
wait_id=2418 seq_num=3113 snap_id=1
wait times: snap=0.000004 sec, exc=0.000004 sec, total=0.000004 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000013 sec of elapsed time
8: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2417 seq_num=3112 snap_id=1
wait times: snap=1.000922 sec, exc=1.000922 sec, total=1.000922 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000009 sec of elapsed time
9: waited for 'RMAN Tape slave I/O'
tape operation=0x72, operation flags=0x0, timeout=0xffffffff
wait_id=2416 seq_num=3111 snap_id=1
wait times: snap=0.000004 sec, exc=0.000004 sec, total=0.000004 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000010 sec of elapsed time
sample interval: 1 sec, max history 120 sec
---------------------------------------------------
[10 samples, 09:07:02 - 09:07:12]
idle wait at each sample
[1 sample, 09:07:01]
waited for 'Backup: MML write backup piece', seq_num: 2450
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.003810 sec (sample interval: 0 sec)
[1 sample, 09:07:00]
idle wait at each sample
[1 sample, 09:06:59]
waited for 'Backup: MML write backup piece', seq_num: 1986
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.003915 sec (sample interval: 0 sec)
[1 sample, 09:06:58]
waited for 'Backup: MML write backup piece', seq_num: 1687
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.004743 sec (sample interval: 0 sec)
[2 samples, 09:06:56 - 09:06:57]
idle wait at each sample
[1 sample, 09:06:55]
waited for 'Backup: MML write backup piece', seq_num: 808
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.004257 sec (sample interval: 0 sec)
[1 sample, 09:06:54]
waited for 'Backup: MML write backup piece', seq_num: 505
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.007716 sec (sample interval: 0 sec)
[1 sample, 09:06:53]
waited for 'Backup: MML write backup piece', seq_num: 482
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 1.060908 sec (sample interval: 0 sec)
[1 sample, 09:06:52]
waited for 'Backup: MML write backup piece', seq_num: 247
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.003911 sec (sample interval: 0 sec)
[1 sample, 09:06:51]
idle wait at each sample
[1 sample, 09:06:50]
waited for 'Backup: MML write backup piece', seq_num: 113
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 1.048607 sec (sample interval: 0 sec)
[49 samples, 09:06:01 - 09:06:49]
idle wait at each sample
[2 samples, 09:05:59 - 09:06:00]
waited for 'Backup: MML create a backup piece', seq_num: 47
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 1.702252 sec (sample interval: 1 sec)
[31 samples, 09:05:28 - 09:05:58]
idle wait at each sample
[session created at: 09:05:28]
---------------------------------------------------
Sampled Session History Summary:
longest_non_idle_wait: 'Backup: MML create a backup piece'
[2 samples, 09:05:59 - 09:06:00]
time_waited: 1.702252 sec (sample interval: 1 sec)
---------------------------------------------------
0: waiting for 'RMAN Tape slave I/O'
tape operation=0x6c, operation flags=0x0, timeout=0xffffffff
wait_id=2425 seq_num=3121 snap_id=1
wait times: snap=0.458402 sec, exc=0.458402 sec, total=0.458402 sec
wait times: max=infinite, heur=0.458402 sec
wait counts: calls=0 os=0
in_wait=1 iflags=0x5a0
Wait State:
fixed_waits=0 flags=0x22 boundary=(nil)/-1
Session Wait History:
elapsed time of 0.000006 sec since current wait
0: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2424 seq_num=3120 snap_id=1
wait times: snap=1.000994 sec, exc=1.000994 sec, total=1.000994 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000014 sec of elapsed time
1: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2423 seq_num=3119 snap_id=1
wait times: snap=1.001005 sec, exc=1.001005 sec, total=1.001005 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000005 sec of elapsed time
2: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2422 seq_num=3118 snap_id=1
wait times: snap=1.001934 sec, exc=1.001934 sec, total=1.001934 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000011 sec of elapsed time
3: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2421 seq_num=3117 snap_id=1
wait times: snap=1.001004 sec, exc=1.001004 sec, total=1.001004 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000007 sec of elapsed time
4: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2420 seq_num=3116 snap_id=1
wait times: snap=1.001013 sec, exc=1.001013 sec, total=1.001013 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000013 sec of elapsed time
5: waited for 'RMAN Tape slave I/O'
tape operation=0x6c, operation flags=0x0, timeout=0xffffffff
wait_id=2418 seq_num=3115 snap_id=2
wait times: snap=0.000003 sec, exc=0.000007 sec, total=0.063029 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000000 sec of elapsed time
6: waited for 'Backup: MML delete backup piece'
=0x0, =0x0, =0x0
wait_id=2419 seq_num=3114 snap_id=1
wait times: snap=0.063022 sec, exc=0.063022 sec, total=0.063022 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000000 sec of elapsed time
7: waited for 'RMAN Tape slave I/O'
tape operation=0x6c, operation flags=0x0, timeout=0xffffffff
wait_id=2418 seq_num=3113 snap_id=1
wait times: snap=0.000004 sec, exc=0.000004 sec, total=0.000004 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000013 sec of elapsed time
8: waited for 'i/o slave wait'
msg ptr=0x7ca25c838, =0x0, =0x0
wait_id=2417 seq_num=3112 snap_id=1
wait times: snap=1.000922 sec, exc=1.000922 sec, total=1.000922 sec
wait times: max=1.000000 sec
wait counts: calls=1 os=1
occurred after 0.000009 sec of elapsed time
9: waited for 'RMAN Tape slave I/O'
tape operation=0x72, operation flags=0x0, timeout=0xffffffff
wait_id=2416 seq_num=3111 snap_id=1
wait times: snap=0.000004 sec, exc=0.000004 sec, total=0.000004 sec
wait times: max=infinite
wait counts: calls=0 os=0
occurred after 0.000010 sec of elapsed time
sample interval: 1 sec, max history 120 sec
---------------------------------------------------
[10 samples, 09:07:02 - 09:07:12]
idle wait at each sample
[1 sample, 09:07:01]
waited for 'Backup: MML write backup piece', seq_num: 2450
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.003810 sec (sample interval: 0 sec)
[1 sample, 09:07:00]
idle wait at each sample
[1 sample, 09:06:59]
waited for 'Backup: MML write backup piece', seq_num: 1986
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.003915 sec (sample interval: 0 sec)
[1 sample, 09:06:58]
waited for 'Backup: MML write backup piece', seq_num: 1687
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.004743 sec (sample interval: 0 sec)
[2 samples, 09:06:56 - 09:06:57]
idle wait at each sample
[1 sample, 09:06:55]
waited for 'Backup: MML write backup piece', seq_num: 808
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.004257 sec (sample interval: 0 sec)
[1 sample, 09:06:54]
waited for 'Backup: MML write backup piece', seq_num: 505
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.007716 sec (sample interval: 0 sec)
[1 sample, 09:06:53]
waited for 'Backup: MML write backup piece', seq_num: 482
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 1.060908 sec (sample interval: 0 sec)
[1 sample, 09:06:52]
waited for 'Backup: MML write backup piece', seq_num: 247
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 0.003911 sec (sample interval: 0 sec)
[1 sample, 09:06:51]
idle wait at each sample
[1 sample, 09:06:50]
waited for 'Backup: MML write backup piece', seq_num: 113
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 1.048607 sec (sample interval: 0 sec)
[49 samples, 09:06:01 - 09:06:49]
idle wait at each sample
[2 samples, 09:05:59 - 09:06:00]
waited for 'Backup: MML create a backup piece', seq_num: 47
p1: ''=0x0
p2: ''=0x0
p3: ''=0x0
time_waited: 1.702252 sec (sample interval: 1 sec)
[31 samples, 09:05:28 - 09:05:58]
idle wait at each sample
[session created at: 09:05:28]
---------------------------------------------------
Sampled Session History Summary:
longest_non_idle_wait: 'Backup: MML create a backup piece'
[2 samples, 09:05:59 - 09:06:00]
time_waited: 1.702252 sec (sample interval: 1 sec)
---------------------------------------------------
Cause
Sign In with your My Oracle Support account |
|
Don't have a My Oracle Support account? Click to get started |
My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms