Startup Hangs at Mount Stage after AIO is Enabled on Linux (Doc ID 1367724.1)

Last updated on SEPTEMBER 02, 2016

Applies to:

Oracle Database - Enterprise Edition - Version 10.2.0.1 and later
Linux x86
Linux x86-64
***Checked for relevance on 02-Sep-2016***

Symptoms

1. After enabling AIO using the steps below, when attempting to start up the database, it hangs at mount stage.

1) Set instance startup parameters:

SQL> alter system set disk_asynch_io=TRUE scope=spfile;
SQL> alter system set filesystemio_options=asynch scope=spfile;


2) On the OS, set the parameter

/proc/sys/fs/aio-max-nr 1048576


3) Startup the database



2. From the system state dump when the startup is hanging, the process is hanging at 'control file parallel write':

PROCESS 15:
----------------------------------------
SO: 0x90e6f098, type: 2, owner: (nil), flag: INIT/-/-/0x00
(process) Oracle pid=15, calls cur/top: 0x90fb6218/0x90fb6218, flag: (0) -
int error: 0, call error: 0, sess error: 0, txn error 0
(post info) last post received: 0 0 25
last post received-location: ksasnr
last process to post me: 90e6b0d8 1 6
last post sent: 0 0 24
last post sent-location: ksasnd
last process posted by me: 90e6b0d8 1 6
(latch info) wait_event=0 bits=0
Process Group: DEFAULT, pseudo proc: 0x90eb3790
O/S info: user: oracle, term: pts/3, ospid: 17387
OSD pid info: Unix process pid: 17387, image: oracle@www.dbxen.com (TNS V1-V3)
Short stack dump:
ksdxfstk()+32<-ksdxcb()+1573<-sspuser()+111<-__restore_rt()+0<-io_submit()+7
<-skgfr_lio_listio64()+212<-skgfqio()+2929<-ksfdgo()+1488<-ksfdwtio()+1576
<-ksfdbio()+6048<-kccwbp()+1660<-kccchb()+5269<-kccocx()+2972<-kccocf()+362
<-kcfcmb()+882<-kcfmdb()+71<-adbdrv()+66087<-opiexe()+13745<-opiosq0()+3398
<-kpooprx()+318<-kpoal8()+783<-opiodr()+1184<-ttcpip()+1226<-opitsk()+1310
<-opiino()+1024<-opiodr()+1184<-opidrv()+548<-sou2o()+114<-opimai_real()+163
<-main()+116<-__libc_start_main()+244<-_start()+41

SO: 0x90f90478, type: 4, owner: 0x90e6f098, flag: INIT/-/-/0x00
(session) sid: 159 trans: 0x8f727660, creator: 0x90e6f098, flag: (41) USR/- BSY/-/-/-/-/-
DID: 0000-000F-00000008, short-term DID: 0000-0000-00000000
txn branch: (nil)
oct: 35, prv: 0, sql: 0x90ae37f0, psql: 0x90ae5680, user: 0/SYS
service name: SYS$USERS
O/S info: user: oracle, term: pts/3, ospid: 17359, machine: www.dbxen.com
program: sqlplus@orcl (TNS V1-V3)
application name: sqlplus@orcl (TNS V1-V3), hash value=1929608025
waiting for 'control file parallel write' wait_time=0, seconds since wait started=3744
files=1, block#=1, requests=1
blocking sess=0x(nil) seq=23
Dumping Session Wait History
for 'control file sequential read' count=1 wait_time=0.000023 sec
file#=0, block#=1, blocks=1
for 'control file sequential read' count=1 wait_time=0.000040 sec
file#=2, block#=3, blocks=8



3. From the strace output of the process that runs the 'startup' command, the process hangs waiting for OS call 'read'.

20515 read(0, "alter database mount;\n", 4096) = 22
20515 write(9, "\1\r\0\0\6\0\0\0\0\0\21i\010067\37\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0"..., 269) = 269
20515 read(10, <unfinished ...>



4. The database files (control file, redologs, data files etc.) are placed on a tmpfs file system mounted at /dev/shm/

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms