Solaris - Oracle Database Performance Problem Due To Veritas odm_io_lock mutex lock contention (Doc ID 2109077.1)

Last updated on OCTOBER 03, 2022

Applies to:

Symptoms

Customer experience an IO bottleneck in the server and a slow write performance to database redo disks.

- Storage vendor EMC and Oracle database investigated the issue and did not find any problems in that layers.

- No problem found on fc hbas, no FC errors reported by luxadm -e rdls or fcinfo

- No fmdump events while the problem was present

- No errors on messages neither when the problem was present

Configuration is :

This Solaris 11.1 M6-32 server has a huge configuration, there are 26 Oracle Emulex 16GB FC HBA Dual port, accessing to EMC storage arrays,
more that 3000 LUNs mapped from the EMC storage arrays to this server, VxFS 6.1.1.0 and ODM 6.1.0 are installed.

Symantec needs to investigate how to reduce the contention on the odm_io_lock mutex lock.

Additionally vx_sched_lk spin lock is being observed in the lockstat outputs,
which indicates that multiple worker threads were trying to acquire this same odm_io_lock global lock, adding to contention and CPU usage.

Cause

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.