Hitachi Data Systems (HDS) SVPmon shows IO Bottleneck Issues Which They Claim is Caused by Server HBA Limitation
(Doc ID 2567112.1)
Last updated on OCTOBER 05, 2022
Applies to:
Solaris Operating System - Version 11.1 to 11.4 [Release 11.0]Information in this document applies to any platform.
Symptoms
HDS requested the customer to investigate IO bottleneck issue.
HDS wrote:
First of all, there is no hardware issue or performance bottleneck found on internal array which might be related.
Instead, with the SVPmon data in the dump, we could see clearly what is causing the damage..
With SVPmon data in 1-minute interval, the read transfer rate on all the reported ports (5678AB) were sustained at 400MB/s as a flat line as highlighted below.
Usually, such situation is taken as the indication of the bottleneck on the data transfer in read direction, most likely to be related with the HBA port limitation.
In total, it is up to 3.5GB/s in read direction to the reported server. Such high read workload would have chance to trigger the HBA port saturation.
As a result, the read latency was badly impacted on those ports which was perfectly in line with the excessive read workload, especially on port 7A/8A!
The write latency was impacted similarly, highest latency found on ports 7A/8A as well. Both read and write IOs being impacted are also one of the typical symptom once HBA ports reaching its physical limitation in read direction.
What is more, with SVPmon data in 1-hour interval, we could see that similar workload was actually occurring on daily basis over last month
As a result, the high latency issue occurred every day and it is always ports 7A/8A experiencing the highest latency<85> The alignment between the excessive read workload and the higher latency is conclusive.
Server HBA to HDS port mapping:
Switch fabric 0:
HBA Controller/PWWN HDS Port HDS PWWN
2100000E1EC8C1D0 5A 50060E8007E3DC40
2100000E1EC8C230 7A 50060E8007E3DC60
2100000E1EC8C220 5B 50060E8007E3DC41
2100000E1EC8C0F0 7B 50060E8007E3DC61
Switch Fabric 1:
HBA Controller PWWN HDS Port HDS PWWN
2100000E1EC8C1D1 6A 50060E8007E3DC50
2100000E1EC8C231 8A 50060E8007E3DC70
2100000E1EC8C221 6B 50060E8007E3DC51
2100000E1EC8C0F1 8B 50060E8007E3DC71
Changes
Cause
To view full details, sign in with your My Oracle Support account. |
|
Don't have a My Oracle Support account? Click to get started! |
In this Document
Symptoms |
Changes |
Cause |
Solution |