How to Change MTU Size in Exadata Environment (Doc ID 1586212.1)

Last updated on NOVEMBER 09, 2016

Applies to:

Exadata X3-8 Hardware - Version All Versions and later
Oracle Exadata Hardware - Version 11.2.0.2 and later
Oracle Exadata Storage Server Software - Version 11.2.2.4.2 to 11.2.3.2.1 [Release 11.2]
Exadata Database Machine X2-2 Hardware - Version All Versions and later
Exadata Database Machine V2 - Version All Versions and later
Linux x86-64

Goal

Having a larger value for MTU such as 65520 bytes = 64kb will require more memory pages to support IPoIB, the large frame size is derived from HCA IB Card with driver support at the o/s level. In some cases it may be required to reduce the size of the frame to avoid memory page fragmentation issues.

Value of 65520 bytes would require 18 contiguous memory pages based on a default kernel pagesize.


IP over IB similar to TCP on Ethernet does contain additional packet overhead.

IPoIB Packet Frame consists of -->

 | Infiniband Header | EoIB Encapsulation Header | IPoIB Header |  IP DATA |



Lets assume the kernel by default will use a pagesize of 4k, you can verify this value -->



For more in-depth analysis slabinfo can be used.

Solution

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms