SPARC T5-4/T5-8 persistently generates the following event PCIEX-8000-V2 even after replacing the HBA ( Pallene-Q) card (Doc ID 1568920.1)

Last updated on JULY 26, 2017

Applies to:

SPARC T5-8 - Version All Versions to All Versions [Release All Releases]
SPARC T5-4 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

The SPARC T5-4/T5-8 server has generated an FMA event with error code PCIEX-8000-V2, the event itself does not panic the system but a noticeable performance degradation is observered from a Pallene-Q PCIE card.

The output of Solaris command "prtdiag -v"  shows that the Pallene-Q HBA is running at 4 lanes

/SYS/RCSA/PCIE10  PCIE  SUNW,qlc-pciex1077,2532           QLE2562      2.5GTx4 
/pci@480/pci@1/pci@0/pci@4/SUNW,qlc@0
/SYS/RCSA/PCIE10  PCIE  SUNW,qlc-pciex1077,2532           QLE2562      2.5GTx4                       
 /pci@480/pci@1/pci@0/pci@4/SUNW,qlc@0,1

 We expect the output of the Solaris command "prtidag-v" for the Pallene-Q HBA to be running at 8 lanes

/SYS/RCSA/PCIE10  PCIE  SUNW,qlc-pciex1077,2532           QLE2562      2.5GTx8 
/pci@480/pci@1/pci@0/pci@4/SUNW,qlc@0
/SYS/RCSA/PCIE10  PCIE  SUNW,qlc-pciex1077,2532           QLE2562      2.5GTx8                      
/pci@480/pci@1/pci@0/pci@4/SUNW,qlc@0,1

The ILOM fault management shell ( faultmgmt ) was flagging the HBA card on the PCIE Carrier Assembly as faulty.

 
 -> show faulty
         Target                                 | Property                                         | Value
 ---------------------------------------+-----------------------------------------+---------------------------------------------------
/SP/faultmgmt/0                            | fru                                                 |
/SYS/MB /SP/faultmgmt/0/faults/0  | class                                              | fault.io.pciex.bus-linkbw-down
/SP/faultmgmt/0/faults/0                | component                                     | /SYS/RCSA/PCIE10/CAR/CARD
/SP/faultmgmt/0/faults/0                | uuid                                               | c18248fa-2cc6-e9e5-ac44-fbb59b81ec6f
/SP/faultmgmt/0/faults/0                | timestamp                                      | 2013-06-21/02:32:49
/SP/faultmgmt/0/faults/0                | system_component_serial_number   | AK00XXXXXX
/SP/faultmgmt/0/faults/0                | system_component_part_number     | 31806934+1+1
/SP/faultmgmt/0/faults/0                | system_component_name               | SPARC T5-8
/SP/faultmgmt/0/faults/0                | system_component_manufacturer    | Oracle Corporation
/SP/faultmgmt/0/faults/0                | chassis_serial_number                    | AK00XXXXXX
/SP/faultmgmt/0/faults/0                | chassis_part_number                      | 31806934+1+1
/SP/faultmgmt/0/faults/0                | chassis_name                                 | SPARC T5-8
/SP/faultmgmt/0/faults/0                | chassis_manufacturer                     | Oracle Corporation
/SP/faultmgmt/0/faults/0                | system_serial_number                    | AK00XXXXXX
/SP/faultmgmt/0/faults/0                | system_part_number                      | 31806934+1+1
/SP/faultmgmt/0/faults/0                | system_name                                | SPARC T5-8
/SP/faultmgmt/0/faults/0                | system_manufacturer                     | Oracle Corporation
/SP/faultmgmt/0/faults/0                | fru_name                                      | ASSY,MB,MM,T5-4,T5-8
/SP/faultmgmt/0/faults/0                | fru_manufacturer                           | Oracle Corporation
/SP/faultmgmt/0/faults/0                | fru_serial_number                          | 465769T+13195203F2
/SP/faultmgmt/0/faults/0                | fru_rev_level                                 | 03
/SP/faultmgmt/0/faults/0                | fru_part_number                           | 7070931
/SP/faultmgmt/0/faults/0                | mod-version                                  | 1.16
/SP/faultmgmt/0/faults/0                | mod-name                                    | eft
/SP/faultmgmt/0/faults/0                | severity                                        | Major
 
Solaris FMA ( fmadm faulty ) has flagged the PCIE card and the Motherboard (/SYS/MB)
 
[
root@aur7703s:~# fmadm faulty
--------------- ------------------------------------  -------------- ---------
TIME            EVENT-ID                              MSG-ID         SEVERITY
--------------- ------------------------------------  -------------- ---------
Jun 21 02:32:49 c18248fa-2cc6-e9e5-ac44-fbb59b81ec6f  PCIEX-8000-V2  Major

Problem Status    : solved
Diag Engine       : eft / 1.16
System
    Manufacturer  : Oracle-Corporation
    Name          : SPARC-T5-8
    Part_Number   : 31806934+1+1
    Serial_Number : AK00XXXXXXX
    Host_ID       : 8635XXXX

----------------------------------------
Suspect 1 of 3 :
   Fault class : fault.io.pciex.bus-linkbw-down
   Certainty   : 33%
   Affects     : dev:////pci@480/pci@1/pci@0/pci@4/SUNW,qlc@0,1
   Status      : faulted but still in service

   FRU
     Location         : "PCIE10"
     Manufacturer     : unknown
     Name             : unknown
     Part_Number      : unknown
     Revision         : unknown
     Serial_Number    : unknown
     Chassis
        Manufacturer  : Oracle Corporation
        Name          : SPARC T5-8
        Part_Number   : 31806934+1+1
        Serial_Number : AK00XXXXXXX
        Status        : faulty
----------------------------------------
Suspect 2 of 3 :
   Fault class : fault.io.pciex.bus-linkbw-down
   Certainty   : 33%
   Affects     : dev:////pci@480/pci@1/pci@0/pci@4
   Status      : faulted but still in service

   FRU
     Location         : "/SYS/MB"
     Manufacturer     : unknown
     Name             : unknown
     Part_Number      : 7070931
     Revision         : 03
     Serial_Number    : 465769T+13195203F2
     Chassis
        Manufacturer  : Oracle Corporation
        Name          : SPARC T5-8
        Part_Number   : 31806934+1+1
        Serial_Number : AK00XXXXXXX
        Status        : faulty
----------------------------------------
Suspect 3 of 3 :
   Fault class : fault.io.pciex.bus-linkbw-down
   Certainty   : 33%
   Affects     : dev:////pci@480/pci@1/pci@0/pci@4/SUNW,qlc@0
   Status      : faulted but still in service

   FRU
     Location         : "PCIE10"
     Manufacturer     : unknown
     Name             : unknown
     Part_Number      : unknown
     Revision         : unknown
     Serial_Number    : unknown
     Chassis
        Manufacturer  : Oracle Corporation
        Name          : SPARC T5-8
        Part_Number   : 31806934+1+1
        Serial_Number : AK00XXXXXXX
        Status        : faulty

Description : A decrease in PCIe link bandwidth has been detected.

Response    : None.

Impact      : Potential performance degradation for the devices associated with
              this fault.

Action      : Use 'fmadm faulty' to provide a more detailed view of this event.
              Please refer to the associated reference document at
              http://support.oracle.com/msg/PCIEX-8000-V2 for the latest
              service procedures and policies regarding this diagnosis.
 

 

 

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms