My Oracle Support Banner

ZFS Pool Recovery: "zpool status" reports "One or more devices could not be used because the label is missing or invalid." (Doc ID 1516050.1)

Last updated on JUNE 06, 2018

Applies to:

Solaris Operating System - Version 10 3/05 and later
Information in this document applies to any platform.

Symptoms

There are two commands which will show this scenario.  The first is 'zpool status' which shows a status of "One or more devices could not be used because the label is missing or invalid", eg:

   pool: t90-z
state: FAULTED
status: One or more devices could not be used because the label is missing
       or invalid.  There are insufficient replicas for the pool to continue
       functioning.
action: Destroy and re-create the pool from
       a backup source.
  see: http://www.sun.com/msg/ZFS-8000-5E
scrub: none requested
config:

       NAME                 STATE     READ WRITE CKSUM
       t90-z                FAULTED      0     0     0  corrupted data
         emc_clariion1_369  UNAVAIL      0     0     0  corrupted data

The second place to look is in FMA using "fmdump -e" which reports 'vdev.bad_label'

--- fmdump-e.out ----
Jan 01 10:22:32.2346 ereport.fs.zfs.vdev.bad_label  
Jan 01 10:22:32.2346 ereport.fs.zfs.zpool        
--- end ---

 Use 'fmdump -eV' to look at the detail for the events to confirm the pool and disk.

Jan 01 2013 10:22:32.234692250 ereport.fs.zfs.vdev.bad_label
nvlist version: 0
        class = ereport.fs.zfs.vdev.bad_label
        ena = 0x34ed9c8f5c13401
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = zfs
                pool = 0x66f03ea70241a896
                vdev = 0x8d9059fad30bfe8
        (end detector)

        pool = t90-z
        pool_guid = 0x66f03ea70241a896
        pool_context = 1
        pool_failmode = wait
        vdev_guid = 0x8d9059fad30bfe8
        vdev_type = disk
        vdev_path = /dev/dsk/emc_clariion1_369s0
        parent_guid = 0x66f03ea70241a896
        parent_type = root
        prev_state = 0x7
        __ttl = 0x1
        __tod = 0x50e30d48 0xdfd1e9a

 

Cause

To view full details, sign in with your My Oracle Support account.

Don't have a My Oracle Support account? Click to get started!


In this Document
Symptoms
Cause
Solution
 Step #1 - Data Collection
 Step #2 - Review the data looking for errors
 Step #3 - Validating the VTOC Type and Geometry
 Step #4 - Understanding the on-disk VDEV Label format
 Step #5 - Look at the on-disk corruption within the vdev labels
 Step 5.1 - Confirm all 4 labels are identical
 Step 5.2 - Look at the raw labels using zdb (1M) and xxd(1M)
 Step 5.3 - Importing the pool
 Step #6 - Identifying the pool belonging to the rogue Uberblocks
  Step #7 - How did this happen?
References

My Oracle Support provides customers with access to over a million knowledge articles and a vibrant support community of peers and Oracle experts.