[SOLVED] Partition vermisst nach RAID-Z2 rebuild

r4dh4l · Feb 14, 2018

Naja, /dev/sda hatte ich ja ganz zu Anfang mit dd "leicht" überschrieben, aber unabängig davon:

Code:

# zpool online rpool /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4ND0PVXEA-part2
cannot online /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4ND0PVXEA-part2: no such device in pool

wolfgang · Feb 14, 2018

Ich habe grade gelernt da die Informationen auch noch am ende der Partitionen gespeichert werden.

Code:

dd if=/dev/zero of=/dev/sdd bs=1M count=512
dd bs=512 if=/dev/zero of=/dev/sdd count=40000 seek=$((`blockdev --getsz /dev/sdd` - 40000))
sgdisk --replicate=/dev/sdd /dev/sdc
sgdisk --randomize-guids /dev/sdd
grub-install /dev/sdd
zpool replace rpool /dev/sda /dev/sdd2

r4dh4l · Feb 14, 2018

Hm, leider wieder am Ende der gleiche "Fehler":

Code:

# dd if=/dev/zero of=/dev/sdd bs=1M count=512
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 4.29517 s, 125 MB/s

# dd bs=512 if=/dev/zero of=/dev/sdd count=40000 seek=$((`blockdev --getsz /dev/sdd` - 40000))
40000+0 records in
40000+0 records out
20480000 bytes (20 MB, 20 MiB) copied, 0.754198 s, 27.2 MB/s

# sgdisk --replicate=/dev/sdd /dev/sdc
The operation has completed successfully.

# sgdisk --randomize-guids /dev/sdd
The operation has completed successfully.

# grub-install /dev/sdd
Installing for i386-pc platform.
Installation finished. No error reported.

# zpool replace rpool /dev/sda /dev/sdd2
cannot replace /dev/sda with /dev/sdd2: no such device in pool

wolfgang · Feb 14, 2018

Versuch es mal mit sda2
Wenn es nicht geht bitte wieder denn ganzen ou

Code:

zpool status
zdb
blkid

r4dh4l · Feb 14, 2018

Okay, nur um sicherzugestehen, weil ich aktuell nicht ganz sicher bin, ob die Umsetzung des Wiki-Beispiels nun in meinem Fall "sda" oder "sda2" betreffen sollte:

Code:

dd if=/dev/zero of=/dev/sda bs=1M count=512
dd bs=512 if=/dev/zero of=/dev/sda count=40000 seek=$((`blockdev --getsz /dev/sdd` - 40000))
sgdisk --replicate=/dev/sda /dev/sdc
sgdisk --randomize-guids /dev/sda
grub-install /dev/sda
zpool replace rpool /dev/sda2 /dev/sdd2

..."go"?

wolfgang · Feb 15, 2018

Es reicht wenn du nur

r4dh4l said:
zpool replace rpool /dev/sda2 /dev/sdd2

machst da die Platte eh noch leer sein sollte.

r4dh4l · Feb 15, 2018

Teilerfolg:

Code:

# zpool replace rpool /dev/sda2 /dev/sdd2
Make sure to wait until resilver is done before rebooting.

Aber:

Code:

# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu Feb 15 08:40:46 2018
        13.8G scanned out of 7.81T at 70.5M/s, 32h11m to go
        2.79G resilvered, 0.17% done
config:

        NAME             STATE     READ WRITE CKSUM
        rpool            DEGRADED     0     0     0
          raidz2-0       DEGRADED     0     0     0
            replacing-0  DEGRADED     0     0     0
              sda        OFFLINE      0     0     0
              sdd2       ONLINE       0     0     0  (resilvering)
            sda2         FAULTED      0     0     0  corrupted data
            sdb2         ONLINE       0     0     0
            sdc2         ONLINE       0     0     0

errors: No known data errors

...wieso wird jetzt sda ersetzt, obwohl ich doch sda2 angegeben habe zu ersetzen?

wolfgang · Feb 15, 2018

r4dh4l said:
...wieso wird jetzt sda ersetzt, obwohl ich doch sda2 angegeben habe zu ersetzen?

Weiß ich auch nicht, habe sowas noch nie gesehen.
Das Problem ist das die Platen cross referenziert sind, also genau verdreht sind.

Wenn das resilvering fertig ist bitte mal ne komplette Übersicht schicken.

r4dh4l · Feb 16, 2018

Spannend. sdd2 ist jetzt zumindest wieder "da":

Code:

# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: resilvered 1.31T in 28h29m with 0 errors on Fri Feb 16 13:10:11 2018
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       DEGRADED     0     0     0
          raidz2-0  DEGRADED     0     0     0
            sdd2    ONLINE       0     0     0
            sda2    FAULTED      0     0     0  corrupted data
            sdb2    ONLINE       0     0     0
            sdc2    ONLINE       0     0     0

errors: No known data errors


# zdb | less
rpool:
    version: 5000
    name: 'rpool'
    state: 0
    txg: 496201
    pool_guid: 9973961061107860467
    errata: 0
    hostid: 656640
    hostname: 'proxmox-server'
    com.delphix:has_per_vdev_zaps
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 9973961061107860467
        children[0]:
            type: 'raidz'
            id: 0
            guid: 10834227445461462896
            nparity: 2
            metaslab_array: 128
            metaslab_shift: 36
            ashift: 12
            asize: 12002313371648
            is_log: 0
            create_txg: 4
            com.delphix:vdev_zap_top: 66
            children[0]:
                type: 'disk'
                id: 0
                guid: 11560873476689254094
                path: '/dev/sdd2'
                whole_disk: 0
                DTL: 161
                create_txg: 4
                com.delphix:vdev_zap_leaf: 67
            children[1]:
                type: 'disk'
                id: 1
                guid: 1020653270431596650
                path: '/dev/sda2'
                whole_disk: 0
                DTL: 79
                create_txg: 4
                com.delphix:vdev_zap_leaf: 68
            children[2]:
                type: 'disk'
                id: 2
                guid: 1321917959934574704
                path: '/dev/sdb2'
                whole_disk: 0
                DTL: 78
                create_txg: 4
                com.delphix:vdev_zap_leaf: 69
            children[3]:
                type: 'disk'
                id: 3
                guid: 14728949233396094063
                path: '/dev/sdc2'
                whole_disk: 0
                DTL: 31
                create_txg: 4
                com.delphix:vdev_zap_leaf: 70
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
(END)

# blkid
/dev/sda2: LABEL="rpool" UUID="9973961061107860467" UUID_SUB="3746872896469901947" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="efe9bd9d-1c2e-4049-9025-477e48b7706b"
/dev/sdb2: LABEL="rpool" UUID="9973961061107860467" UUID_SUB="1321917959934574704" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="7736d425-ea52-495e-b1f9-049675c77374"
/dev/sdc2: LABEL="rpool" UUID="9973961061107860467" UUID_SUB="14728949233396094063" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="13c960a5-b62a-412e-810c-d6075fffe28a"
/dev/zd0: UUID="e5da7fd2-666c-47fd-bb92-d6d240fd3765" TYPE="swap"
/dev/zd32p1: UUID="46e5a07d-b4b6-45f6-8017-063774842f42" TYPE="ext2" PARTUUID="ea8a27ed-01"
/dev/zd32p5: UUID="jSddmO-1Z7l-GGxb-aETI-3jPg-rTU1-UBhGtw" TYPE="LVM2_member" PARTUUID="ea8a27ed-05"
/dev/zd80: UUID="f5255bd3-60ef-4889-b6b4-91170b652755" TYPE="crypto_LUKS"
/dev/zd128p1: UUID="46e5a07d-b4b6-45f6-8017-063774842f42" TYPE="ext2" PARTUUID="ea8a27ed-01"
/dev/zd128p5: UUID="jSddmO-1Z7l-GGxb-aETI-3jPg-rTU1-UBhGtw" TYPE="LVM2_member" PARTUUID="ea8a27ed-05"
/dev/sda1: PARTUUID="74de145e-5369-4539-be44-9e8c33ae3b2b"
/dev/sda9: PARTUUID="892388f6-7102-4d88-8270-128d291a2d25"
/dev/sdb1: PARTUUID="2ddebef9-19fb-4c36-b18f-1ae8e4039685"
/dev/sdb9: PARTUUID="4e9be946-f966-4739-a284-0b3afef4bfe3"
/dev/sdc1: PARTUUID="f0fabc35-29e8-43fc-b446-af41f7f16186"
/dev/sdc9: PARTUUID="f6693a61-d245-4e42-86ff-68441f4796de"
/dev/zd32: PTUUID="ea8a27ed" PTTYPE="dos"
/dev/zd128: PTUUID="ea8a27ed" PTTYPE="dos"
/dev/sdd1: PARTUUID="a00f5062-08ab-42d1-bd84-39e87562c29b"
/dev/sdd2: LABEL="rpool" UUID="9973961061107860467" UUID_SUB="11560873476689254094" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="ad293612-809a-4003-a556-d5bbc0685f84"
/dev/sdd9: PARTUUID="96656295-0d21-41ab-a081-730f88fbb328"

Wenn

Code:

zpool replace rpool /dev/sda2 /dev/sdd2

nun doch tatsächlich sdd2 auf "Position 1" wiederhergestellt hat (wo sda2 eigentlich liegen sollte/mal gelegen hat), sda2 jetzt aber als "faulted" auf "Position 2" aufgeführt wird, wäre jetzt die spannende Frage, was sich hinter sda2 verbirgt... oder jetzt einfach mal ein...

Code:

zpool replace rpool /dev/sda2

...probieren?

PS: Falles es noch hilft:

Code:

# zdb -l /dev/sda2 | less

failed to unpack label 0
failed to unpack label 1
------------------------------------
LABEL 2
------------------------------------
    version: 5000
    name: 'rpool'
    state: 0
    txg: 404450
    pool_guid: 9973961061107860467
    errata: 0
    hostid: 656640
    hostname: 'proxmox-server'
    top_guid: 10834227445461462896
    guid: 3746872896469901947
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 10834227445461462896
        nparity: 2
        metaslab_array: 128
        metaslab_shift: 36
        ashift: 12
        asize: 12002313371648
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 3746872896469901947
            path: '/dev/sda1'
            devid: 'ata-WDC_WD30EFRX-68EUZN0_WD-WCC4ND0PVXEA-part1'
            phys_path: 'pci-0000:01:00.0-ata-1'
            whole_disk: 1
            DTL: 28
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 1020653270431596650
            path: '/dev/sdb2'
            whole_disk: 0
            DTL: 79
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 1321917959934574704
            path: '/dev/sdc2'
            whole_disk: 0
            DTL: 78
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 14728949233396094063
            path: '/dev/sdd2'
            whole_disk: 0
            DTL: 31
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
    labels = 2
failed to unpack label 3

r4dh4l · Feb 24, 2018

Ich habe den Freitagabend mal genutzt, den Proxmox-Server neu zu starten und siehe da, mit diesem Output konnte ich etwas anfangen:

Code:

# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: resilvered 1.31T in 28h29m with 0 errors on Fri Feb 16 13:10:11 2018
config:

        NAME                     STATE     READ WRITE CKSUM
        rpool                    DEGRADED     0     0     0
          raidz2-0               DEGRADED     0     0     0
            sdb2                 ONLINE       0     0     0
            1020653270431596650  FAULTED      0     0     0  was /dev/sda2
            sdc2                 ONLINE       0     0     0
            sdd2                 ONLINE       0     0     0

# zdb | less
rpool:
    version: 5000
    name: 'rpool'
    state: 0
    txg: 496201
    pool_guid: 9973961061107860467
    errata: 0
    hostid: 656640
    hostname: 'proxmox-server'
    com.delphix:has_per_vdev_zaps
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 9973961061107860467
        children[0]:
            type: 'raidz'
            id: 0
            guid: 10834227445461462896
            nparity: 2
            metaslab_array: 128
            metaslab_shift: 36
            ashift: 12
            asize: 12002313371648
            is_log: 0
            create_txg: 4
            com.delphix:vdev_zap_top: 66
            children[0]:
                type: 'disk'
                id: 0
                guid: 11560873476689254094
                path: '/dev/sdd2'
                whole_disk: 0
                DTL: 161
                create_txg: 4
                com.delphix:vdev_zap_leaf: 67
            children[1]:
                type: 'disk'
                id: 1
                guid: 1020653270431596650
                path: '/dev/sda2'
                whole_disk: 0
                DTL: 79
                create_txg: 4
                com.delphix:vdev_zap_leaf: 68
            children[2]:
                type: 'disk'
                id: 2
                guid: 1321917959934574704
                path: '/dev/sdb2'
                whole_disk: 0
                DTL: 78
                create_txg: 4
                com.delphix:vdev_zap_leaf: 69
            children[3]:
                type: 'disk'
                id: 3
                guid: 14728949233396094063
                path: '/dev/sdc2'
                whole_disk: 0
                DTL: 31
                create_txg: 4
                com.delphix:vdev_zap_leaf: 70
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data

# lsblk
NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda         8:0    0  2.7T  0 disk
├─sda1      8:1    0 1007K  0 part
├─sda2      8:2    0  2.7T  0 part
└─sda9      8:9    0    8M  0 part
sdb         8:16   0  2.7T  0 disk
├─sdb1      8:17   0 1007K  0 part
├─sdb2      8:18   0  2.7T  0 part
└─sdb9      8:25   0    8M  0 part
sdc         8:32   0  2.7T  0 disk
├─sdc1      8:33   0 1007K  0 part
├─sdc2      8:34   0  2.7T  0 part
└─sdc9      8:41   0    8M  0 part
sdd         8:48   0  2.7T  0 disk
├─sdd1      8:49   0 1007K  0 part
├─sdd2      8:50   0  2.7T  0 part
└─sdd9      8:57   0    8M  0 part
zd0       230:0    0    7G  0 disk [SWAP]
zd16      230:16   0  4.5G  0 disk
zd32      230:32   0    8G  0 disk
├─zd32p1  230:33   0  243M  0 part
├─zd32p2  230:34   0    1K  0 part
└─zd32p5  230:37   0  7.8G  0 part
zd48      230:48   0  4.5G  0 disk
zd64      230:64   0  4.5G  0 disk
zd80      230:80   0  3.5T  0 disk
zd96      230:96   0  4.5G  0 disk
zd112     230:112  0  4.5G  0 disk
zd128     230:128  0    8G  0 disk
├─zd128p1 230:129  0  243M  0 part
├─zd128p2 230:130  0    1K  0 part
└─zd128p5 230:133  0  7.8G  0 part

Entsprechend der Anleitung:

Code:

# sgdisk --replicate=/dev/sda /dev/sdb
The operation has completed successfully.
# sgdisk --randomize-guids /dev/sda
The operation has completed successfully.
# grub-install /dev/sda
Installing for i386-pc platform.
Installation finished. No error reported.
# zpool replace rpool 1020653270431596650 /dev/sda2
Make sure to wait until resilver is done before rebooting.
# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Feb 24 00:07:29 2018
        23.8G scanned out of 7.83T at 54.2M/s, 41h57m to go
        4.87G resilvered, 0.30% done
config:

        NAME                       STATE     READ WRITE CKSUM
        rpool                      DEGRADED     0     0     0
          raidz2-0                 DEGRADED     0     0     0
            sdb2                   ONLINE       0     0     0
            replacing-1            DEGRADED     0     0     0
              1020653270431596650  OFFLINE      0     0     0  was /dev/sda2/old
              sda2                 ONLINE       0     0     0  (resilvering)
            sdc2                   ONLINE       0     0     0
            sdd2                   ONLINE       0     0     0

errors: No known data errors

...das sieht doch ganz gut aus. Die "Reihenfolge" ist immernoch etwas durcheinander, aber ansonsten scheinen bald wieder alle 4 Platten mit allen Partitionen wiederhergestellt zu sein und damit kann ich gut leben.
Danke wolfgang/proxmox für die großartige Unterstützung. Hoffe, ich werde sie nicht allzu bald wieder in Anspruch nehmen müssen.

Search

Search

[SOLVED] Partition vermisst nach RAID-Z2 rebuild

r4dh4l

Active Member

wolfgang

Proxmox Retired Staff

r4dh4l

Active Member

wolfgang

Proxmox Retired Staff

r4dh4l

Active Member

wolfgang

Proxmox Retired Staff

r4dh4l

Active Member

wolfgang

Proxmox Retired Staff

r4dh4l

Active Member

r4dh4l

Active Member