ProxMox VE 6 - replacing drive in zfs boot mirror

E

elsteve

Member
Sep 10, 2019
5
0
6
56
I've been following the information here:
https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks (Replacing a failed disk in the root pool)

but am getting jammed up on the grub-install /dev/sdx
I keep getting this: this GPT partition label contains no BIOS Boot Partition; embedding won't be possible
error: filesystem `zfs' doesn't support blocklists.

My rpool looks like this:

pool: rpool
state: ONLINE
scan: resilvered 126G in 0 days 00:31:12 with 0 errors on Tue Sep 10 12:26:20 2019
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-SPCC_Solid_State_Disk_E9DF075515C800418419 ONLINE 0 0 0
ata-SPCC_Solid_State_Disk_EB98075515F000418541 ONLINE 0 0 0

root@bedrock:/sbin# gdisk /dev/sdn
GPT fdisk (gdisk) version 1.0.3

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sdn: 468862128 sectors, 223.6 GiB
Model: SPCC Solid State
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 468862094
Partitions will be aligned on 2048-sector boundaries
Total free space is 3181 sectors (1.6 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 468844543 223.6 GiB BF01 zfs-3395a06313cc159a
9 468844544 468860927 8.0 MiB BF07

root@bedrock:/sbin# gdisk /dev/sdo
GPT fdisk (gdisk) version 1.0.3

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sdo: 468862128 sectors, 223.6 GiB
Model: SPCC Solid State
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 468862094
Partitions will be aligned on 2048-sector boundaries
Total free space is 3181 sectors (1.6 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 468844543 223.6 GiB BF01 zfs-3fb9effbf83b8152
9 468844544 468860927 8.0 MiB BF07



* Note that both drives are fine, they are just cheap-o's and I want to replace them with something better * (and faster)

Both drives are attached to motherboard SATA ports as /dev/sdn and /dev/sdo - My understanding is that the motherboard boots off the drive in the lowest port, in this case /dev/sdn (all other HDD's are on an HBA if that matters)

All my efforts have been replacing the /dev/sdn drive (but it shouldn't matter, right?) so confused... ;(
 
Last edited:
how did you install that system? it's not how the PVE installer sets it up, and looks like there is some extra boot device on another disk?
 
Right off the proxmox install iso - booted from flash drive. Only other devices are 12 HDD which are all in a zfs pool...
Based on your question, I'm tempted to backup the config/VM's & reinstall from scratch on the new SSD's (after downloading a fresh copy of the iso)
 
are you sure you are looking at the right disks?
 
Positive.
root@bedrock:/RAIDZ3# smartctl -i /dev/sdn ; fdisk -l /dev/sdn ;gdisk /dev/sdn
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-5.0.21-1-pve] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model: SPCC Solid State Disk
Serial Number: E9DF075515C800418419
Firmware Version: S8FM08.2
User Capacity: 240,057,409,536 bytes [240 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ATA8-ACS (minor revision not indicated)
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Thu Sep 12 11:56:42 2019 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Disk /dev/sdn: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Disk model: SPCC Solid State
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 98286682-CB8D-D747-A2E0-CD1C6B24AC09

Device Start End Sectors Size Type
/dev/sdn1 2048 468844543 468842496 223.6G Solaris /usr & Apple ZFS
/dev/sdn9 468844544 468860927 16384 8M Solaris reserved 1
GPT fdisk (gdisk) version 1.0.3

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sdn: 468862128 sectors, 223.6 GiB
Model: SPCC Solid State
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 98286682-CB8D-D747-A2E0-CD1C6B24AC09
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 468862094
Partitions will be aligned on 2048-sector boundaries
Total free space is 3181 sectors (1.6 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 468844543 223.6 GiB BF01 zfs-3395a06313cc159a
9 468844544 468860927 8.0 MiB BF07


NAME STATE READ WRITE CKSUM
rpool DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
ata-SPCC_Solid_State_Disk_E9DF075515C800418419 ONLINE 0 0 0
 
Just for kicks I re-installed to two other SSD's - partition table look like below: Is that what it should be? So strange the way the other drives were set up...

Number Start (sector) End (sector) Size Code Name
1 34 2047 1007.0 KiB EF02
2 2048 1050623 512.0 MiB EF00
3 1050624 488397134 232.4 GiB BF01
 
In short:
Yes, that's normal, because the 512M will be used for EFI boot as you can see.
Other zfs disks with only 2 partitions "just" part of a pool, there are no reservation for any boot.

Bash: 
Disklabel type: gpt
Disk identifier: BDB522F2-A808-4C1F-8B7E-83E1CABD0D43

Device       Start       End   Sectors   Size Type
/dev/sdc1       34      2047      2014  1007K BIOS boot
/dev/sdc2     2048   1050623   1048576   512M EFI System
/dev/sdc3  1050624 937703054 936652431 446.6G Solaris /usr & Apple ZFS
----
Disklabel type: gpt
Disk identifier: 3F845005-7DD0-4DCC-895C-A56EA8CF1986

Device       Start       End   Sectors   Size Type
/dev/sdd1       34      2047      2014  1007K BIOS boot
/dev/sdd2     2048   1050623   1048576   512M EFI System
/dev/sdd3  1050624 937703054 936652431 446.6G Solaris /usr & Apple ZFS
 
Thanks for the info - think I'll just stay with my newly formatted SSD's instead of trying to troubleshoot the old ones.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom