Need Guidance on Replacing SSDs in ZFS Pool with Limited Disk Slots

ana_mera

New Member
Jun 17, 2024
12
0
1
Hello Proxmox Community,

I’m currently running a Proxmox VE setup on a Hetzner AX161 server.
I need to replace my current SSDs with larger ones to gain more space. However, I don’t have any additional slots to add the new SSDs without removing the old ones first. I recently upgraded my HDDs (which store backups), but since they aren’t bootable, it was a simpler process.

Current Setup:

Boot Disk:

Purpose: Hosting the Proxmox system, root filesystem, and subvolumes for containers (CTs).
Configuration: 2 SSDs, mirrored ZFS pool (rpool)
Partitions: Each SSD has partitions for BIOS boot, EFI, and ZFS.

Data Disk:

Purpose: Used primarily for storing VZDump backups.
Configuration: 2 HDDs, mirrored ZFS pool (backup)

proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with legacy bios
A1EA-B59D is configured with: uefi (versions: 5.3.18-3-pve, 5.4.60-1-pve, 5.4.78-2-pve), grub (versions: 5.13.19-6-pve, 5.15.158-1-pve, 5.15.158-2-pve)
A1EB-05FB is configured with: uefi (versions: 5.3.18-3-pve, 5.4.60-1-pve, 5.4.78-2-pve), grub (versions: 5.13.19-6-pve, 5.15.158-1-pve, 5.15.158-2-pve)

My plan:

Enable autoexpand:
zpool set autoexpand=on rpool

Detach the old SSD:
zpool detach rpool /dev/disk/by-id/OLD-DEVICE-ID

Shutdown the server before Hetzner change SSD:
shutdown -h now

Partition the new SSD:
sgdisk /dev/disk/by-id/EXISTING-DEVICE-ID -R /dev/disk/by-id/NEW-DEVICE-ID
sgdisk -G /dev/disk/by-id/NEW-DEVICE-ID

Attach the new SSD:
zpool attach rpool /dev/disk/by-id/EXISTING-DEVICE-ID /dev/disk/by-id/NEW-DEVICE-ID

Monitor Resilvering:
zpool status

Rewrite Bootloader:
proxmox-boot-tool format /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool init /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool status

Clean up old entries:
proxmox-boot-tool refresh
proxmox-boot-tool clean

Questions:

Is this approach correct?
Are there any risks I should be aware of?
How can I ensure the first new SSD is bootable before replacing the second SSD?
Are there any checks I can perform before removing the second SSD to avoid system failure? My CTs are critical.
 
This should about work, you do not need the detach, after shutdown and physical disk replacement just copy the partition table :

Copy Partition, remove uuid and label in the process (so that they are not duplicates)

sfdisk -d /dev/WORKING | sed ’s/, uuid.*//; /label-id/d;’ |sfdisk /dev/REPLACEMENT

Replace DISK in ZFS

zpool replace zp_pve /dev/disk/by-id/nvme-OLD-part3 /dev/disk/by-id/nvme-iREPLACEMENT-part3

Monitor Resilvering:
zpool status

Rewrite Bootloader:
proxmox-boot-tool format /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool init /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool status

Clean up old entries:
proxmox-boot-tool refresh
proxmox-boot-tool clean
 
This should about work, you do not need the detach, after shutdown and physical disk replacement just copy the partition table :

Copy Partition, remove uuid and label in the process (so that they are not duplicates)

sfdisk -d /dev/WORKING | sed ’s/, uuid.*//; /label-id/d;’ |sfdisk /dev/REPLACEMENT

Replace DISK in ZFS

zpool replace zp_pve /dev/disk/by-id/nvme-OLD-part3 /dev/disk/by-id/nvme-iREPLACEMENT-part3

Monitor Resilvering:
zpool status

Rewrite Bootloader:
proxmox-boot-tool format /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool init /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool status

Clean up old entries:
proxmox-boot-tool refresh
proxmox-boot-tool clean
Hey thanks for your answer

I think i cannot use the replace command because i need to first remove ssd and then add new one, since i do not have enough slots.
Thats why i dont have anything to "replace" it with and thats why i wanted to use detach/attach :)
 
No, you can replace the missing disk ;)
Ok i understand :) Could you also tell me how can I ensure the first new SSD is bootable before replacing the second SSD?
Are there any checks I can perform before removing the second SSD to avoid system failure? My CTs are critical.
 
You can check with proxmox-boot-tool status if the bootloaders is installed on both disks, unfortunatly you can not really check if the system will boot without booting it ;)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!