Need Guidance on Replacing SSDs in ZFS Pool with Limited Disk Slots

[ana_mera]

Hello Proxmox Community,

I’m currently running a Proxmox VE setup on a Hetzner AX161 server.
I need to replace my current SSDs with larger ones to gain more space. However, I don’t have any additional slots to add the new SSDs without removing the old ones first. I recently upgraded my HDDs (which store backups), but since they aren’t bootable, it was a simpler process.

Current Setup:

Boot Disk:

Purpose: Hosting the Proxmox system, root filesystem, and subvolumes for containers (CTs).
Configuration: 2 SSDs, mirrored ZFS pool (rpool)
Partitions: Each SSD has partitions for BIOS boot, EFI, and ZFS.

Data Disk:

Purpose: Used primarily for storing VZDump backups.
Configuration: 2 HDDs, mirrored ZFS pool (backup)

proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with legacy bios
A1EA-B59D is configured with: uefi (versions: 5.3.18-3-pve, 5.4.60-1-pve, 5.4.78-2-pve), grub (versions: 5.13.19-6-pve, 5.15.158-1-pve, 5.15.158-2-pve)
A1EB-05FB is configured with: uefi (versions: 5.3.18-3-pve, 5.4.60-1-pve, 5.4.78-2-pve), grub (versions: 5.13.19-6-pve, 5.15.158-1-pve, 5.15.158-2-pve)

My plan:

Enable autoexpand:
zpool set autoexpand=on rpool

Detach the old SSD:
zpool detach rpool /dev/disk/by-id/OLD-DEVICE-ID

Shutdown the server before Hetzner change SSD:
shutdown -h now

Partition the new SSD:
sgdisk /dev/disk/by-id/EXISTING-DEVICE-ID -R /dev/disk/by-id/NEW-DEVICE-ID
sgdisk -G /dev/disk/by-id/NEW-DEVICE-ID

Attach the new SSD:
zpool attach rpool /dev/disk/by-id/EXISTING-DEVICE-ID /dev/disk/by-id/NEW-DEVICE-ID

Monitor Resilvering:
zpool status

Rewrite Bootloader:
proxmox-boot-tool format /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool init /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool status

Clean up old entries:
proxmox-boot-tool refresh
proxmox-boot-tool clean

Questions:

Is this approach correct?
Are there any risks I should be aware of?
How can I ensure the first new SSD is bootable before replacing the second SSD?
Are there any checks I can perform before removing the second SSD to avoid system failure? My CTs are critical.

 

[ubu]

This should about work, you do not need the detach, after shutdown and physical disk replacement just copy the partition table :

Copy Partition, remove uuid and label in the process (so that they are not duplicates)

sfdisk -d /dev/WORKING | sed ’s/, uuid.*//; /label-id/d;’ |sfdisk /dev/REPLACEMENT

Replace DISK in ZFS

zpool replace zp_pve /dev/disk/by-id/nvme-OLD-part3 /dev/disk/by-id/nvme-iREPLACEMENT-part3

Monitor Resilvering:
zpool status

Rewrite Bootloader:
proxmox-boot-tool format /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool init /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool status

Clean up old entries:
proxmox-boot-tool refresh
proxmox-boot-tool clean

 

[ana_mera]

This should about work, you do not need the detach, after shutdown and physical disk replacement just copy the partition table :

Copy Partition, remove uuid and label in the process (so that they are not duplicates)

sfdisk -d /dev/WORKING | sed ’s/, uuid.*//; /label-id/d;’ |sfdisk /dev/REPLACEMENT

Replace DISK in ZFS

zpool replace zp_pve /dev/disk/by-id/nvme-OLD-part3 /dev/disk/by-id/nvme-iREPLACEMENT-part3

Monitor Resilvering:
zpool status

Rewrite Bootloader:
proxmox-boot-tool format /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool init /dev/disk/by-id/NEW-DEVICE-ID-part2
proxmox-boot-tool status

Clean up old entries:
proxmox-boot-tool refresh
proxmox-boot-tool clean

Hey thanks for your answer

I think i cannot use the replace command because i need to first remove ssd and then add new one, since i do not have enough slots.
Thats why i dont have anything to "replace" it with and thats why i wanted to use detach/attach

 

[ubu]

No, you can replace the missing disk

 

[ana_mera]

No, you can replace the missing disk

Ok i understand Could you also tell me how can I ensure the first new SSD is bootable before replacing the second SSD?
Are there any checks I can perform before removing the second SSD to avoid system failure? My CTs are critical.

 

[ubu]

You can check with proxmox-boot-tool status if the bootloaders is installed on both disks, unfortunatly you can not really check if the system will boot without booting it