Proxmox USB/SD Card Boot Recommendations?

[n1nj4888]

In my home lab, I have 2 Intel NUCs (both only allow 1 * SSD and 1 * NVMe) and I’m currently booting them (UEFI) from the internal SSD. The 2 NUCs are setup in a cluster with an external qdevice for corosync quorum votes.

I would like to use these internal SSDs as VM storage to maximise performance and replicate the storage across the nodes using ZFS (or other?) and Proxmox Replication but I understand that the boot drive cannot currently be set as ZFS under UEFI boot ... Therefore I’d like to explore using either a USB key or SD card to boot each of the nodes and using the internal SSDs as replicated VM storage across the two nodes...

I understand from other posts on this forum that booting from SD / USB keys is not recommended due to the way that Proxmox (Debian) writes to the drives but wondering whether this is outdated information and whether this recommendation only applies to a certain type of SD/USB key storage?

For example. If the SD/USB key uses V-NAND technology (like the internal Samsung EVO SSDs) or similar, are these concerns unfounded?

Thanks!

 

[guletz]

Hi,

SD cards are not design for such use cases. SD cards do not have any controller(like sata/ide/scsi), and the use case is to write on them from time to time secvential data, and to read most of the time. Proxmox(OS) will write a lot of data, most of the time, so the life expectation will be low for a SD!

Good luck!

 

[Vladimir Bulgaru]

Agree with @guletz,

The SD cards can work, as far as i know, with ESXi.

But when it comes to Proxmox - this is basically Debian. And it does write a lot of data on the system partition.

When it comes to the question of quality and technology used to produce the SD card, think that regardless of the technology the product vendors never plan for these SD cards to be constantly bombarded with data. I assume the cards that you mention may last longer, but will eventually die out. Moreover, these cards are not likely to offer you good data integrity and redundancy protection.

 

[LnxBil]

You can always just move your bootloader (EFI) and your kernel to an external usb drive (or sd if you can boot uefi from it). Here is a howto:

https://pve.proxmox.com/wiki/Booting_a_ZFS_root_file_system_via_UEFI

 

[ramblurr]

FWIW, I have the exact same use case. But rather than an sd card or normal usd flash drive, I am using a M.2 SSD to USB enclosure and an M.2 SSD as the proxmox install disk. This leaves the two internal disks free for storage.

 

[cyneric]

When using a USB Stick with 3D-NAND isn't it pretty much the same as an external SSD?
What about MLC ist this the one to prefer?

 

[Dunuin]

When using a USB Stick with 3D-NAND isn't it pretty much the same as an external SSD?
What about MLC ist this the one to prefer?

USB sticks usually use the crappiest NAND chips you can get and there is no advanced controller chip that does proper wear leveling and so on, no DRAM cache or power-loss protection for optimizing writes for less write amplification, ... .
And yes, there are industial grade USB sticks using SLC/eMLC/MLC NAND and these might be fine but also cost a magnitude more than buying a proper NVMe SSD and putting it in an USB enclosure creating your own USB flash storagewith proper duability.

 

[cyneric]

Thanks for replying.
I got the point, but these are not that expensive anymore and would look cleaner on an Intel nuc that unfortunately only offers 1 m.2 Slot. I thought Something Like what I came arround a "Transcend 64GB JetFlash 780" which comes with MLC flash that costs 35€. You got an opinion on that?

 

[Dunuin]

Still no stuff like wear leveling, DRAM caching so small writes won't cause massive write ampliofication and so on. And the "Transcend 64GB JetFlash 780" is at best a pro-sumer stick. Proper industrial grade sticks are more like 160€ for a 32GB one.

 

[LnxBil]

I just bought a 960 GB used enterprise SSD for 39 euro on ebay ... just for comparison. I don't get why people buy those NUC things and want to pack them with storage. That's not the use case they're built for.

 

[Dunuin]

I just bought a 960 GB used enterprise SSD for 39 euro on ebay ... just for comparison. I don't get why people buy those NUC things and want to pack them with storage. That's not the use case they're built for.

I don't understand that too. NUCs aren't cheap, aren't silent, no expandability, no flexibility, badly fixable because of propietary parts and high integration. Only point I understand is the low idle power because of the mobile hardware. Small footprint isn't a point once you add a bunch of external USB devices because you can't put stuff inside which would be more reliable, more silent and better cooled.
In nearly all cases it would be in my opinion better to build something like those small ITX cubes.
It's just a shame that you only get power efficient mobile hardware on ITX/uATX from chinese board manufacturers and that those always won't support ECC.