SD card deployment

[tcabernoch]

I'm converting an old ESXi VSAN cluster to CEPH.

ESXi ran on a Dell IDSM, which is a RAID 1 pair of SD cards in a device to make it look like 1 card.

I want all the SSD disks I can use for CEPH, so I want to deploy PVE on the old SDs.
I did set up the cluster on the SD cards once. it worked.
- Deploy on LVM, not ZFS, for fewer disk writes.
- LVM rocket surgery to remove the swap and data volume and reallocate the space to root.

That was functional. I burned it down because I was testing other things, but I can go back to it just by switching the boot disk.

Question is, is this a good idea?
Anybody have experience running PVE on LVM formatted SD cards like ESXi? (Definitely not ZFS.)
How did that go?

 

[BD-Nets]

Depends on the used SD-Cards. VMware uses the card only as boot image provider and runs fully from RAM.
With VMware Version 7.0.0. till 7.0.2 there was an issue (scatchpad area assigned to SD-Card) which quickly (2-3 months) destroyed SD-Cards.
Debian/Proxmox was not written with SD-Card in mind (unlike Debian/Raspbian) so yourd SD-Card might die soon, too.

 

[Kingneutron]

It's probably not a good idea. SD cards are OK if you want to boot a recovery environment, but not if you're running a server 24/7.

You can do things to mitigate writes like log2ram, sending syslog to a central instance, noatime, etc... but realistically Proxmox is not ESXI. It doesn't run the same way.

You'd be better off installing pve to an external SSD like samsung T7. Your I/O will definitely be faster than SD. You can specify the swap size, etc during install by using the advanced options when it asks what disk(s) to install to.

 

[tcabernoch]

Thanks guys. I appreciate the potential workaround suggestions.
I had already come to the conclusion that I'm gonna lose two capacity drives just to the OS.
You've confirmed.

What's up Neutron? It's been a minute. I've been very busy. Too busy to play here. Oh, and I found Lemmy to fill my time. :]

 

[readyspace]

I normally use NVMe running onboard without RAID because it is already on HA cluster mode for PVE host install for performance purpose.

Using SDCard sounds very risky to me.

 

[UdoB]

Question is, is this a good idea?

Before I go that route I would take two (or three!) USB-to-NVMe or -to-SATA Adapters and put some small known-good devices in there.

That's not really professional, and I have always said that USB is not a good idea in this context. I had run a MiniPC for some time this way with a triple mirror - because of malfunctions...

 

[tcabernoch]

Hey Udo.
Ya, final shape was 1 OS disk + 7 CEPH disk.
I do have the formatted and built out SD card array as an alt-boot standby if I lose an OS disk, but the OS disks are essentially new SSD. I'm fairly confident in them.

 

[LnxBil]

unlike Debian/Raspbian) so yourd SD-Card might die soon, too.

That is not really the case. If you do the same as PVE does (write constantly to e.g. update the graphs and PVE filesystem), you would also wear out the same SD cards easily with Raspbian ... the userland is both the same. Both PVE ans raspbian are Debian-based.

 

[LnxBil]

Ya, final shape was 1 OS disk + 7 CEPH disk.

Have you tried putting in a (used) Dell BOSS card? They work prett well and also do not take up disk slots, but a PCIe slot