Proxmox on SD card

[redguy]

You could wrap the tar -czf ... in a systemd unit and add a systemd drop-in configuration for whatever service that needs /var/lib/pve-cluster (I guess it's just pve-cluster.service) containing something along the lines of

[Unit]
Requires=the-tar-backup.service
After=the-tar-backup.service

 

[robetus]

I found this, may be of interested to this thread: https://www.mouser.com/catalog/additional/SanDisk_WD_Write-Intensive_ProdBrief_US_1017161.pdf

 

[no-usernames-left]

I bought a few Kingston Industrial SD cards to use as my boot drives for machines which don't support booting from NVMe (Dell R710):
https://www.kingston.com/datasheets/sdit_en.pdf

They're run in pSLC mode which greatly increases endurance, "up to" 1920 TBW which applies to the largest capacity (64 GB). Since TBW is proportionate to capacity, my 16 GB cards should have an endurance of 480 TBW... more than enough for a disk which will almost never be written to (only used to boot the box and pass control to the ZFS NVMe mirror).

Be warned with buying these on Amazon etc; since AFAIK Amazon stock is commingled with third-party FBA stock, you might wind up with counterfeit cards even if you choose the item which is listed as being sold by Amazon themselves. To avoid this, I bought them from Mouser, who guarantees in writing on the packing slip that they're either purchased directly from either the manufacturer or their authorised supply chain.

 

[LnxBil]

I bought a few Kingston Industrial SD cards to use as my boot drives for machines which don't support booting from NVMe (Dell R710):

I used an old usb stick I had lying around for the boot device on one machine with the same problem as you described. The boot device does not have problems with too many writes, just the ordinary PVE root.

 

[crashtest]

Digging this back up.

A standard ubuntu installation allows one to use a Dell IDSDM(basically an internal SD card mirror) exclusively for the boot partition (/boot), with a seperate file system (on NVME or whatever) mounted at / .

Can this not be done with proxmox? Can you not put the boot partition on the SD Card with the root file system placed elsewhere, like say on a zfs pool?

 

[spirit]

Digging this back up.

A standard ubuntu installation allows one to use a Dell IDSDM(basically an internal SD card mirror) exclusively for the boot partition (/boot), with a seperate file system (on NVME or whatever) mounted at / .

Can this not be done with proxmox? Can you not put the boot partition on the SD Card with the root file system placed elsewhere, like say on a zfs pool?

you can install proxmox on a internal nvme m2 drive (enterprise/dc grade, with a superpacitor).
If you have DEll, look a boss
https://www.dell.com/support/manuals/en-us/poweredge-r7425/boss_s1_ug_publication

proxmox is continuously writing small write (for stats, graphs, cluster exchange,....).

if you use sdcard,you are going to burn them after 1 or 2 months.

(even vmware don't support to use sdcard anymore for install)

 

[no-usernames-left]

Digging this back up.

A standard ubuntu installation allows one to use a Dell IDSDM(basically an internal SD card mirror) exclusively for the boot partition (/boot), with a seperate file system (on NVME or whatever) mounted at / .

Can this not be done with proxmox? Can you not put the boot partition on the SD Card with the root file system placed elsewhere, like say on a zfs pool?

This is absolutely possible because I am doing it today on an R710 which cannot boot from NVMe.

I bought a high-quality industrial pSLC SD card, but you can use a USB flash drive if you want since it is never written to other than for kernel updates.

I set up EFI and boot partitions and initialized it with proxmox-boot-tool, and the R710 boots to that internal SD card which then passes execution to the root partition which is a ZFS mirror on a pair of NVMe SSDs on a PCI card equipped with a PCIe switch (because no bifurcation either).

When Proxmox has a new kernel to install, it updates the boot partitions on the SSDs (which are never used) and also the SD card (which is).

 

[no-usernames-left]

you can install proxmox on a internal nvme m2 drive (enterprise/dc grade, with a superpacitor).
If you have DEll, look a boss
https://www.dell.com/support/manuals/en-us/poweredge-r7425/boss_s1_ug_publication

proxmox is continuously writing small write (for stats, graphs, cluster exchange,....).

if you use sdcard,you are going to burn them after 1 or 2 months.

(even vmware don't support to use sdcard anymore for install)

Re-read the post to which you replied: they want to use it for /boot only. Proxmox never writes there except to install a kernel update.

 

[crashtest]

This is absolutely possible because I am doing it today on an R710 which cannot boot from NVMe.

I bought a high-quality industrial pSLC SD card, but you can use a USB flash drive if you want since it is never written to other than for kernel updates.

I set up EFI and boot partitions and initialized it with proxmox-boot-tool, and the R710 boots to that internal SD card which then passes execution to the root partition which is a ZFS mirror on a pair of NVMe SSDs on a PCI card equipped with a PCIe switch (because no bifurcation either).

When Proxmox has a new kernel to install, it updates the boot partitions on the SSDs (which are never used) and also the SD card (which is).

It seemed weird that so many were saying it totally shouldn’t be done without ever acknowledging that only the boot partition goes on the SD card.

So it seems it takes a few extra steps to make it happen though? Is the documentation pretty straight forward for it or, if not, are you aware of a decent guide?

 

[LnxBil]

It seemed weird that so many were saying it totally shouldn’t be done without ever acknowledging that only the boot partition goes on the SD card.

It seems weird to me to do this in the first place. Why split your system on 4 devices instead of 2 without any upside. I suspect that therefore a lot of people misunderstand this setup. It makes absolutely no sense to me.

 

[no-usernames-left]

It seemed weird that so many were saying it totally shouldn’t be done without ever acknowledging that only the boot partition goes on the SD card.

Rule number one is that users don't read. We've all been guilty of it from time to time.

So it seems it takes a few extra steps to make it happen though? Is the documentation pretty straight forward for it or, if not, are you aware of a decent guide?

I wrote it up here.

 

[no-usernames-left]

It seems weird to me to do this in the first place. Why split your system on 4 devices instead of 2 without any upside. I suspect that therefore a lot of people misunderstand this setup. It makes absolutely no sense to me.

The box cannot boot from NVMe because the BIOS is too old to support it. (It can apparently be modded and reflashed but fuck that noise.) Therefore, some other intermediate boot device is required.

The "system" is not split, there is simply another copy of the boot partition, which is managed entirely automatically with proxmox-boot-tool.

 

[LnxBil]

The box cannot boot from NVMe because the BIOS is too old to support it. (It can apparently be modded and reflashed but fuck that noise.) Therefore, some other intermediate boot device is required.

Oh yes, that's the usecase I missed ... and yes, we often don't read (correctly) or just don't reread the whole thread.

 

[crashtest]

Rule number one is that users don't read. We've all been guilty of it from time to time.


I wrote it up here.

Sweet thanks!

As it turns out, I’m in the research phase of implementing proxmox in my home lab…. On a dell T430 with the IDSDM and a couple NVMEs thrown in on pcie add-in cards.

Thanks everyone for the input