Proxmox on SD card

aleno

New Member
Oct 18, 2017
1
0
1
42
Hello,

Is it a good idea to install Proxmox VE 5 on an SD card?

I have a new Dell R630 server with a dual SD card and I would avoid using my SSD disks to install the system.

Thank you
 

fireon

Famous Member
Oct 25, 2010
3,923
333
103
40
Austria/Graz
iteas.at
Is it a good idea to install Proxmox VE 5 on an SD card?
Proxmox is based on Debian. So it is a normal Linuxdistribution with all these features, logs and and and. So the card will not hold on a long time, maybe a half a year. So "NO" it is not a good idea. Use some Professional/Enterprise SSD(s).
 

kobuki

Well-Known Member
Dec 30, 2008
470
26
48
I wonder about that too. If you put the log dirs (eg. /var/log) on a different disk the write load and thus wear on the card should be almost non-existent.
 

fireon

Famous Member
Oct 25, 2010
3,923
333
103
40
Austria/Graz
iteas.at
One disk one card... single point of failure. And the most flashcards are extremly slow. HP's default card in the last server what would like to buy was 13MB/s write and 25MB/s read. So first you need to buy an expensive flash and fast flash. And second better would be two of them ;)
 

kobuki

Well-Known Member
Dec 30, 2008
470
26
48
These Dell servers have a dual SD drive, so you can use a mirror. In contrast, none of the Raspberry Pis that I've been using had a failed SD card, while I've been even using mysql on some of them - hobby stuff, nothing serious, but still. For a boot device their speed is more than adequate. Normally you only need to reboot and use them more extensively every few months or more for kernel updates (having put logs elsewhere). Any decent vendor's cards are far from the anecdotal unreliability that circulates around on the net.
 

Rhinox

Active Member
Sep 28, 2016
272
38
28
32
Those SD-slots on Dell (and many other) servers are good if you are using ESXi or any other OS running basically from memory (but that's not the case of PVE). If you do not want to use hdd/ssd for PVE, better choice is USB-stick or CF-card (especially those using SLC-NANDs)...
 

kobuki

Well-Known Member
Dec 30, 2008
470
26
48
To me this fear of SD cards is not realistic. What wears out these cards is excessive write. If you keep most on separate partitions on spinning disks or SSDs I see no actual downside to them as boot devices. For practical purposes, you can consider a booted PVE system like a VMWare appliance.
 
  • Like
Reactions: El Tebe

wolfgang

Proxmox Retired Staff
Retired Staff
Oct 1, 2014
6,496
498
103
Hi,

@kobuki We write per day without optimization about 10GB. Now you can consider if you SQL on you Raspberry-PI have the same load.

Dell as an 100% over provisioned SD what will wear about 500 write cycles per cell.
This would reach the theoretically ware out of a 16GB SD card in 600 days.

write intensive dir are
/var/log
/var/lib/pve-cluster

Also if HA is not needed you can turnoff pve-ha-lrm and pve-ha-crm to reduce writes.
 
  • Like
Reactions: pvps1 and El Tebe

kobuki

Well-Known Member
Dec 30, 2008
470
26
48
With that kind of write load I'd suggest moving away mentioned dirs (or the whole /var) to other disks, especially in production. But then I think there should be no problems with the cards.
 

Rhinox

Active Member
Sep 28, 2016
272
38
28
32
Is it possible to disable services (i.e. pve-ha-lrm & pve-ha-crm) from web-interface now? I remember I had to do it from command-line (systemctl). There was only stop/start/restart in web-interface, so after rebooting pve-host those services were running again...
 

wolfgang

Proxmox Retired Staff
Retired Staff
Oct 1, 2014
6,496
498
103
Is it possible to disable services (i.e. pve-ha-lrm & pve-ha-crm) from web-interface now? I remember I had to do it from command-line (systemctl). There was only stop/start/restart in web-interface, so after rebooting pve-host those services were running again...
Correct you have to use the command line.
Be aware if you like ha later this services must run.
 

tommytom

Active Member
Aug 22, 2015
32
5
28
Those SD-slots on Dell (and many other) servers are good if you are using ESXi or any other OS running basically from memory (but that's not the case of PVE). If you do not want to use hdd/ssd for PVE, better choice is USB-stick or CF-card (especially those using SLC-NANDs)...

can you recomment me such drives? I Run HP-DL380Gen9 Servers. They have internally 1 SD Cards Slot and also USB3.0 Slots
i would like to boot from either SD or USB stick. But i am not sure which one would work best here.
I also could attache a USB SSD with cable somehow. but i would like to have a stick instead
 
Jun 4, 2019
7
2
8
47
Hi,

@kobuki We write per day without optimization about 10GB. Now you can consider if you SQL on you Raspberry-PI have the same load.

Dell as an 100% over provisioned SD what will wear about 500 write cycles per cell.
This would reach the theoretically ware out of a 16GB SD card in 600 days.

write intensive dir are
/var/log
/var/lib/pve-cluster

Also if HA is not needed you can turnoff pve-ha-lrm and pve-ha-crm to reduce writes.
Hi, I'd like to use Dual SD Card Module on Dell server.
Could I mount write intensive directories on RAM Disk?

Does /var/lib/pve-cluster (for example) need permanent storage?
I have a lot of RAM, but I don't want to waste disk bays for the operating system - I want to use them only for CEPH.
 

Dunuin

Famous Member
Jun 30, 2020
8,119
2,028
149
Germany
Where do you get that 1920 TBW? Can't find any datasheet mentioning a TBW for any Kingston SD card. In general SD cards are missing stuff like wear leveling, RAM/SLC caching and so on so the write endurance is terrible.
There are some durable SD cards using SLC NAND like the Apacer Industrial stuff (but really only SD cards, microSDs got a to small footprint for SLC NAND and don't go above 8GB) but there you pay 200-400 bucks for a single 32GB SD card and not just 10 like for the kingston.
 
Last edited:
Jun 4, 2019
7
2
8
47
Where do you get that 1920 TBW? Can't find any datasheet mentioning a TBW for any Kingston SD card. In general SD cards are missing stuff like wear leveling, RAM/SLC caching and so on so the write endurance is terrible.
There are some durable SD cards using SLC NAND like the Apacer Industrial stuff (but really only SD cards, microSDs got a to small footprint for SLC NAND and don't go above 8GB) but there you pay 200-400 bucks for a single 32GB SD card and not just 10 like for the kingston.
https://www.kingston.com/poland/en/memory-cards/industrial-grade-microsd-uhs-i-u3
Mayby that to true? Marketing?

Probably I'll use pci-e converter with small SSD drives any way.
 

Dunuin

Famous Member
Jun 30, 2020
8,119
2,028
149
Germany
It is only using TLC NAND writing to it in SLC mode, so like SSDs do it with SLC caching instead of using fast and super durable SLC NAND which is super small and expensive.
But atleast they indeed rate it with 1920 TB TBW. But they write "up to 1920 TBW". Usually with SSD they guarantee the drive will survive "at minimum" the TBW they rate the drive for. But I guess in raid1 the MicroSDs should be fine.
 
Jun 4, 2019
7
2
8
47
That would work for a simple setup, but what about a reboot? I cannot rely on stuff that needs manual intervention to work again.
- failure of one node should not impact the entire environment anyway
- SD cards work in RAID 1 - so we are postponing the failure for the single node
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!