[SOLVED] What is preventing Proxmox from being optimized to run on a USB flash drive?

[AaronWalker]

The general consensus seems to be that reading and writing onto a USB often would wear down the flash drive quickly.

However, I believe ESXi loads the hypervisor into memory on boot. What is stopping Proxmox from implementing a similar setup?

 

[fireon]

Proxmox is a normal full Linux Operatingsystem based on Debian and use KVM and LXC. It can more then virtualsations, and that is one of the big advantages. ESXi is optmized to do only virtualsation. So it is more easier to some optimize like running from flash to memory. We running PVE on the HWraid with the vm's or on two extra SSD's.

 

[dietmar]

However, I believe ESXi loads the hypervisor into memory on boot. What is stopping Proxmox from implementing a similar setup?

You can already install Proxmox onto USB flash drives. The result is exactly as useful as installing ESXi on USB flash drive (as reliable as the USB drive).

 

[fireon]

You can already install Proxmox onto USB flash drives. The result is exactly as useful as installing ESXi on USB flash drive (as reliable as the USB drive).

Yes when you see that so thats right, but it will be much slower by updates, installing packages, using the system, and you have an single point of failure. In my eyes it makes no sence to install PVE on a flashdrive. Ok, you can use two sticks with Raid, but i prefer the SSD way

 

[dietmar]

Yes when you see that so thats right, but it will be much slower by updates, installing packages, using the system, and you have an single point of failure.

I just wanted to point out that you have exactly the same disadvantages when you install ESXi - those are problems of USB flash drives in general, and not really related to proxmox ve.

Besides, small SSD flash drives are much more reliable and quite cheap (1€/GB).

 

[fireon]

I just wanted to point out that you have exactly the same disadvantages when you install ESXi - those are problems of USB flash drives in general, and not really related to proxmox ve.

I mean that generic... but i I always thought that esxi is optimized for flash. But yes, the main problem is on every OS the same. Singel point of Failure with OS on only one drive. So for me that flash solution is a no go. For testing and playing around, but not for production.

 

[LnxBil]

ESXi does not write to the disk much as Linux in general does, so one needs a optimisation there. But @dietmar is right, SATA-DOMs are not that expensive anymore and are used in servers directly. They're better than USB disks, faster and more reliable.

The SATA-DOM solution is only recommended if you have a cluster and the vm data is stored in a clustered manner. I'd install ZFS (for easy replication and external backup storage) on the DOM to have a bootable backup in case the flash disk fails. One DOM as spare connected to the backup server and you'd be able to get the server running in almost no time. Yet the SPOF remains for any virtualisation platform.

In general, this depends on your HA policy and your available money for hardware. Even the smallest and slowest hard disk drives with a "real" hardware raid controller (or ZFS) costs a lot more in purchase and running costs than the SATA DOM solution - yet I also prefer that solution.

 

[AaronWalker]

Thanks all for the input. It was helpful to hear the pros and cons of running Proxmox on a USB.
I should have clarified this is for a home environment.
With that being said, I am going to continue running Proxmox on a USB.
If it dies, it dies. If it Proxmox runs a tad slower, meh, I am not on Proxmox much anyway after setup.

 

[Rhinox]

There are a few high-endurance USB-3.0 sticks too, i.e. Mach Extreme MX-ES or Winkom Pendrive. Given that these use SLC-NANDs, they can hold at least as long as any consumer-SSD. With price ~1.5€/GB they are not that expensive...

 

[r4mzy]

I know it's not quite the same, but how about having Proxmox run from RAM like a live disc? Build your Proxmox host with all configs, mappings to shared storage, cluster membership, etc., then roll that system into a live disc to boot from, so reads only happen once on boot. And if you boot from a flash drive, AFAIK it should be possible to write persistent changes to the drive so they are maintained after reboot?
If the booting host is a member of a cluster (or boot scripts make it automatically join one), and all your VM stores are on shared storage, then could the cluster automatically migrate VMs to and from "live" hosts as they come up and down?

 

[Rhinox]

It is actually very different. If you boot live-disc, you can mount some static libs/bins from it, but otherwise you have to keep the whole filesystem in RAM. And in case of PVE it would not be so easy, as it is type-2 hypervisor (on top of full OS), so you could count some GB for it if you want to keep it in memory. And if you do not want to loose any changes, you'd have to re-create image every shutdown/reboot.

ESXi is type-1, very minimalistic OS focused on only one thing. IIRC, it has just about 100MB footprint and very small OS-image, so it is very easy to load the whole OS-image into RAM. But it has its own disadvantages (i.e. difficult "installation" of add-on SW)...

 

[dswartz]

Maybe I am misunderstanding, but there is a fundamental difference here. ESXi does not require the USB drive to be functional. e.g. it can completely die, and ESXi is still fine, since it is in 'run from RAM' mode. I don't think proxmox would work that way, would it?

 

[PMoxSudo]

I run a personal home server from a usb device. Today I've moved /var and /usr to a pair of raid1 wd reds to avoid disk io wait time and minimize writes/reads to/from the usb device

 

[LnxBil]

I still don't get why someone would not want to have 2 GB of space for Proxmox VE on their (hopefully mirrored) 8+ SATA TB drives in a home environment.