[SOLVED] Install Proxmox 5.1 on USB Flash Drive?

[mhayhurst]

Hello Everyone,

According to this older thread installing Proxmox on a USB flash drive was discouraged. Is that still the case with Proxmox 5.1? FreeNAS, for example, encourages installing onto a USB flash drive for their ZFS setup. That's why I was wondering if this has changed for the latest Proxmox.

 

[elmacus]

Usb drives will burn out quickly due to writing on them.

 

[mhayhurst]

Usb drives will burn out quickly due to writing on them.

I was under the impression that the majority of reading and writing would take place on the ZFS HDD's/SSD's, is that not so? I'm just trying to understand why two systems (FreeNAS and Proxmox) both of which use ZFS but one strongly encourages installation on a USB flash drive and the other does not. Yet both systems will be doing a similar amount of reading and writing to the USB flash drive and I have not seen any warnings regarding FreeNAS mentioning concerns with burning out the USB flash drive. If I'm wrong in my thinking and do not care about burning out drives every so many months/years could I simply reinstall Proxmox on a new USB flash drive, import the ZFS pool and everything should be up and running?

 

[LnxBil]

FreeNAS ans Proxmox differ in so many things. FreeNAS is mainly a NAS distribution, so 'network attached storage', whereas PVE is a virtualization software which is not (out-of-the-box) capable of serving files, so not a NAS at all. The former uses FreeBSD and the latter Linux and yes, they both have ZFS, but again, this is also not the same. Both implementation are similar but not equal. FreeBSD is much more advanced and stable than ZFS on Linux, because it is more mature and maintained by FreeBSD itself (part of the OS), whereas ZFS is not part of official Linux, so fewer people maintain it and everything is not as smooth as it is in FreeBSD-based operating systems. Also their open source license is compatible.

But back to the USB question:

The forum has many, many posts about using USB and why it is bad, so I will only summarise it:

PVE write a lot to the disk, mainly synchronising the backend of /etc/pve. Linux or in this instance Debian itself is not optimized for omitting writes. It does this on purpose and it's not going to change in the near future, so you'll going to wear out your USB stick.

Of course you can try it besides all the negative feedback here but please backup regularly because it will fail soon.


Oh ... and the configuration files for your virtual machines are not stored on the ZFS itself, but on the USB stick (in /etc/pve).

Why don't you just install PVE on the ZFS pool itself? It has only a footprint of roughly 2 GB (compressed) and it just works.

 

[mhayhurst]

FreeNAS ans Proxmox differ in so many things. FreeNAS is mainly a NAS distribution, so 'network attached storage', whereas PVE is a virtualization software which is not (out-of-the-box) capable of serving files, so not a NAS at all. The former uses FreeBSD and the latter Linux and yes, they both have ZFS, but again, this is also not the same. Both implementation are similar but not equal. FreeBSD is much more advanced and stable than ZFS on Linux, because it is more mature and maintained by FreeBSD itself (part of the OS), whereas ZFS is not part of official Linux, so fewer people maintain it and everything is not as smooth as it is in FreeBSD-based operating systems. Also their open source license is compatible.

But back to the USB question:

The forum has many, many posts about using USB and why it is bad, so I will only summarise it:

PVE write a lot to the disk, mainly synchronising the backend of /etc/pve. Linux or in this instance Debian itself is not optimized for omitting writes. It does this on purpose and it's not going to change in the near future, so you'll going to wear out your USB stick.

Of course you can try it besides all the negative feedback here but please backup regularly because it will fail soon.


Oh ... and the configuration files for your virtual machines are not stored on the ZFS itself, but on the USB stick (in /etc/pve).

Why don't you just install PVE on the ZFS pool itself? It has only a footprint of roughly 2 GB (compressed) and it just works.

Okay, I understand. I was just assuming since they both used ZFS the frequency of reading and writing would be about the same. Plus, FreeNAS comes with a native virtual machine plugin, so that only added to my thought process that both systems might operate similarly under the hood...as far as reading and writing go. Thank you for the info!

 

[michaelservices]

Wanted to share my experience using USB thumb drives with hypervisors.

While I have not done it with Proxmox (yet) I have used USB thumb drives on about a dozen servers in an Enterprise environment with VMWare ESXi. The whole purpose here was to not waste a perfectly good drive only for the hypervisor (esxi) since it is very small. The other important factor was separating the datastore from the hypervisor.

The configuration we used in an Enterprise environment was datastore was raided or NAS'd, and hypervisor was on USB thumb drive, with an extra one taped, wedged, or zip tied somewhere in the case. The worst case scenario is walk over or call someone near the server to unplug the failed USB and plug in the other one, then reboot, reconnect the datastore and your back in action. no fuss.

All the talk about USB sticks not being reliable I think is mostly hogwash. I have sticks that are 10+ years old still working just fine, some of which don't even have the plastic covers anymore and are just open circuit boards still working perfectly.

Plus any hypervisor should have a replica (of itself and everything on it) somewhere so redundancy and disaster scenarios shouldn't be a problem. Especially this day and age. I can't tell you how many times I've seen professional IT Admins, and companies not have PROPER BACKUPS, and not have everything virtualized + replicated.

 

[geppi]

ESXi does not write to the USB thumb drive after completing the boot process.
It pretty much completely resides in RAM.
ESXi also takes the following measures to avoid writing to the USB thumb drive:

Due to the I/O sensitivity of USB and SD devices the installer does not create a scratch partition on these devices. When installing on USB or SD devices, the installer attempts to allocate a scratch region on an available local disk or datastore. If no local disk or datastore is found, /scratch is placed on the ramdisk. After the installation, you should reconfigure /scratch to use a persistent datastore.
(source: https://kb.vmware.com/s/article/2004784)

The situation with Proxmox is different.

 

[michaelservices]

ESXi does not write to the USB thumb drive after completing the boot process.
It pretty much completely resides in RAM.
ESXi also takes the following measures to avoid writing to the USB thumb drive:

Due to the I/O sensitivity of USB and SD devices the installer does not create a scratch partition on these devices. When installing on USB or SD devices, the installer attempts to allocate a scratch region on an available local disk or datastore. If no local disk or datastore is found, /scratch is placed on the ramdisk. After the installation, you should reconfigure /scratch to use a persistent datastore.
(source: )

The situation with Proxmox is different.

Thanks geppi!

 

[Etienne Charlier]

May I frankenstein this topic

What about usb DOM ( SLC flash ) like
MTEDCAE008SAJ-1N2 ( micron eusb module)
Is it endurent enough for a proxmox system disk ?

 

[elmacus]

May I frankenstein this topic

What about usb DOM ( SLC flash ) like
MTEDCAE008SAJ-1N2 ( micron eusb module)
Is it endurent enough for a proxmox system disk ?

SLC flash should be ok for Debian and Proxmox.

 

[Etienne Charlier]

Many Thanks,
Etienne