Running proxmox from a USB drive? - SOLVED

J

Joseph Chrzempiec

Well-Known Member
Jun 9, 2016
142
8
58
45
Hello i was curious if it would be okay to run proxmox from a USB drive? The reason why I'm asking is because i have a server that only has two sata ports and it is very small form factor case. I need to run the two sata drives in raid one for backup. But i don't have a main boot drive. So i was wondering if i would be okay with that flash drive?


Joseph
 
Normal USB flash drives? No. The main reason is that Proxmox VE writes quite a bit of logs and that will kill your USB flash drive in a short time. Besides that, Proxmox VE will use the flash drive as a normal hard drive and for that, flash drives are in my experience terribly slow.

If you have to use something small you could try to use a USB based SSD? Maybe some other users can contribute some experience with such a setup because I don't have any.
 
i've no problem yet with two systems with 64GB MLC usb key (Transcend 780).
3 GB root used by Proxmox OS.
i disable swap to reduce write activity.

one custom built since feb. 2021
dedicated for Xpenology because passthrough 5 hdd in raid5 by Synology got from faulty atom soc ds1515 thanks to Intel & Synology.

second system since jan. 2022, hp proliant ML 350p , swap disabled.
dedicated ssd ext4/Lvmthin with only a Windows server 2012r2 ssd ext4/Lvmthin VM.
 
Last edited:
Hello, Everyone Thank you all for the information and Help. What I Did at that time I have bought a 64GB usb SSD. and I took my 2 Hard drives raidded them together as one drive. It all works and still running up to this day. Thank you all for the help.


Joseph
 
Dunuin said:
Should work. But they are just empty cases, you still need to buy some more or less durable M.2 SSDs.
Click to expand...

This is a older system no M.2 only have two Sata ports on it. One of them very small old Dell Desktop. But I have gotten it to work from back then.
 
Hello All, What I mange to did back then was bought a USB 32GB ssd it was cheap back then. And used that as my proxmox drive. and use the two sata ports for 2 320GB drives. The system had build in raid anduse them drives as my VM drives.

I still use this system up to this day.


Joseph
 
Joseph Chrzempiec said:
Hello All, What I mange to did back then was bought a USB 32GB ssd it was cheap back then. And used that as my proxmox drive. and use the two sata ports for 2 320GB drives. The system had build in raid anduse them drives as my VM drives.

I still use this system up to this day.


Joseph
Click to expand...
Just curious, you indicate that you still use this system up to this day but you did not indicate when you started using it? just curious if it has been for 2+,3+, 5+ years? (My apologies for bringing up a very old thread.)
 
"USB SSD" should be fine (so a real M.2 or 2.5" SSD but in a Case with USB-Plug or some dedicated USB-SSDs like a Sandisk/Samsung Portable SSD). Just don't use SD-Cards or USB pen drives without wear leveling, cheap NAND, missing SLC/RAM caching and so on.
 
Dunuin said:
"USB SSD" should be fine (so a real M.2 or 2.5" SSD but in a Case with USB-Plug or some dedicated USB-SSDs like a Sandisk/Samsung Portable SSD). Just don't use SD-Cards or USB pen drives without wear leveling, cheap NAND, missing SLC/RAM caching and so on.
Click to expand...
Just found this thread. I'm considering setting up a Proxmox server using an Intel enterprise SATA SSD in a USB 3.0 enclosure, so this is exactly the info I was looking for. :)

(I've only got two NVME slots and want to pass those NVME to a TrueNAS VM, and I've only got one PCIe slot that I don't necessarily want to tie up with additional NVME yet (I don't want to put more money into this box right now, and might want to put a real GPU in that slot later).

Assuming the USB SATA enclosure has UASP support, and the actual disk has high endurance, are there any risks to this approach from, e.g., the USB drive trying to enter a low power state or something? I'm not too worried about drive writes. Those Intel enterprise SATA disks have huge endurance, and I'll be setting up a log server at some point soon.
 
Proxmox VE and Debian doesn't require enterprise SSD themself as writes are minimal compared to VMs.

The two hosts keep running from usb key since my previous post.
 
Last edited:
_gabriel said:
Proxmox VE and Debian doesn't require enterprise SSD themself as writes are minimal compared to VMs.
Click to expand...
I would not rule out Proxmox as it does log quite a lot (and data for graphs) more than plain Debian (or other Linux). And maybe this has been reduced recently, but the pmxcfs also used to have a lot of write amplification.
But I agree that VMs with a lot of (synchronous) I/O, especially when the block sizes don't match, can easily become the major cause when the number of running VMs increases.
 
Just to clarify, my use case is for PVE OS/boot only. So, single disk ZFS stripe.

VM and LXC containers would go on an NVME mirror pool.

I'll keep an eye on the wear out, and replace the disk if needed (I've learned how to migrate a Proxmox install to new boot disks now), but I was more than anything else concerned about the USB connection being too unstable for this.

But it sounds like that's not the case, especially with a powered SATA to USB enclosure.
 
_gabriel said:
even self powered USB enclosure isn't a problem as SSD draw less current than HDD.
Click to expand...
It's less the enclosures I'm worried about and more the hubs. Even more expensive USB hubs are not consistent about powering USB storage devices hanging off of them that are meant to be bus powered, in my experience.

A powered hub works every time (unless you're using a Raspberry Pi 4b with its wonky USB circuits, but that's a whole other thing).
 
I ran Proxmox for a number of years on a Raspberry Pi on a 128gb m.2 with one of the above Amazon USB enclosures, no additional power supply, no other disks. No it wasn't a supercomputer, but it ran a number of LXCs, VMs & host services - and had no issues. It still works as is.
 
  • Like
Reactions: SInisterPisces
I also have the same concerns, I have very limited space, in my set up I have a bunch of usb ports, only one PCIe 4.0 slot and 4 3.5" bays. I'm planning on using the 4 bays just for data storage. In that case, I most likely need to run Proxmox from USB. I had a bunch of M.2 SSD lying around and went ahead and bought an enclosure for one to install Proxmox on it. but I hear a lot of chatter about issues with IOPS if I take that route. I'd appreciate some help in finalizing this set up.

Also some additional questions, I probably have to use the PCIe interface to put in an NVMe drive in there for VMs. The system doesn't support bifurcation, how bad would it be if I put Proxmox on m.2 SSD and run it from the USB port, put the VMs on the NVMe drive and not have any mirrors for either of the dives? And what are the limitations with this set up?

p.s.: I can possibly get a NVMe PCIe adapter with a chip and put 2 NVMe drives on it and mirror the VM drive, but still no way of mirroring the boot drive.
 
hooman64 said:
but still no way of mirroring the boot drive
Click to expand...
I don't see that as such an obstacle. What I would do in your situation:

1. Use a small m.2 drive. 128gb?
2. From time to time (every kernel update etc.), make a dd zipped image of that boot drive to the storage area on the server.
3. Something goes wrong, write that dd image back to the boot drive (or a new one).
4. Make as little changes to the host OS as possible & document all those changes.
5. Document & backup all Proxmox host settings. (Search these forums on which files/folders to backup).
6. You could always re-install Proxmox & redo 4 & 5 above.

For all your VMs & LXCs you must anyway have full & restorable backups.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom