[SOLVED] Install proxmox on an USB 3.0 stick --> Is this a good idea?

fireon

fireon

Distinguished Member
Proxmox Subscriber
Oct 25, 2010
4,493
471
153
Austria/Graz
deepdoc.at
Hello,

I have self build an new Server (Xeon an supermicro board). I have 16 HDD Cases (Proxmox ZFSpool). So my idea was to install Proxmox on an USBstick, so must not set an extra SATA controller. And yes it works! But it is a good idea? I know no Raid, but I'am able to use two USBsticks :)

Can there occur any major problems with this configuration?
...

Thanks and best Regards
 
Hi fireon,

It works great, yet I would suggest using software RAID on the USB sticks and use two different brands to get two different MTBF. Setting up Proxmox on this is a little bit tricky if you're unfamiliar with Linux or Debian yet it can be done in an hour. You can also migrate live to the usb sticks via pvmove. I had to do a similar task last week.

Depending on the USB Sticks it can work a very long or a very short time. Problem with USB sticks is that they wear out on writing and you'll normally have a lot of log switches on a server. A few years back we used CF cards and they held up for about 2 years on a not so IO heavy box. I think you should try to use discard and only use as little space as possible of the usb disk to leave out a lot of space for the wearout logic. We have different configurations with SSD RAID1 with ext4 and use fstrim every day and there are up to 1 GB of freed inodes due to logging, log switches, etc on an otherwise very IO idle box (firewalls, router, etc with logging to syslog server)

Please refer to a SSD guide to get best lifetime out of your stick, e.g. https://wiki.debian.org/SSDOptimization

Best,
LnxBil
 
SATA Dom... really nice thing, i never heard about this before. I see, it can go very bad to use an USBstick... is it easy to over an pciedrive? I've never done this before, but nomaly it should also work great. Has there who experience?
 
spirit said:
Choose quality usbsticks, I have tried 2years ago with a consumer key, I have burn it in 2months.

Maybe a sata dom could be better, if your motherboard support it
Click to expand...

I saw in some other topic:
If you install the Proxmox OS to the SATA DOM (superdom), the OS will kill the drive because of the logging & writing.
 
Huch... this thread is a really long time ago... my experience in this time: NEVER USE SUCH USBDRIVE FOR THE SYSTEM!
I use only big SATA/SAS ZFS rpool. Or two SSDs only for system (ZFS).
 
fireon said:
Huch... this thread is a really long time ago... my experience in this time: NEVER USE SUCH USBDRIVE FOR THE SYSTEM!
I use only big SATA/SAS ZFS rpool. Or two SSDs only for system (ZFS).
Click to expand...

Yep, it's an old post.. i landed here from goggle search:
Ive got four ssd drives and one sata dom (per blade).
So im worried about the lot of writes.. and Im searching for an affordable solution.

So, if i use SATA DOM (64GB), maybe it will die soon (or not?..) :S
 
It depends on the DOM. I've been using USB Sticks on some Hosts but I would recommend against it. If you have some place to spare you could just use an USB Harddrive.

Regards, Jonas
 
El Tebe said:
Yep, it's an old post.. i landed here from goggle search:
Ive got four ssd drives and one sata dom (per blade).
So im worried about the lot of writes.. and Im searching for an affordable solution.

So, if i use SATA DOM (64GB), maybe it will die soon (or not?..) :S
Click to expand...
Putting /tmp and /var/run under tmpfs and place swap another place or entirely do without swap should reduce the write to your SATA DOM. tmpfs has the advantice of growing and shrinking with the size as opposed to ramfs. See https://en.wikipedia.org/wiki/Tmpfs.
 
Since proxmox is based on debian/ubuntu and optimized for harddrive usage, you probably will notice that it wears out flash memory pretty fast.

To counter this you should probably optimize your os + app settings to minimize disk writes.
 
mir said:
Putting /tmp and /var/run under tmpfs and place swap another place or entirely do without swap should reduce the write to your SATA DOM. tmpfs has the advantice of growing and shrinking with the size as opposed to ramfs. See ...
Click to expand...
/tmp and /var/run dir contains will be deleted on reboot. It does not cause erros?
 
Last edited:
zender said:
Since proxmox is based on debian/ubuntu and optimized for harddrive usage, you probably will notice that it wears out flash memory pretty fast.

To counter this you should probably optimize your os + app settings to minimize disk writes.
Click to expand...

Oh thats getting worse.. :(
I have blades (nodes) with 4x Intel 480GB DC S3520 2.5" SATA3 SSDs, and 1x supermicro 64GB SATA DOMs.
I'd like to use the whole sistem for some VMs, and (Docker) containers. Theres no external storage server because its too expensive (and the monthly costs are expensive too).

Maybe it's time to replace two of ssds to hdds from all nodes.. (?)
 
El Tebe said:
/tmp and /var/run dir contains will be deleted on reboot. It does not cause erros?
Click to expand...
Debian will delete any content in /tmp/ and /var/run (and /run for that matter) when you stop the server anyway so this is nothing to be concerned with. Things which much survive a reboot must be placed in /var/tmp
 
mir said:
Debian will delete any content in /tmp/ and /var/run (and /run for that matter) when you stop the server anyway so this is nothing to be concerned with. Things which much survive a reboot must be placed in /var/tmp
Click to expand...
Thank you! This is a huge step forward :)
 
Last edited:
Just monitor your system with iotop and look for the writes it does. /tmp and /var/run etc. are normally ram-based (tmpfs), so no real writes take place on your disks.
 
LnxBil said:
Just monitor your system with iotop and look for the writes it does. /tmp and /var/run etc. are normally ram-based (tmpfs), so no real writes take place on your disks.
Click to expand...
And what about the /var/lib/pve-cluster folder (sqlite's / pmxcfs "continous" writes)?
I have not enough drive place to add more hdds :(
 
My situation is that I have a 1u server with an internal usb port that I would like to utilize for the hypervisor because I want to use the two pcie slots for redundant vm storage. because of this I plan to use a 256GB usb (muf-256ab) so that it will have a lot of spar room for writes.

I'm installing proxmox in place of a server that was running esxi. My question is this: Because esxi is based on freebsd, and both freebsd and linux are posix compliant, shouldn't it be possible to configure proxmox or any other linux distro in such a way that would be optimized for installation on a modern usb drive long term.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back