Storage types clarification

[Zack Coffey]

https://pve.proxmox.com/wiki/Storage_Model So I'm going through this document and I setup a local RAID10 to be LVM. It worked ok for the basic VM creation, but when I tried doing more I ran into walls. According to the document, it's "an ideal solution". However, the document also seems to contradict itself in a few places and other websites seem to confirm that LVM is not very good for many things, (ie. snapshots).

So I'm giving up on LVM and I've decided to either use NFS or directory. However, I can't seem to find decent info on which would be better.

Which would be faster? More versatile? More resilient? Which file system makes a good base layer (ext4, btrfs, xfs, zfs?)

I'd like to have some templates setup of a basic Ubuntu and Windows desktop systems that our engineers can spin up, do some work and then discard. However, I also need to be able to take snapshots so we can reproduce situations before and after changes.

 

[hec]

How many servers do you have?

There will never be the perfect solution which will work for all scenarios. You need always a solution which fits to your needs.

I work with NetApp as shared storage with NFS. Snapshots are created on the NetApp so i do the whole backup logic on the storage system and not with proxmox.

 

[Zack Coffey]

Right now we are still testing the waters, so we have anywhere from 1-4 metal servers and between whatever we come up with will be 6-24 virtuals.

These virtuals will mainly be used as management hosts for running tests on other equipment. So people aren't running all their testing from their office machines.

So they won't be under a huge production mission critical load all the time, we'll need more flexibility than anything else I think.

According to the document, NFS and local directory have the same flexibility. So then my next question will be about performance. Which has the best and worst performance?

 

[LiSergey]

with NFS as storage for VM images, the speed of performance is mainly impacted by network.
if you have separate VLAN between Proxmox host and storage host, and no other trafic in that VLAN, then it should be best.

in my scenario the network between Proxmox and NFS was full of messy broadcasts and other trafic, so I've seen the difference in speed of launching VMs over NFS in daytime (overloaded lan) and nighttime (spare lan).

also, you need to adjust number of NFS server processes and other NFS-/network- specific options.

if you can do LAG on NFS-server and in switch, and bonding in Proxmox, that would also help alot.
e.g. from backup log

INFO: transferred 236223 MB in 1484 seconds (159 MB/s)
INFO: archive file size: 80.05GB
INFO: Finished Backup of VM 202 (00:24:47)

for the flexibility I'd prefer and recommend to store images over NFS, also backup VMs over NFS.

 

[hec]

If you have too many broadcasts you have a too big network. Today i would use 2x10G for each server this should be enough for most scenarios.
For testing or systems which does not need high performance you could use 4x1G in LACP.

NFS/TCP tuning is very important search in the forum i posted this in some threads.

I would always prefer NFS over local storage.