How to share storage between VMs and proxmox?

[oiwruk]

Hi,
I have attached a large HDD over USB3.0 to my Proxmox node. I use it to store backups of VMs and containers.

However, I also would like to run many media-heavy Docker containers in a VM (Plex, family photos, Nextcloud, etc) and would have to store the media on that said HDD. What is the best way in terms of setup and performance to share the HDD's content between Proxmox itself and the VM?

I am afraid that mounting it on Proxmox and sharing it over NFS would hamper performance, while I've read USB passthrough (and then NFS share to Proxmox) is not reliable. Would the performance hit be that high over NFS for a gigabit network?

Thanks!

 

[Dunuin]

Hi,
I have attached a large HDD over USB3.0 to my Proxmox node. I use it to store backups of VMs and containers.

However, I also would like to run many media-heavy Docker containers in a VM (Plex, family photos, Nextcloud, etc) and would have to store the media on that said HDD. What is the best way in terms of setup and performance to share the HDD's content between Proxmox itself and the VM?

I am afraid that mounting it on Proxmox and sharing it over NFS would hamper performance, while I've read USB passthrough (and then NFS share to Proxmox) is not reliable. Would the performance hit be that high over NFS for a gigabit network?

Thanks!

First you are not limited to Gigabit. As long as the packets don't leave your host the network speed inside your host will only be limited by the performance of your CPU. If your CPU is fast enough your bridges and virtual virtio NICs will do way more like 10Gbit so your CPU or USB-HDD performance should be the bottleneck and not the NFS.

With VMs you can't bind-mount folders so your only options would be to use NFS/SMB as a network share or virtual disks. I would use NFS because that way more than one VM could access the same folders and your backups and media can share the same space so you are not wasting that much capacity.

 

[oiwruk]

@Dunuin thanks for the explanation!
Just to confirm - do I understand correctly that you are saying I should mount the HDD in Proxmox and then create NFS shares? These NFS shares would then be persistently mounted in every VM I need them in.

When I do backups (stop) of the VMs these NFS shares won't be crawled, right?

 

[Dunuin]

@Dunuin thanks for the explanation!
Just to confirm - do I understand correctly that you are saying I should mount the HDD in Proxmox and then create NFS shares? These NFS shares would then be persistently mounted in every VM I need them in.

When I do backups (stop) of the VMs these NFS shares won't be crawled, right?

Yes, mount that USB HDD on the host using fstab to for example /mnt/usbhdd. Then create a folder for your vzdump backups like /mnt/usbhdd/backup and create a storage of type "directory" with content type "vzdump" and point that to the folder /mnt/usbhdd/backup. You might need to edit the storage config file (/etc/pve/storage.cfg) and add the line is_mountpoint 1 below your newly created storage so you don't get errors that the folder isn't empty when rebooting.
Then you can create a folder like /mnt/usbhdd/nfs where you create the folder you want to share to the VMs. You could install a NFS server directly on your host but I personally would create a new privileged LXC using the debian template. In that debian LXC you could then install openmediavault (here is the install script) so you get a WebUI for managing your NFS shares and rights. That way it is also easier if you need to reinstall PVE because you just need to restore that LXC from your backups and add that line to fstab to get your NFS working again. Then you bind-mount the folder /mnt/usbhdd/nfs on your host into the LXC to something like /mnt/nfs inside your LXC and use that /mnt/nfs folder to create NFS shares.