[SOLVED] Shared Storage

[Bugbear]

Hi,

Actually I'm a beginner, which is why I wanted to ask you a basic question.
I would like to know if it's possible to connect a hard drive to the Proxmox computer, which can be accessed from several VMs at the same time.
Do you know if this is possible, and if so, how would you recommend me to implement it?

Thanks in advance

 

[Dunuin]

No, you can't do that with VMs. The idea of VMs is, that these are completely isolated and sharing a HDD would break that concept.
But you could do that with containers (LXCs).

If you just want to share a folder you could create a network share and connect that to all VMs.

 

[Bugbear]

Thanks for the clarification!

Sharing a folder would also work for me.
What does "network share" mean in concrete terms, do I set it up via the Proxmox Interface or something like NFS or SMB?

 

[DrSpeedy]

Thanks for the clarification!

Sharing a folder would also work for me.
What does "network share" mean in concrete terms, do I set it up via the Proxmox Interface or something like NFS or SMB?

Like a Samba/CIFS or an NFS share. Here's how I do mine.

My drives are a typical ext4 format on gpt. I have them mounted each to /data_drives/. On the drives there a is a directory designated for shares, such as /data_drives/<drive_name>/Shares/Media. This is then soft linked to /mnt/shares/media (sudo ln -sf ... ...). My Samba (/etc/samba/smb.conf) config then looks something like this:

### Top of file....
[Media]
  comment = Media share
  path = /mnt/shares/media
  follow symlinks = yes
  wide links = yes
  valid users = @smb-media
  browseable = yes
  read only = no
  writeable = yes
  guest ok = no
  # Enable the recycle bin
  vfs object = recycle
  recycle:repository = .recycle
  recycle:touch = Yes
  recycle:keeptree = Yes
  recycle:versions = Yes
  recycle:noversions = *.tmp,*.temp,*.o,*.obj,*.TMP,*.TEMP
  recycle:exclude = *.tmp,*.temp,*.o,*.obj,*.TMP,*.TEMP
  recycle:excludedir = /.recycle,/tmp,/temp,/TMP,/TEMP

On all VMs, I mount this share with autofs. You will also need to configure users for samba and permissions for the directories being shared/ I suggest doing group ownership (smb-media above) and doing a chmod -R g+rsw ./ from the root of your share

 

[Bugbear]

Thank you, you, both, really helped me out!

 

[voarsh]

I think it's about setting up a NFS-Server

 

[Dunuin]

Yes, SMB or NFS shares will both work. But you can't create a share using the GUI. So manual setup over CLI is needed. You could run an SMB/NFS server directly on your proxmox host to bypass all that extra virtualization or run a SMB/NFS server in a VM sharing a folder of a virtual HDD.

 

[Bugbear]

Okay, I think I'll setup a NFS server on the proxmox host.
Thank you all very much for the great ideas!

 

[voarsh]

Yes, SMB or NFS shares will both work. But you can't create a share using the GUI. So manual setup over CLI is needed. You could run an SMB/NFS server directly on your proxmox host to bypass all that extra virtualization or run a SMB/NFS server in a VM sharing a folder of a virtual HDD.

Is that even a good idea (running on the host)?

Also, have you got any links on setting up NFS server for PROXMOX?

 

[Bugbear]

Yes, I understand his thoughts on making the setup easier.

I think I'm able to find some decent references:
https://www.tecmint.com/install-nfs-server-on-ubuntu/
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-20-04
https://computingforgeeks.com/install-and-configure-nfs-server-on-ubuntu-debian/
https://wiki.archlinux.org/index.php/NFS

Since I'll be using the CLI for the setup anyway, the installationan and configuration process shouldn't differ too much from other distros, except for the packages.
Also I think I'll manage it to integrate the storage in the proxmox datacenter, if needed.

The only problem I see here is the performance: I've got a 2TB SATA-SSD with about 500 MB/s read- and writespeed.
But my Ethernet connnection is 1 Gigabit (1000 Mb/s = 125 MB/s). So it would bottleneck big file transfers and be about four times slower, wouldn't it?

 

[DrSpeedy]

Yes, I understand his thoughts on making the setup easier.

I think I'm able to find some decent references:
https://www.tecmint.com/install-nfs-server-on-ubuntu/
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-20-04
https://computingforgeeks.com/install-and-configure-nfs-server-on-ubuntu-debian/
https://wiki.archlinux.org/index.php/NFS

Since I'll be using the CLI for the setup anyway, the installationan and configuration process shouldn't differ too much from other distros, except for the packages.
Also I think I'll manage it to integrate the storage in the proxmox datacenter, if needed.

The only problem I see here is the performance: I've got a 2TB SATA-SSD with about 500 MB/s read- and writespeed.
But my Ethernet connnection is 1 Gigabit (1000 Mb/s = 125 MB/s). So it would bottleneck big file transfers and be about four times slower, wouldn't it?

Use VirtIO for the VM's network device model. They are 10Gbps. You may need to configure OVS ports instead of your standard linux bridge on the host. I do not remember if that is necessary, but it is easy none the less

 

[voarsh]

I think I'm able to find some decent references:
https://www.tecmint.com/install-nfs-server-on-ubuntu/
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-20-04
https://computingforgeeks.com/install-and-configure-nfs-server-on-ubuntu-debian/
https://wiki.archlinux.org/index.php/NFS

I've yet to check these links, but, pardon me, but the host is debian, not ubuntu? Or it doesn't matter in this case?

Looks like I've got a lot of reading to do to setup NFS-Server to share a directory across my nodes....

 

[Dunuin]

Is that even a good idea (running on the host)?

That depends. From the security standpoint it would be better to run them isolated in a VM. And if you run them on the host you get alot of more work to do if you need to reinstall proxmox because you need to do it all again.
On the other hand your network share can use datasets and needs no virtualization if you run them bare metal. So you get less overhead, because you don't need to use a virtual SCSI controller, virtual HDD and additional filesystem on that virtual HDD. So its faster and more efficient.

I've yet to check these links, but, pardon me, but the host is debian, not ubuntu? Or it doesn't matter in this case?

Proxmox and Ubuntu are based on Debian. So Ubuntu tutorials should work most of the time too.

 

[voarsh]

That depends. From the security standpoint it would be better to run them isolated in a VM. And if you run them on the host you get alot of more work to do if you need to reinstall proxmox because you need to do it all again.
On the other hand your network share can use datasets and needs no virtualization if you run them bare metal. So you get less overhead, because you don't need to use a virtual SCSI controller, virtual HDD and additional filesystem on that virtual HDD. So its faster and more efficient.

Proxmox and Ubuntu are based on Debian. So Ubuntu tutorials should work most of the time too.

Thanks, you're right.

But setting up looks like it could take less than 30 minutes?
How often do you need to reinstall proxmox?
Supposedly you could write a script that'll do it all for you afterwards.

I noticed someone commented about after restarting a client machine for the NFS, it must be re-mounted?
Anyone experienced this?

 

[Bugbear]

I noticed someone commented about after restarting a client machine for the NFS, it must be re-mounted?
Anyone experienced this?

Yes, that's absolutely normal. After shutting down your Linux Machine all drives (NFS too) are unmounted automaticly.
If you want them to be mounted automaticly at system boot you have to edit your /etc/fstab (as described in the second article).

 

[Bugbear]

Use VirtIO for the VM's network device model. They are 10Gbps. You may need to configure OVS ports instead of your standard linux bridge on the host. I do not remember if that is necessary, but it is easy none the less

I'm not sure how this should change my situation.
The entire data would nevertheless go over the local 1Gbps network and be bottlenecked there, wouldn't it?

 

[Dunuin]

You got virtual 10G NICs and these are connected to a virtual 10G bridge. As long as everything is on the same host your network should work with 10G and only gets slowed down to 1G if it leaves your machine.

 

[Bugbear]

Ah ok, so the traffic won't leave the machine as long it's the same host. Thanks.
Happy New Year!

[marked topic as SOLVED]

 

[voarsh]

I setup the export on a proxmox host, with the host IP.

On a host of another node I installed the client:
mount 192.168.168.144:/mnt/bindmounts/shared /mnt/nfs_clientshare

It just sits there with no errors.
I turned off the firewall, I am not getting any errors, it just hangs?

 

[Bugbear]

Please create a new thread for this, so that the individual topics can be a good reference point for others in the future.