proxmox as file server

[Nik2016]

Hello. So I'm in the process of trying to use proxmox to build a file/media server. Current plan is to hook up my media disks to the server, mount it to the container with ubuntu server on it and share across the network with samba. I'm new to proxmox, so not really sure how this will turn out. Will this work? Is there another method I should use instead?

Secondly my other issue is that my media drives are external usb 3, so need to mount them in ubuntu. I started following this guide: https://pve.proxmox.com/wiki/USB_Devices_in_Virtual_Machines, but seems I don't need to pass over anything as I can already see the uuid with lsusb in linux server. Can I just edit fstab to auto mount the drive? Thanks for the help =)

 

[valeech]

You won't need to pass through the USB devices because you are using a container. The container should have access to the drives.

You have the basic idea. Now you just need to mount the drives in the container by using a mount point. You will need to add the USB drives as storage at the data center level. I would recommend that you first add the USB drives as LVM-Thin pool(s) and then add them as storage points at the data center level. This will allow you to take snapshots if needed.

Once you have the drives set as a mount point in the container, you just need to setup samba to share out that mount point.

 

[Nik2016]

You won't need to pass through the USB devices because you are using a container. The container should have access to the drives.

You have the basic idea. Now you just need to mount the drives in the container by using a mount point. You will need to add the USB drives as storage at the data center level. I would recommend that you first add the USB drives as LVM-Thin pool(s) and then add them as storage points at the data center level. This will allow you to take snapshots if needed.

Once you have the drives set as a mount point in the container, you just need to setup samba to share out that mount point.

Thank you very much for your reply Valeech. Unfortunatly I completely missed a big detail in my original post. The usb drives are already ntfs. I'm gonna guess this makes them impossible to mount at the data centre level. Is that the case? I've tried to research this and ntfs-3g might be a work around. Anyone know?

 

[valeech]

Yes, you will need ntfs-3g in order to mount those partitions. Once they are mounted on the proxmox system, you can add them at the data center level as directories and your vm's/containers will see them.

And if you intend on leaving those drives on the proxmox server, I would highly recommend you move the data elsewhere, re-partition those drives and move the data back. ntfs-3g is a user space driver and you will not get great performance. At least, that has been my experience.

 

[Nik2016]

Thanks to your help I think I've just about managed to pull this off. Tested with a spare old usb drive (not risking my usb data drive until fully tested) these are the commands i used: -

fdisk /dev/sdc (new partition, primary, make first on disk, change partition type, change to lvm partition)
pvcreate /dev/sdc1
vgcreate Data-Storage /dev/sdc1
lvcreate -L 460G -n media-data Data-Store
lvconvert --type thin-pool Data-Storage/media-data

Any problems with the above? As far as I can tell it worked and was able to add it as a lvm-thin drive at data centre level and mounted in my ubuntu container. I also got samba working and was able to connect from windows. But there was an issue. When I made transfers to and from the drive the transfer speed dropped greatly, the system looked like it hanged, then suddenly picked up and ram usage went up massively. Transfers finished, but ram usage didn't drop down afterwards so I had to reboot the container. I'm assuming it's some sort of cache issue, maybe linked to slow read/write on the old drive perhaps? Question being could this happen to my usb 3 drive to during transfers?

Also if I was to use only a single usb drive in end for data sharing couldn't I use a directory instead of lvm-thin drive? I read that I could also backup my vm's to it, rather then just container images with the lvm-thin. What do you think? any advantages/disadvantages to this?

 

[valeech]

The commands look good.

You can use a directory on the USB drive as a mount point in the container(s). You will lose the ability to snapshot containers as a directory type filesystem is not a support filesystem for snapshotting. So just be aware of that. You could also use a LVM storage without making it thin. Then you could store other data types there as well, like VM backups and what not.

Not sure why the transfer speeds dropped drastically.It could be a number of things. How does your smb.conf look?

 

[Nik2016]

The commands look good.

You can use a directory on the USB drive as a mount point in the container(s). You will lose the ability to snapshot containers as a directory type filesystem is not a support filesystem for snapshotting. So just be aware of that. You could also use a LVM storage without making it thin. Then you could store other data types there as well, like VM backups and what not.

Not sure why the transfer speeds dropped drastically.It could be a number of things. How does your smb.conf look?

Ah, I read the wiki page wrong then. I thought it said directories could do both. Is the only advantage of lvm-thin storage that you can expand it with additional drives? using LVM storage does sound useful and convient having container and vm's backed up on my data disk.

The transfer speed problem doesn't seem to be samba related I think. Before I even setup the samba config I tested the drive with an ftp transfer of a large file (3-4GB) to the mount point directory. Same issue happend. The smb.conf is a bit messy as I was in a rush to just test it could work. Ran testparm and everything checked out. Thanks for the help again Valeech =)

 

[valeech]

LVM thin simply allows thin provisioning of volumes. This can have a drawback in that you can over-provision.

And I mistyped. LVM does not support snapshotting. See here: https://pve.proxmox.com/wiki/LVM2#LVM_vs_LVM-Thin

Always glad to help. Good luck!

 

[Nik2016]

I just ran another test, this time with an internal hard drive added as LVM disk in system and mounted to my ubuntu container. Started the ubuntu container and did a 3-4 GB ftp transfer to the mount point directory and memory usage (1.5 GB assigned) hit 100%. Didn't go back down after finishing the transfer. However I deleted the transfered file and memory usage went back down to 10%. Is this normal behavior for proxmox/containers? Always assumed that memory would only be used during transfer and released once data was copied to drive. Can anyone explain this so I know whats going on here? Thanks.