Storage Upgrades: Data Partitions

[MicahB]

I have been googling and have yet to find a solution to this issue.

History: I am running Proxmox 5.2-1 and am migrating from XenServer Community due to the removal of dynamic memory allocation. In XenServer my NextCloud VM booted from one drive and then saved all the data on a 4 drive raid mounted as the data pool for my cloud storage.

The "Server":
AMD Opteron (4 Core)
8GB DDR3 RAM
MSI Radeon 6850
1x 240GB SSD - ProxMox & Boot
4x Sata 384GB - future nextCloud data
Dual 1Gb NICs

Intent:
Setup a NextCloud container / VM with an 8G "boot" drive, connect the 4 x SATA 384GB drives as a VM exclusive data partition.

What I tried:
I formatted and mounted the 4 drives as ext4 in the vmserver (Proxmox) at /mnt/DataPoolx, then created a mergerFS "drive" at /mnt/DataPool. I successfully mounted the various drives and added them to /etc/fstab.

Since then I have tried various commands (found in the Proxmox wiki, forum posts, & documentation) and can get the partition to show up in Proxmox Storage View (adding a directory in storage), but then am having issues getting it to show in the client container, additionally, the "DataPool" mount is reporting the same available space as the local LVM and local-thin LVM instead of the ~1.4TB it should have.

The Question(s):
Is there a way to do this so that the data can be directly accessed/recovered if the Container/VM/Proxmox server die?

• ie. boot from a recovery disk, mount the partition, and copy the data to another drive / device...
• Yes, a backup solution is planned, but simplicity of design and application is preferred.

How do I go about making the space of these 4 drives available to my nextcloud container/VM?

Most Recent Attempts:

When following: https://pve.proxmox.com/wiki/Physical_disk_to_kvm the qm set <id> /dev/disk/by-id/<device id> command returns: "Configuration file 'nodes/moxy/qemu-server/<id>.conf' does not exist"

Bonus points: A future proof solution (like a ProxMox hosted globally accessible & scalable data pool)!!

 

[frenchfries]

I'm not totally sure what you're trying to do so I'm gonna take two stabs at figuring it out. It's obviously unsupported though.

1. In the storage tab in the GUI simply add the partition/pool you want as a directory. Set the content to whatever you want it to be eg. disk image, iso etc. Then you should be able to create images on it.
2. If you just want it as raw storage available to your VMs, passthrough like, then you probably need to use NFS or CIFS and have the client OS mount the CIFS storage as a filesystem.

I suppose another possibility is to install docker directly on the host and just run nextcloud from there and you can simply bind mount the data directories. Pretty sure the proxmox guys would hate this though.

You could also move away from mergerFS and snapraid to either ZFS raidz1 or to a simple mdadm array.

 

[MicahB]

I have 5 Hard Drives in my "server." There is no other storage anywhere on my network, especially not a CIFS/NFS/SAN box, all the other computers on the network are Windows 10 workstations.

In the Server there are:
1 drive (240GB SSD) = Proxmox Boot (LVM & lvm-thin) volume
4 drives are "data" drives and I would like for them to be used exclusively for file storage by VM 104 - A NextCloud container / VM - Storing mp3's, docx, xlsx, and other office type documents. I do not want to store backups, disk images, ISO's, Containers, or any other VM related files on these drives.

Ideally, these drives would be formatted and used in a way that they are quickly recoverable should the system die. (I plan to work on a backup solution, but this has to be up and working first).

Due to the low amount of system RAM (8GB), I was under the impression that ZFS was not a good choice as they recommend an extra 1GB RAM per 1 TB of storage.

"Obviously unsupported"... Please explain... Additionally, what methods of VM level exclusive storage are supported that don't require external network storage and are quickly recoverable?

When I add the partition to the Storage View should it show the same amount of available space as the lvm & lvm-thin?

To be clear, I am not trying to do anything that isn't supported, I just find that the documentation for varying use cases is very vague or non-existent. The documentation I am finding (seems) to be lacking directions that would help a user to connect new/existing hardware available to the host to a client.

I have connected 4 new hard drives, formatted and mounted them at the host level... I am now trying to connect this new storage capacity to the VM they were intended for and am seemingly unable to do so without adding it to the Host's data lvm...which by nature of the LVM allows anything on the host to use the storage (unless I get into disk quotas).

 

[frenchfries]

So you just want to have the VM use each disk as an individual filesystem rather than having an image?

Well that's easy then, simply passthrough the raw disks to the VM and manage it all there. The VM will have normal access to them so you can partition/combine them through mergerFS etc./whatever.

If your VM dies for whatever reason you could simply mount the disks on the local host to copy off whatever you needed to.

When I say not supported, it means just that. It may work but you wont get any love from the proxmox guys for it.

 

[petitminion]

hey,

I want to do the same thing here but with a encrypted disk.

"simply passthrough the raw disks to the VM " : How do you do that with lxc ? ^^

 

[MicahB]

Thanks to frenchfries, I was able to resolve my quest via passthrough by adding something like this to the config for the VM:

virtio1: /dev/disk/by-id/ata-ST3400832NS_<uuid>-part1,backup=0,size=381552M

That said, looking at it 2 years later there doesn't seem to be a way to mark this as resolved?