VM on rpool

[alesema]

Hi everyone,

I have a home server with four Samsung SSD 980 PRO 1TB drives. I installed Proxmox VE and set up rpool as a mirror using two of the drives. This gives me a 1TB rpool. My question is: Is it okay to create VMs on rpool? I’ve read that it’s not recommended, but I’d like to understand why and hear your thoughts.

Here are the potential downsides I’ve come across:

• Risk of filling the root partition: Storing VMs on rpool could lead to running out of space, which might affect the stability of the Proxmox host.
• Complicates management and backups: Mixing system files and VM data in the same pool can make it harder to manage and back up data effectively.
• Risk of system corruption: If something goes wrong with the VMs, it could potentially impact the entire system.

I’ve also seen recommendations to use rpool for VMs only in test or small environments. But what about production environments? Is it ever acceptable to use rpool for VMs in a more serious setup?

I’d really appreciate hearing from the community and experienced users. What are your opinions or best practices for this situation?

Thanks in advance for any advice or insights!

 

[Kingneutron]

Best practice (nearly always) is to separate OS + Data -- if you ever have to reinstall, the PVE ISO is going to wipe the target disk(s) for boot/root.

You don't specify what your use-case is, but I would not put anything besides a small, non-critical vm or LXC on rpool unless it was homelab or a recovery/utility environment. And still, remember that having backups is essential.

Having a "too-large" rpool isn't the worst thing bc it should last longer with wear levelling. But you should be able to downsize rpool to 2x512GB SSD mirror if you want, and re-use the 1TBs for extra space in the VM pool by adding another mirror column.

https://github.com/kneutron/ansites...replace-zfs-mirror-boot-disks-with-smaller.sh

 

[UdoB]

I am with @Kingneutron - my dayjob's professional cluster definitely separates OS and VMs. Maintenance is easier this way

For my homelab I need to be more flexible as a lot of those small Mini-PCs just have not enough connectors for NVMe and/or SATA to build multiple pools - possible with multiple vdevs and all of them mirrored.

So I have just a single "rpool" and on some nodes an additional pool, depending on the specific use cases - e.g. for a fileserver.

 

[leesteken]

The Proxmox installation (rpool) often has different requirements (fine to be slow but durable like a HDD) than the VMs need for their storage (high IOPS, like an enterprise SSD), so it's not strange to separate them.
EDIT: If you only have just drives for one kind of storage, it's also find to put both on the rpool (or LVM).

 

[alesema]

Thank you all for your suggestions. I took your advice and purchased two additional 500 GB SATA drives. Now, the rpool is set up on these disks. I’m also planning to create two 1 TB mirrors: one for virtual machines and another for a file server.