[SOLVED] Best practice for Truenas virtualized on Proxmox

[TErxleben]

You should always pass-through a controller, never disks.

Background, while old still valid: https://www.truenas.com/community/r...guide-to-not-completely-losing-your-data.212/

Oops,

could it be problematic to set up a PVE with your favorite file system, ZFS, and then double-dub it in a VM running on it?

As a "solution," you'd simply pass through an entire controller instead of individual disks? Even the latter is a shady setup and merely a shady workaround.

You can't argue away the inherent performance loss of ZFS; you can only minimize it with dirty "tricks" and all their resulting consequences.

Doesn't sound very trustworthy.

P.S.:
Wouldn't it perhaps be smarter to use LVM/ext4 as the base on the PVE and only run the VMs with ZFS when needed?

 

[UdoB]

Doesn't sound very trustworthy.

I did not recommend it. But if I would have to create a virtual TrueNAS (I don't) then this is a requirement...

 

[TErxleben]

I did not recommend it. But if I would have to create a virtual TrueNAS (I don't) then this is a requirement...

Well, your signature and our previous conversations suggest something different.

It's rather irrelevant anyway.

But we do agree that a ZFS running on a ZFS base isn't exactly a performance marvel, right? Even though ZFS offers many undeniable advantages, it's not known for its exceptional performance.

 

[UdoB]

But we do agree that a ZFS running on a ZFS base isn't exactly a performance marvel, right?

Correct. That's another thing I would never do or recommend

(( With one single exception from my personal point of view: teaching/learning/testing/debugging. I have a virtual cluster with six "hosts" = zfs-on-zfs. It works surprisingly well without production load. That's in my homelab; in my dayjob I have old, expired hardware to test with. ))

 

[Kalapahar]

You should always pass-through a controller, never disks.

Background, while old still valid: https://www.truenas.com/community/r...guide-to-not-completely-losing-your-data.212/

Thanks, I looked in to the article and since I am running of a consumer motherboard and relatively generic PCIe to SATA controller of ASM106X. So in order to achieve that, I need to pass the PCIe device to the VM.
PS: My VM runs of nvme SSD and I want to use the drives purely for storage purpose. In that case, is there alternative way instead of using NAS software.
PS2: Using ProxMox to handle the drives (2x 6TB) as RAID1, is it a good idea or bad idea?

 

[UdoB]

PS2: Using ProxMox to handle the drives (2x 6TB) as RAID1, is it a good idea or bad idea?

I cannot answer that as "it depends", sorry.

On all of my PVEs the host handles (local) storage, exclusively. In my Homelab the largest storage (in the cluster) is served by an LXC container (not: VM) to some clients. The host owns the storage and backup and management is handled there. ( --> "mount-points".)

Technically you may absolutely create a 5 (or more) TB virtual disk and give it to a VM. I am just not sure if handling that one is fun.

I really prefer to have a separate storage/NAS, be it TrueNAS or anything else. This way I can separate it, and the whole construction is simpler - avoiding one specific Single-Point-of-Failure. Complexity is my declared enemy ;-) --> https://en.wikipedia.org/wiki/KISS_principle

 

[Kalapahar]

Update:
Since I can not pass the motherboard attached SATA controller to ProxMox (as ProxMox itself is running of Motherboard SATA controller) I tried to pass through the separate ASM1061 controller and successfully created the Pass Through config file as per Google. BUT every time I tried to run the VM with attached PCIe device (ASM1061), the whole server hangs and I had to do hard reset. Upon searching more and AI, it appears that, ASM1061 controller is indeed not ideal for Pass Through.
I will look into better SATA controller for the system or at the least transfer my ProxMox installer to my nvme and try passing my Motherboard controller.
Or if my patience run out, I'll go for a bare metal server dedicated for NAS.
Thanks everyone for your valuable guides and suggestions. I learnt something new.