Using Proxmox with ZFS

[ohmer]

Hello,

I just reinstalled my testing PVE box with ZFS instead of ext4/LVM because of (very) low LXC IO performance. My test box have a single SATA HDD and a small SSD for caching (not setuped yet). This seem a lot better now, but I have some questions coming from a ZFS noob.

When I installed Proxmox, there was no settings for ZFS w/o using RAID. So, I installed it with RAID0 with only one device. Is it fine? Do you see any potential problems with this setup?

ZFS is pretty new for me and I still have a lot to learn about it. My primary concern is about RAM use. Should I limit the RAM ZFS can use with zfs_arc_max? If my node need RAM for my containers, will ZFS reduce itself its ARC cache or my node will miss memory?

On my production server, I will have an hardware RAID controller with battery backup setupped with RAID10. I read this is better to use the RAID functionality of ZFS instead of using the RAID capability of my controller. If I just want to use basic ZFS features like snapshot and subvolume, is it ok to use ZFS on top of my RAID hardware controller? I would prefer that because hdd swap is more easy with hw raid where there is a little LED indicating what drive to replace (this seem dumb, but it's very handy when the server is in a remote datacenter).

Thanks!

 

[wolfgang]

When I installed Proxmox, there was no settings for ZFS w/o using RAID. So, I installed it with RAID0 with only one device. Is it fine? Do you see any potential problems with this setup?

This is ok and normal if you have only one drive.

If my node need RAM for my containers, will ZFS reduce itself its ARC cache or my node will miss memory?

This is a max cache usage but zfs but you need a minimum of 4 GB Ram + 1 GB per 1TB Disk.
Yes it will if possible give Ram back.

ZFS and HW Raid controller do not work together. Use an HBA controller or flash you raid on HBA mode.
You can lose data in worse case if you use this setup.

 

[ohmer]

Thanks for the answer!

ZFS and HW Raid controller do not work together. Use an HBA controller or flash you raid on HBA mode.
You can lose data in worse case if you use this setup.

Then, ZFS is not suitable for us. I have no problem using ext4/LVM, but as you can see in this forum, there is a lot of reports of slow IO performance (because of barrier=1 I think). A job that take 8min19sec with LXC/ZFS take 22min25sec with LXC/ext4. I wouild appreciate a fix or at least a workaround for that issue. I don't think I can promote proxmox to production with poor performance like that.

 

[fireon]

For ZFS you need really right hardware, hdd's and controller. Then it's work much more better an every HW Raid. Supermicro is ZFS hot only an example. But we use ZFS also with cheap dellservers and have no problem, best performance. We use Enterprise HDD and Enterprise SSD and 10GBbit.