Proxmox 4.0 ZFS disk configuration with L2Arc + Slog. 2nd opinion

[elurex]

Its Ubuntu Kenel based, so I cant see a reason why it can run in a VM. The issues is as far as I know, clonezilla cant live backup an OS yet. We actually keep an SSD ready with a "base install" of Proxmox ready to go. We have all the config files backing up on each host via Rsync. Once we get the new SSD in, we simply drop the config files in place, reboot and add the host to cluster.

if you install pve4 on zfs as root fs, you can do live backup with zfs snapshot and then use zfs send/recv or zfssync. It's better and faster than rsync.

To take full advantage of zfs, you really need more RAM. and RAM to zil is 1:1 so your zil should be 32 gb , and RAM to L2arc is 1:2 and if you only use 4gb as zfs arc, then, you can never use more than 16gb for l2arc.

Enable lz4, it actually brings more performance and adds very few wo for the cpu.

Sent from my Nexus 6 using Tapatalk

 

[Erk]

You might be loading up the E3 a bit. I know they like to limit them to 32GB and a lot of the support chips will only drive at SATA II speeds so watch out for that. I like to give ZFS 1GB of ARC per VM which sounds like it would be getting tight on your machine. If it were me I would be trying to reduce the number of VMs and spread them across two servers instead. L2arc and ZIL are important if you are going to drive a machine hard, however ARC RAM is more important than both.

You might also look into what's creating your process load, I have an E3 with 7 Linux VMs on it, and it only goes above 1.0 when there is a vzdump or some other compression related activity going on.

 

[sigxcpu]

You cannot compare ARC to slog (not ZIL) stating that "ARC RAM" is more important. Even if you have a huge amount of RAM, your sync writes will be written in sync (with sync=standard). Either this is done in a special area in the pool or on your slog (external device for ZIL).

That is why it is recommended to attach a SSD for slog. Your mechanical drives simply cannot cope with a medium-high level of sync writes (e.g. database logs, /var/mail) so they should go to a fast random-write device until they can be commited by txg_sync (every 5 seconds by default).
If there is no failure and you gracefully shutdown your server, the slog device is never read. In case of failure (e.g. kernel crash, power loss), on the next boot, the uncommited *sync* transactions will be replayed from the slog device.

 

[Erk]

Most applications are have asynchronous writes, which is the default. If you host some big databases they will do synchronous writes. In a memory constrained system like in the OP. L2ARC will significantly increase the amount of RAM required by about 200bytes per record, so you have to not go overboard. The 940GB of L2ARC in the OP could be a serious RAM problem. The record size will vary depending on the application, but in a worst case scenario where the records are small, say 1 x 512byte sector each, and you have 200 bytes of RAM used per record for the L2ARC headers, then 940GB L2ARC would need 367GB of RAM. Hopefully the average record size is more like 8k (16 sectors) which would bring it down to around 23GB of RAM required to support the 940GB L2ARC.

 

[sigxcpu]

1. I've though that I've set the record straight on the 940GB L2ARC in post #5, so you just repeat that to argue with me?
2. There is no 512 bytes/record. Usually Linux filesystems have 4K blocks, so that will be the minimum record size in the pool.
3. The L2ARC header size varies too (see #12)

I've never said that you don't need ARC for L2ARC. I simply stated that you cannot say that ARC is more important than a log device, because you compare apples with oranges.