Recent content by vkhera

I think this is a special case for 2 node cluster. You can still have quorum in a 4 node cluster when one node goes away... but you do risk split brain when there is a partition of two and two. Not sure how gluster works in that situation. But basically, gluster with two nodes is a disaster...

Gluster has its very own quorum computation. It is unrelated to proxmox quorum. The latest versions of gluster allow you to use a non-data storage node to add to the quorum decision. If you have a third small node for proxmox quorum votes, then you can use that as well for gluster.

Personally I have had just the most horrible slow performance with Gluster. So much so that I decided to stick with local storage and lose the benefits of the shared storage. Also, with two nodes, you really have no safety if one of your nodes dies, and that node happens to be the first node you...

Such a live migration is actually easier with ZFS send/receive. I will hope that if/when this feature is built that zfs storage is supported.

Has setting zfs_arc_lotsfree_percent=0 caused you any problems?

The noVNC console works for me with Sierra's Safari. My PVE is still running 4.4-1.

As your screenshots show, the "local" is a directory. It just happens to live on a ZFS file system. Your "local-zfs" is a ZFS storage, which means that when you create a disk, it will create a ZFS volume to store the raw data. This is the most efficient storage you can have for this purpose...

Live migration requires a shared storage. ZFS is not shared.

So the definitive answer I have gotten is that with 2 data nodes, if the "first" node goes down, the cluster stops working, and you get this corruption because the clients don't seem to stop trying to write. You'd think that the client would get a write error and panic or something, but it does not.

When you use bonded interfaces, I do not believe that a single connection will use more than the single NIC. Everything I've read about bonded interfaces is that they spread the load across multiple clients only.

I would recommend to use the built in backup/restore process rather than trying to copy the disk and config files manually.

If you are measuring fsync's then having a cache on your RAID controller (assuming it is battery backed up and gets actually used safely) is going to make it faster. Having an SSD SLOG device on your ZFS is the way to speed up that operation in ZFS.

Is your existing vm to import stored as a backup file? PVE should just do the right thin when restoring it.

I'm running 3.5.2 as comes with PVE. From where did you get those gluster settings? I'm using the recommended "virt" settings from the gluster wiki. There the cache is disabled. I don't understand that, but that seems to be the recommended configuration.

The ZIL is *never* read unless it is replaying after a crash. You cannot lose data until your system crashes, and the only window of opportunity to lose the data is during the interval where the flush happens. After the SLOG device fails, further ZIL data is written to the pool as if that device...