Triple check if there is an IP address conflict.
If some other device grabs that IP address - or it is assigned by a DHCP server - then these effects are normal. It also is normal, that Containers and/or VMs are not affected...
PVE should have...
Hi everyone, I'd like to summarize this investigation in case someone encounters a similar issue in the future.
First of all, thank you to everyone who took the time to review the logs and suggest ideas. This was my first serious OpenZFS...
Good point!
I realized that I wrote the incorrect model, they are actually M90q G5:
So I am pretty sure they will be possible to fit with this little fella that I also did buy together with the cards:
It's a riser card that should fit...
I am not really against a short one. I say "I do not need that as it has not necessarily any advantage". ;-)
Possibly. Except that my Zabbix instance is "HA" too and has a shorter replication interval than we are discussing here :-)
Not...
Hardware-Raid is fine while Software-Raid (mdraid) is not officially supported.
Personally I prefer ZFS, if possible; you will waive some nice features without it...
A lot of people use small/tiny systems for low power consumption.
You may find that 10 GBit/s over copper adds some Watts. If possible: measure it before and after...
I set the interval to twice per 24h, which should be fine with me. It's all a question on how much one is willing to loose. If I have to go back in time 24 hours on my Home Assistant, or Hermes agent - it's not a big deal. It's more important...
Fine!
I feel the need to elaborate on my statement as it was recognized as too infrequently:
My example: I run DNS servers. (Among other mostly-static-data VMs.) DNS has "primaries" and "secondaries". To replicate the secondaries with a short...
Irrelevant, since the idea of a Proxmox mcp server is flawed by design. This is exactly what several people tried to explain to you and potential readers. Because you won't to stop using your new toy but hopefully readers will read the arguments...
HA with ZFS (let's call it Pseudo-HA) definitely works if you can live with a minimal dataloss (the default is 15 minutes, can be reduced one minute). This doesn't change the need for a dedicated corosync network though which is deficult to...
Actually I usually set the replication interval to 24h, once per night. For both my $dayjob's cluster and my homelab, and for most - but not all - VMs.
Remember: data loss from "too-long-an-interval" will only happen when a node crashes. It is...
Doch natürlich, es guckt sich die Dateien auf dem Dateisystem an und ob da sich was geändert hat. Es braucht dafür keine Features des Dateisystems (etwa zfs send/receive) außer den Standardmetadaten (also Erstellungs- und Änderungstimestamps...
@Impact Thanks for pointing that :)
In my defense:
while composing that previous post I searched for the detailed link in the first document and there was no ToC at all. Now I've had a closer look at that page and can see that this was due to...
My opinion is that if you really want a properly running cluster with HA enabled, then SFF/USFF machines are mostly a dead end. Either find mini-towers so you can have dual 10GbE, a boot mirror and a some drives for Ceph, or forgo HA and just...
In fact the pve manual recommends not to have a qdevice if you have at least three nodes and an odd number of nodes due to the way corosync handles them:
Don't cluster them but use the DatacenterManager for migrating vms between the nodes and pve-zsnc for replicating via zfs. You will need to start them manually instead of HA though. Alternatively buy two USB-NICs to have a dedicated cluster...
You dont need a qdevice for a 3 node cluster; it will just serve as a 4th vote that doesnt have any consequence since its not a tiebreaker anyway.
The lessons here are well learned, namely:
1. your cluster is only as stable as its network design...
"all_system_memory - 1 GiB" - Wow! I didn't notice that one yet.
This shows how confident the developers are, that release/shrink memory of ARC does really work nowadays :)
Though... I would not bet my cluster on that assumption...