Hello,
I can confirm the same behaviour reported earlier in this thread, on a different cluster, with additional diagnostic steps that rule out a "config not applied" or "service not restarted" explanation.
Environment
- Proxmox VE: 9.2.4
- Corosync: 3.1.10-pve2
- Transport: knet
- 5-node cluster, Corosync/Live Migration on a dedicated VLAN (bond0.121), separate from VM and storage traffic
Configuration
totem {
version: 2
config_version: X
cluster_name: ...
ip_dscp: ef
...
}
I also tested ip_dscp: 46 and ip_dscp: 0 (to rule out a symbolic-name parsing issue) — same result in all cases.
Step 1 — Config propagation via pmxcfs
The file is managed at /etc/pve/corosync.conf (replicated by pmxcfs to /etc/corosync/corosync.conf on all nodes). Editing it alone triggers an automatic hot-reload attempt from pmxcfs, which correctly logs:
Modified entry 'totem.ip_dscp' in corosync.conf cannot be changed at run-time
This confirms ip_dscp requires a full process restart, as documented.
Step 2 — Explicit cold restart, confirmed clean
To make sure the parameter was actually loaded fresh (not just reloaded), I ran systemctl restart corosync and checked the full journal for that specific PID transition:
Stopping corosync.service...
[MAIN ] Corosync Cluster Engine exiting normally
Stopped corosync.service.
Starting corosync.service...
[MAIN ] Corosync Cluster Engine starting up
[TOTEM ] Initializing transport (Kronosnet).
[TOTEM ] totemknet initialized
...
[QUORUM] Members[5]: 1 2 3 4 5
[MAIN ] Completed service synchronization, ready to provide service.
No cannot be changed at run-time warning appears in this cold-start sequence — this was a genuine fresh process start with a new PID, and the cluster regained full quorum normally afterwards (pvecm status confirmed 5/5 members quorate).
Step 3 — Capture after confirmed cold restart
Bash:
tcpdump -v -i bond0.121 udp portrange 5404-5412 -c 5
IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 108)
192.168.88.2.5405 > 192.168.88.5.5405: UDP, length 80
tos stays at 0x10 (DSCP 4) regardless of whether ip_dscp was set to ef, 46, or 0 — the value never changes, which suggests it isn't reading the config at all on the knet code path, rather than applying a wrong value.
Workaround in use
Since the native option doesn't seem to take effect with knet, I've marked the traffic at the netfilter layer instead, which works reliably:
Bash:
nft add table inet mangle
nft add chain inet mangle output { type filter hook output priority mangle \; }
nft add rule inet mangle output ip daddr 192.168.88.0/24 udp dport 5404-5412 ip dscp set ef
This confirms the DSCP marking itself isn't blocked anywhere downstream (switches, bonding, VLAN subinterfaces) — packets do leave with tos 0xb8 once marked this way, so the issue appears isolated to Corosync/knet not applying totem.ip_dscp to the socket.
Happy to provide further logs (journalctl -u corosync full startup, corosync-cfgtool -s, or anything else) if useful for tracking this down. Given this reproduces on two independent clusters with different hardware, it does look like a genuine bug rather than an environment-specific issue.
