Virtual Machines not able to make synchronous/DIRECT IO after update?

U

uberarchiver

Member
Proxmox Subscriber
May 18, 2024
53
12
8
I updated my PVE yesterday to the newest updates available on the 'enterprise ready' sources.

Now I am wondering what changed. ZFS is reporting literally no sync writes to the SLOG device of my ZFS pool.
Something changed other than all of my VMs.

Here are my installed versions:

Code: 
proxmox-ve: 8.3.0 (running kernel: 6.8.12-8-pve)
pve-manager: 8.3.3 (running version: 8.3.3/f157a38b211595d6)
proxmox-kernel-helper: 8.1.0
pve-kernel-5.15: 7.4-4
proxmox-kernel-6.8: 6.8.12-8
proxmox-kernel-6.8.12-8-pve-signed: 6.8.12-8
proxmox-kernel-6.8.12-6-pve-signed: 6.8.12-6
proxmox-kernel-6.8.12-4-pve-signed: 6.8.12-4
proxmox-kernel-6.8.8-4-pve-signed: 6.8.8-4
proxmox-kernel-6.8.8-2-pve-signed: 6.8.8-2
proxmox-kernel-6.5.13-6-pve-signed: 6.5.13-6
proxmox-kernel-6.5: 6.5.13-6
pve-kernel-5.15.108-1-pve: 5.15.108-1
pve-kernel-5.15.74-1-pve: 5.15.74-1
amd64-microcode: 3.20240820.1~deb12u1
ceph-fuse: 16.2.15+ds-0+deb12u1
corosync: 3.1.7-pve3
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.4
libpve-access-control: 8.2.0
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.10
libpve-cluster-perl: 8.0.10
libpve-common-perl: 8.2.9
libpve-guest-common-perl: 5.1.6
libpve-http-server-perl: 5.2.0
libpve-network-perl: 0.10.0
libpve-rs-perl: 0.9.1
libpve-storage-perl: 8.3.3
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.5.0-1
openvswitch-switch: 3.1.0-2+deb12u1
proxmox-backup-client: 3.3.2-1
proxmox-backup-file-restore: 3.3.2-2
proxmox-firewall: 0.6.0
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.3.1
proxmox-mini-journalreader: 1.4.0
proxmox-widget-toolkit: 4.3.4
pve-cluster: 8.0.10
pve-container: 5.2.3
pve-docs: 8.3.1
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.2
pve-firewall: 5.1.0
pve-firmware: 3.14-3
pve-ha-manager: 4.0.6
pve-i18n: 3.3.3
pve-qemu-kvm: 9.0.2-5
pve-xtermjs: 5.3.0-3
qemu-server: 8.3.7
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve1

Has anyone else experienced this?
 
A simple zfs set sync=always [pool/volume/whatever] on a vm did in fact start doing sync writes.
Either there has been a latent problem from before where in fact I experimented with a poolwide sync=always or then the behavior of vms has indeed changed. The amount of sync writes has dropped seriously anyways.

All my writes were not sync before but a significantly larger amount. I have databases that do like to write synchronously to the disk.

Maybe this will remain a mystery.
 
sync=always is almost always wrong, it forces *all* I/O to be treated as sync, instead of just the I/O that is requested as sync by applications for consistency reasons.
 
sync I/O is not better than async I/O - if you make everything sync, you basically lose the advantage of having an SLOG device in the first place.. (also, making everything sync on the highest layer, doesn't mean that consistency guarantees on lower layers are actually valid - if you think some layer is dropping the 'sync' property of I/O requests, you still need to fix that)
 
you can obviously ignore my advice - I am telling you that *if there is an issue related to sync IO on your setup caused by the upgrade*, setting sync to always doesn't fix it at all, it just pretends it does while hurting performance. without more information (like which version you were previously on, how your VMs are configured, how your guest OS setups look like) it will be hard to tell whether there is such an issue in the first place though.
 
And I gave him free advice. Completely off topic. But still free. As did he.

I do not understand why people very often have a predetermined view on what is the problem. .

I am a paid subscriber and I am Entitled to something by that.

Obviously sharing my problems so that someone could learn is not the way here. I need to be hammered on my head.

I do not need to write anything else here. Obviously that is not needed. So be it then.

My Advice: "Close the forum"
 
uberarchiver said:
Go inhale deep. Talk to your boss. Take a vacation.
Click to expand...
uberarchiver said:
And I gave him free advice. Completely off topic. But still free. As did he.

I do not understand why people very often have a predetermined view on what is the problem. .

I am a paid subscriber and I am Entitled to something by that.

Obviously sharing my problems so that someone could learn is not the way here. I need to be hammered on my head.

I do not need to write anything else here. Obviously that is not needed. So be it then.

My Advice: "Close the forum"
Click to expand...

uberarchiver said:
I am a paid subscriber and I am Entitled to something by that.
Click to expand...

Imo, what you are in the first place, is a snooty and unpleasant person; to say it friendly...
Apart from that, your supposed entitlement has to go here:
https://my.proxmox.com
 
  • Like
Reactions: RolandK
You must log in or register to reply here.
Top Bottom