LVM on iSCSI from NFS

[markusbernhard]

Hi.
I'm about to move from NFS to LVM on multipath iSCSI as shared storage in our proxmox cluster.

I understand that I need to change the disk definitions on the VM's to use async IO native instead of the default io_uring, and I need to restart the VM's to do that.

We have about 230 VM's to move, so it's not made in a day - I was wondering if it's safe to change the Async io to native and still run them on NFS until I get about to move the storage?

The NFS shares are of different kinds. Some are truenas with ZFS below, and some are plain old NAS's.

best regards
--
Markus

 

[Kingneutron]

If you want to bypass the GUI: backup everything, power down the VMs and you can try sed on the entries in /etc/pve/qemu-server/*.conf

This is the difference:


scsi0: local-lvm:vm-100-disk-0,aio=native,cache=writeback,discard=on,iothread=1,size=31G,ssd=1

scsi1: tosh10-xfs-multi:100/vm-100-disk-22.raw,backup=0,cache=writeback,discard=on,size=1G,ssd=0

.

Do some testing with 1..5 VMs and see if I/O is still good

See also:

https://github.com/kneutron/ansitest/blob/master/proxmox/proxmox-migrate-disk-storage.sh

https://github.com/kneutron/ansitest/blob/master/proxmox/bkpcrit-proxmox.sh

Lots of other good stuff in that repo ;-)

 

[bbgeek17]

Hi @markusbernhard,

You are not required to change away from io_uring when moving to LVM on iscsi.

The main downside of aio=native, when used with file based storage, it can block inline of io submission in some exceptional cases. This would show up as high latency in the guest. If you go this route, I'd recommend using an io thread.

You might find this overview helpful - https://kb.blockbridge.com/technote/proxmox-aio-vs-iouring/#proxmox-io-options

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox