VM create/move fails on NetApp shared LVM storage with qcow2 disk

B

bpedrant

New Member
Sep 26, 2025
2
0
1
Hello all,

We have a very strange and frustrating issue right now.
On one of our 4-node clusters, we have a NetApp provide, 20TB NVMe over IP, LVM disk.

This setup was working fine until we performed updates+rebooting while our backup software was still performing its job (creating snaps, etc.). Little miscommunication here. Anyway, the VG was reporting the metadata was corrupt. We fought with it for a while, and decided to wipe it, remove the storage from the cluster, recreate the volume on the NetApp, create it back on the PVE cluster. All is good, right?

The storage shows up on each node, with 20TB of free space.
I CAN create a new VM with a 32GB 'RAW' disk.
I CANNOT create a new VM with a 32GB 'qcow2' disk. It gives a failure:

"Rounding up size to full physical extent <32.01 GiB
Logical volume "vm-44444-disk-0.qcow2" created.
Formatting '/dev/pvensll3na1/vm-44444-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
Logical volume "vm-44444-disk-0.qcow2" successfully removed.
TASK ERROR: unable to create VM 44444 - unable to create image: qemu-img: /dev/pvensll3na1/vm-44444-disk-0.qcow2: Failed to clear the new image's first sector: Invalid argument
"

Same error happens when I try to 'Move Storage' from one disk to the new NetApp volume.

Some other info:
- I am logged in as root (to rule out permissions)
- I tried on multiple nodes
- I have rebooted nodes
- Fully up to date on patches
- We are testing CEPH on these nodes as well. They are working perfectly.

Thanks!
Brian
 
Sorry, no solution or actual help from my side.
bpedrant said:
TASK ERROR: unable to create VM 44444 - unable to create image: qemu-img: /dev/pvensll3na1/vm-44444-disk-0.qcow2: Failed to clear the new image's first sector: Invalid argument
Click to expand...
I get the exact same error on a brand new test-installation of TrueNAS/SCALE (released version). And as you already noted "RAW" does work on the same device.

I am just playing around, nothing productive. That's why I did not open a bug report under https://bugzilla.proxmox.com/enter_bug.cgi?product=pve
 
recreate the volume on the NetApp
Click to expand...
Are you sure this is created the same way as before? I found in NetApp documentation [0][1] that there are certain setting needed for compatibility, but I am also not sure if this is related here.

[0] https://docs.netapp.com/us-en/ontap/san-admin/io-misalignments-properly-aligned-luns-concept.html
[1] https://kb.netapp.com/on-prem/ontap/da/SAN/SAN-KBs/How_to_identify_unaligned_IO_on_LUNs
[2] https://gitlab.com/qemu-project/qemu/-/issues/3127
 
FYI, I have this same error running PVE 9.1.1. The only way I've been able to overcome it is to downgrade pve-qemu-kvm (10.1.2-3) to (10.0.2-4)


Here's error when Ii try to create a VM:
===============================

Code: 
  Rounding up size to full physical extent 4.00 MiB

  Logical volume "vm-104-disk-0.qcow2" created.

Formatting '/dev/vg_nvme2/vm-104-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=540672 lazy_refcounts=off refcount_bits=16

  Logical volume "vm-104-disk-0.qcow2" successfully removed.

TASK ERROR: unable to create VM 104 - unable to create image: qemu-img: /dev/vg_nvme2/vm-104-disk-0.qcow2: Failed to clear the new image's first sector: Invalid argument


Here's my upgradelog from this morning:
=================================

Bash: 
Start-Date: 2025-11-19  08:00:25

Commandline: apt-get dist-upgrade

Install: netavark:amd64 (1.14.0-2, automatic), containernetworking-plugins:amd64 (1.1.1+ds1-3+b17, automatic), golang-github-containers-image:amd64 (5.34.2-1, automatic), proxmox-kernel-6.17.2-1-pve-signed:amd64 (6.17.2-1, automatic), python3-cffi-backend:amd64 (1.17.1-3, automatic), python3-importlib-resources:amd64 (6.5.2-1, automatic), python3-pefile:amd64 (2024.8.26-2.1, automatic), isc-dhcp-common:amd64 (4.4.3-P1-8, automatic), proxmox-kernel-6.17:amd64 (6.17.2-1, automatic), isc-dhcp-client:amd64 (4.4.3-P1-8, automatic), aardvark-dns:amd64 (1.14.0-3, automatic), skopeo:amd64 (1.18.0+ds1-1+b5, automatic), libgpgme11t64:amd64 (1.24.2-3, automatic), python3-bcrypt:amd64 (4.2.0-2.1+b1, automatic), python3-virt-firmware:amd64 (24.11-2, automatic), gpg-wks-client:amd64 (2.4.7-21+b3, automatic), golang-github-containers-common:amd64 (0.62.2+ds1-2, automatic), python3-cryptography:amd64 (43.0.0-3, automatic)

Upgrade: libpam-wtmpdb:amd64 (0.73.0-3, 0.73.0-3+deb13u1), pve-docs:amd64 (9.0.8, 9.1.1), pve-edk2-firmware-ovmf:amd64 (4.2025.05-1, 4.2025.05-2), console-setup:amd64 (1.240, 1.242~deb13u1), proxmox-default-kernel:amd64 (2.0.0, 2.0.1), dhcpcd-base:amd64 (1:10.1.0-11, 1:10.1.0-11+deb13u1), proxmox-widget-toolkit:amd64 (5.0.6, 5.1.2), libpve-rs-perl:amd64 (0.10.10, 0.11.3), libldb2:amd64 (2:2.11.0+samba4.22.4+dfsg-1~deb13u1, 2:2.11.0+samba4.22.6+dfsg-0+deb13u1), pve-qemu-kvm:amd64 (10.0.2-4, 10.1.2-3), libpve-cluster-api-perl:amd64 (9.0.6, 9.0.7), pve-edk2-firmware-legacy:amd64 (4.2025.05-1, 4.2025.05-2), pve-ha-manager:amd64 (5.0.5, 5.0.8), libpve-apiclient-perl:amd64 (3.4.0, 3.4.2), swtpm-libs:amd64 (0.8.0+pve2, 0.8.0+pve3), swtpm-tools:amd64 (0.8.0+pve2, 0.8.0+pve3), libpve-storage-perl:amd64 (9.0.14, 9.0.18), openssl-provider-legacy:amd64 (3.5.1-1+deb13u1, 3.5.4-1~deb13u1), libwbclient0:amd64 (2:4.22.4+dfsg-1~deb13u1, 2:4.22.6+dfsg-0+deb13u1), swtpm:amd64 (0.8.0+pve2, 0.8.0+pve3), libxml2:amd64 (2.12.7+dfsg+really2.9.14-2.1+deb13u1, 2.12.7+dfsg+really2.9.14-2.1+deb13u2), pve-yew-mobile-i18n:amd64 (3.6.1, 3.6.2), pve-cluster:amd64 (9.0.6, 9.0.7), console-setup-linux:amd64 (1.240, 1.242~deb13u1), libcurl3t64-gnutls:amd64 (8.14.1-2, 8.14.1-2+deb13u2), proxmox-ve:amd64 (9.0.0, 9.1.0), lxc-pve:amd64 (6.0.5-1, 6.0.5-3), proxmox-backup-file-restore:amd64 (4.0.19-1, 4.0.20-1), virtiofsd:amd64 (1.13.2-1, 1.13.2-1+deb13u1), pve-xtermjs:amd64 (5.5.0-2, 5.5.0-3), qemu-server:amd64 (9.0.24, 9.0.30), libpve-access-control:amd64 (9.0.3, 9.0.4), libsmbclient0:amd64 (2:4.22.4+dfsg-1~deb13u1, 2:4.22.6+dfsg-0+deb13u1), pve-container:amd64 (6.0.13, 6.0.18), pve-i18n:amd64 (3.6.1, 3.6.2), base-files:amd64 (13.8+deb13u1, 13.8+deb13u2), libtdb1:amd64 (2:1.4.13+samba4.22.4+dfsg-1~deb13u1, 2:1.4.13+samba4.22.6+dfsg-0+deb13u1), libcurl4t64:amd64 (8.14.1-2, 8.14.1-2+deb13u2), proxmox-backup-client:amd64 (4.0.19-1, 4.0.20-1), libpve-network-api-perl:amd64 (1.1.8, 1.2.3), distro-info-data:amd64 (0.66, 0.66+deb13u1), libtevent0t64:amd64 (2:0.16.2+samba4.22.4+dfsg-1~deb13u1, 2:0.16.2+samba4.22.6+dfsg-0+deb13u1), smbclient:amd64 (2:4.22.4+dfsg-1~deb13u1, 2:4.22.6+dfsg-0+deb13u1), libwtmpdb0:amd64 (0.73.0-3, 0.73.0-3+deb13u1), proxmox-firewall:amd64 (1.2.0, 1.2.1), pve-manager:amd64 (9.0.11, 9.1.1), libpve-common-perl:amd64 (9.0.13, 9.0.15), libpve-network-perl:amd64 (1.1.8, 1.2.3), samba-libs:amd64 (2:4.22.4+dfsg-1~deb13u1, 2:4.22.6+dfsg-0+deb13u1), libpve-notify-perl:amd64 (9.0.6, 9.0.7), keyboard-configuration:amd64 (1.240, 1.242~deb13u1), libssl3t64:amd64 (3.5.1-1+deb13u1, 3.5.4-1~deb13u1), wtmpdb:amd64 (0.73.0-3, 0.73.0-3+deb13u1), samba-common:amd64 (2:4.22.4+dfsg-1~deb13u1, 2:4.22.6+dfsg-0+deb13u1), pve-yew-mobile-gui:amd64 (0.6.3, 0.6.4), curl:amd64 (8.14.1-2, 8.14.1-2+deb13u2), libtalloc2:amd64 (2:2.4.3+samba4.22.4+dfsg-1~deb13u1, 2:2.4.3+samba4.22.6+dfsg-0+deb13u1), pve-edk2-firmware:amd64 (4.2025.05-1, 4.2025.05-2), postfix:amd64 (3.10.4-1~deb13u1, 3.10.5-1~deb13u1), openssl:amd64 (3.5.1-1+deb13u1, 3.5.4-1~deb13u1), proxmox-termproxy:amd64 (2.0.2, 2.0.3), libpve-cluster-perl:amd64 (9.0.6, 9.0.7)

End-Date: 2025-11-19  08:01:44

root@proxmoxve:~#

root@proxmoxve:~#


And here is how I was able to overcome the error condition ( by rolling back to pve-qemu-kvm 10.0.2-4 ):
======================================================================================

Bash: 
root@proxmoxve:~#

root@proxmoxve:~# apt install pve-qemu-kvm=10.0.2-4

The following packages were automatically installed and are no longer required:

  proxmox-kernel-6.14.11-1-pve-signed  proxmox-kernel-6.14.11-3-pve-signed  proxmox-kernel-6.14.8-2-pve-signed  proxmox-rrd-migration-tool

Use 'apt autoremove' to remove them.


DOWNGRADING:

  pve-qemu-kvm


Summary:

  Upgrading: 0, Installing: 0, Downgrading: 1, Removing: 0, Not Upgrading: 0

  Download size: 32.2 MB

  Freed space: 3,809 kB


Continue? [Y/n] y

Get:1 https://enterprise.proxmox.com/debian/pve trixie/pve-enterprise amd64 pve-qemu-kvm amd64 10.0.2-4 [32.2 MB]

Fetched 32.2 MB in 3s (11.8 MB/s) 

dpkg: warning: downgrading pve-qemu-kvm (10.1.2-3) to (10.0.2-4)

(Reading database ... 94738 files and directories currently installed.)

Preparing to unpack .../pve-qemu-kvm_10.0.2-4_amd64.deb ...

Unpacking pve-qemu-kvm (10.0.2-4) over (10.1.2-3) ...

Setting up pve-qemu-kvm (10.0.2-4) ...

Processing triggers for man-db (2.13.1-1) ...

root@proxmoxve:~#


Once I roll back pve-qemu-kvm to 10.0.2-4, I am able to create the VM without any disk creation errors using the same parameters I'd specified before. Here's the successful creation status.
==========================================================================================================================

Code: 
  Rounding up size to full physical extent 4.00 MiB
  Logical volume "vm-104-disk-0.qcow2" created.
Formatting '/dev/vg_nvme1/vm-104-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=540672 lazy_refcounts=off refcount_bits=16
transferred 0.0 B of 528.0 KiB (0.00%)
transferred 528.0 KiB of 528.0 KiB (100.00%)
transferred 528.0 KiB of 528.0 KiB (100.00%)
INFO: reading raw edk2 varstore from /usr/share/pve-edk2-firmware//OVMF_VARS_4M.ms.fd
INFO: var store range: 0x64 -> 0x40000
efidisk0: successfully created disk 'nvme1:vm-104-disk-0.qcow2,efitype=4m,ms-cert=2023,pre-enrolled-keys=1,size=528K'
  Rounding up size to full physical extent <100.02 GiB
  Logical volume "vm-104-disk-1.qcow2" created.
Formatting '/dev/vg_nvme1/vm-104-disk-1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=107374182400 lazy_refcounts=off refcount_bits=16
scsi0: successfully created disk 'nvme1:vm-104-disk-1.qcow2,cache=writeback,iothread=1,size=100G'
TASK OK
 
Last edited:
  • Like
Reactions: waltar
UdoB said:
I get the exact same error
Click to expand...
With the latest updates the error is gone. I can successfully create snapshots now :-)
 
UdoB said:
With the latest updates the error is gone. I can successfully create snapshots now :-)
Click to expand...
What version of 'pve-qemu-kvm' do you have installed now?

(In the enterprise channel, I still see 10.1.2-4, which I believe has the bug.)

-Brian
 
bpedrant said:
What version of 'pve-qemu-kvm' do you have installed now?
Click to expand...
Sorry, I have no access to my subscribed hosts at 9 p.m. ;-)

My homelab/non-sub has
Code: 
~# apt policy pve-qemu-kvm
pve-qemu-kvm:
  Installed: 10.1.2-4
  Candidate: 10.1.2-4
 
You must log in or register to reply here.
Top Bottom