Hello all,
We have a very strange and frustrating issue right now.
On one of our 4-node clusters, we have a NetApp provide, 20TB NVMe over IP, LVM disk.
This setup was working fine until we performed updates+rebooting while our backup software was still performing its job (creating snaps, etc.). Little miscommunication here. Anyway, the VG was reporting the metadata was corrupt. We fought with it for a while, and decided to wipe it, remove the storage from the cluster, recreate the volume on the NetApp, create it back on the PVE cluster. All is good, right?
The storage shows up on each node, with 20TB of free space.
I CAN create a new VM with a 32GB 'RAW' disk.
I CANNOT create a new VM with a 32GB 'qcow2' disk. It gives a failure:
"Rounding up size to full physical extent <32.01 GiB
Logical volume "vm-44444-disk-0.qcow2" created.
Formatting '/dev/pvensll3na1/vm-44444-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
Logical volume "vm-44444-disk-0.qcow2" successfully removed.
TASK ERROR: unable to create VM 44444 - unable to create image: qemu-img: /dev/pvensll3na1/vm-44444-disk-0.qcow2: Failed to clear the new image's first sector: Invalid argument
"
Same error happens when I try to 'Move Storage' from one disk to the new NetApp volume.
Some other info:
- I am logged in as root (to rule out permissions)
- I tried on multiple nodes
- I have rebooted nodes
- Fully up to date on patches
- We are testing CEPH on these nodes as well. They are working perfectly.
Thanks!
Brian
We have a very strange and frustrating issue right now.
On one of our 4-node clusters, we have a NetApp provide, 20TB NVMe over IP, LVM disk.
This setup was working fine until we performed updates+rebooting while our backup software was still performing its job (creating snaps, etc.). Little miscommunication here. Anyway, the VG was reporting the metadata was corrupt. We fought with it for a while, and decided to wipe it, remove the storage from the cluster, recreate the volume on the NetApp, create it back on the PVE cluster. All is good, right?
The storage shows up on each node, with 20TB of free space.
I CAN create a new VM with a 32GB 'RAW' disk.
I CANNOT create a new VM with a 32GB 'qcow2' disk. It gives a failure:
"Rounding up size to full physical extent <32.01 GiB
Logical volume "vm-44444-disk-0.qcow2" created.
Formatting '/dev/pvensll3na1/vm-44444-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
Logical volume "vm-44444-disk-0.qcow2" successfully removed.
TASK ERROR: unable to create VM 44444 - unable to create image: qemu-img: /dev/pvensll3na1/vm-44444-disk-0.qcow2: Failed to clear the new image's first sector: Invalid argument
"
Same error happens when I try to 'Move Storage' from one disk to the new NetApp volume.
Some other info:
- I am logged in as root (to rule out permissions)
- I tried on multiple nodes
- I have rebooted nodes
- Fully up to date on patches
- We are testing CEPH on these nodes as well. They are working perfectly.
Thanks!
Brian