Virtual Maschine corrupt after failed snapshot

[Henrik Gießel]

Hello,

i have an Issue where the creation of a snapshot failed, leading to a corrupting databases on this machine.
We are currently recoverying the database. We also want to know why the snapshot failed, because its the 2nd time within several weeks, that we have corrupted machines in two different environments. Basically we cant trust the snapshot function.

Here the current Cluster:
-------------------------------------------------------------------------
proxmox-ve: 5.2-2 (running kernel: 4.15.18-1-pve)
pve-manager: 5.2-6 (running version: 5.2-6/bcd5f008)
pve-kernel-4.15: 5.2-4
pve-kernel-4.15.18-1-pve: 4.15.18-17
pve-kernel-4.13.8-2-pve: 4.13.8-28
pve-kernel-4.10.17-3-pve: 4.10.17-23
pve-kernel-4.4.83-1-pve: 4.4.83-96
pve-kernel-4.2.8-1-pve: 4.2.8-41
pve-kernel-3.10.0-10-pve: 3.10.0-34
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-37
libpve-guest-common-perl: 2.0-17
libpve-http-server-perl: 2.0-9
libpve-storage-perl: 5.0-24
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-3
lxcfs: 3.0.0-1
novnc-pve: 1.0.0-2
proxmox-widget-toolkit: 1.0-19
pve-cluster: 5.0-29
pve-container: 2.0-24
pve-docs: 5.2-5
pve-firewall: 3.0-13
pve-firmware: 2.0-5
pve-ha-manager: 2.0-5
pve-i18n: 1.0-6
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.2-1
pve-xtermjs: 1.0-5
qemu-server: 5.0-30
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3

---------

An here is the log from the failed task:

Formatting '/var/lib/vz/images/106/vm-106-state-RO_user_for_BMD.raw', fmt=raw size=17704157184
VM 106 not running
snapshot create failed: starting cleanup
TASK ERROR: VM 106 qmp command 'snapshot-drive' failed - client closed connection


Does anyone has an idea how to fix/avoid the problem in the future?

 

[LnxBil]

According to https://pve.proxmox.com/wiki/Storage, you cannot create snapshots on RAW files, only on raw devices like LVM/LVM-thin, ZVOL and QCOW2.

 

[Henrik Gießel]

I dont know why PVE is using a raw disk. The said VM has following config where no raw-disks are included:

#Firebird Datenbank f%C3%BCr blp Formica Software
#
#CentOS 7.2 mit Firebird SuperClassic Server V2.5 aus EPEL
#
#IP%3A 172.16.4.7
balloon: 0
bootdisk: virtio0
cores: 2
ide2: none,media=cdrom
memory: 8192
name: firebird25
net0: virtio=AA:49:4F:59:5D:48,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
protection: 1
smbios1: uuid=42b7d700-c55f-4534-b0b0-c97d6a914eba
sockets: 2
virtio0: local:106/vm-106-disk-1.qcow2,format=qcow2,size=20G
virtio1: local:106/vm-106-disk-2.qcow2,format=qcow2,size=100


Also we are using snapshots to backup the vms every night and it works for years.
This is the storage-config - The said VM is located on "local":

dir: local
path /var/lib/vz
content iso,images,rootdir,vztmpl
maxfiles 0

nfs: NAS2
export /volume1/vm-backup
path /mnt/pve/NAS2
server 172.16.4.21
content backup
maxfiles 1
options vers=3