snapshot problem since 3.4

holest · Feb 25, 2015

Hello!

I have some problem yesterday i have installed, 3.4.
Now i can't snapshot my VM-s, it's start "formating" but never ends.

syslog:

Code:

Feb 25 12:24:01 p2 pvedaemon[390575]: <root@pam> starting task UPID:p2:000611BD:008A5E77:54EDB0D1:qmsnapshot:119:root@pam:
Feb 25 12:24:01 p2 pvedaemon[397757]: <root@pam> snapshot VM 119: pregui
Feb 25 12:24:36 p2 pvedaemon[393333]: got timeout
Feb 25 12:24:54 p2 pvedaemon[390575]: worker exit
Feb 25 12:24:54 p2 pvedaemon[3631]: worker 390575 finished

but in the proxmox web interface it's still "working"

can someone tell me where shuold i look, for the solution of this problem?

Holest

ps.:

now

Code:

[COLOR=#000000][FONT=tahoma]TASK ERROR: command '/bin/nc -l -p 5900 -w 10 -c '/usr/bin/ssh -T -o BatchMode=yes 10.3.1.2 /usr/sbin/qm vncproxy 119 2>/dev/null'' failed: exit code 255
[/FONT][/COLOR]

spirit · Feb 25, 2015

can you post your vm config file ?

(to see what kind of storage you use).

holest · Feb 25, 2015

Code:

root@p2:~# cat /etc/pve/qemu-server/119.conf
bootdisk: virtio0
cores: 2
ide2: nas:iso/ubuntu-14.04.2-server-amd64.iso,media=cdrom
lock: snapshot
memory: 3000
name: superbuntu.nemzet.hu
net0: virtio=46:3B:46:AB:2D:D1,bridge=vmbr0
numa: 0
ostype: l26
smbios1: uuid=5a9f1b1e-34a1-48a6-a960-c43fa9ca0ece
sockets: 2
virtio0: prox2:119/vm-119-disk-1.qcow2,format=qcow2,size=50G


[pregui]
bootdisk: virtio0
cores: 2
ide2: nas:iso/ubuntu-14.04.2-server-amd64.iso,media=cdrom
machine: pc-i440fx-2.1
memory: 3000
name: superbuntu.nemzet.hu
net0: virtio=46:3B:46:AB:2D:D1,bridge=vmbr0
numa: 0
ostype: l26
smbios1: uuid=5a9f1b1e-34a1-48a6-a960-c43fa9ca0ece
snapstate: prepare
snaptime: 1424863441
sockets: 2
virtio0: prox2:119/vm-119-disk-1.qcow2,format=qcow2,size=50G
vmstate: prox2:119/vm-119-state-pregui.raw
root@p2:~#

holest · Feb 25, 2015

now spanpshotting finished with this:

Code:

[COLOR=#000000][FONT=tahoma]Formatting '/mnt/pve/prox2/images/119/vm-119-state-pregui.raw', fmt=raw size=6815744000 [/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]snapshot create failed: starting cleanup[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]TASK ERROR: query-savevm returned status 'failed'[/FONT][/COLOR]

holest · Feb 26, 2015

still have this problem.
can someone tell me, how to start proxmox with old (3.3) kernel?

holest · Feb 26, 2015

installed a fresh proxmox server (Proxmox VE 3.4 ISO Installer) on a test machine, with "default" settings, local storage

made a test VM and stuck at snapshotting with this computer too

Code:

[COLOR=#000000][FONT=tahoma]Formatting '/var/lib/vz/images/100/vm-100-state-teszt.raw', fmt=raw size=1598029824 [/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]snapshot create failed: starting cleanup[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]TASK ERROR: query-savevm returned status 'failed'[/FONT][/COLOR]

none else have problem with snapshotting?

silverado · Feb 26, 2015

Yes, I have this problem with snapshotting, too. Snapshots seem to work with Windows VMs but fail with Linux (Ubuntu) VMs. Only when I choose not to include RAM Ubuntu snapshots work. For me this problem started with version 3.3 and still exists in 3.4.

I thought there might be some problem with my configuration because I always install Proxmox manually on top of a fully encrypted Debian. But when you say that this also happens with a "normally" installed Proxmox there could be a general problem with snapshots.

holest · Feb 27, 2015

silverado said:
I thought there might be some problem with my configuration because I always install Proxmox manually on top of a fully encrypted Debian. But when you say that this also happens with a "normally" installed Proxmox there could be a general problem with snapshots.

yes, i can confirm that, the problem is with a pure installation. i made it again on an another machine, and i cant snapshot my debian vm

should i reinstall my nodes with the 3.3 iso? or what should i do?

lwidomski · Mar 12, 2015

We have the same problem with Windows machines (after upgrade from 3.3 to 3.4)

holest · Mar 12, 2015

Can someone confirm to me, if i change my subscription from community to basic, someone from "Proxmox Server Solutions GmbH" will answer for my questions?
And what happens if they can't solve my ticket? Refund the price of my licence?

spirit · Mar 12, 2015

lwidomski said:
We have the same problem with Windows machines (after upgrade from 3.3 to 3.4)

Only when you snapshot with memory state ? Do you have problem with live migration with theses windows vm ?

#pveversion -v ?

lwidomski · Mar 12, 2015

Yes. Error exists only with "Include RAM" options enabled

I can't test live migration on those pve, because this is only one node

Code:

root@pve3:~# pveversion -v
proxmox-ve-2.6.32: 3.3-147 (running kernel: 2.6.32-37-pve)
pve-manager: 3.4-1 (running version: 3.4-1/3f2d890e)
pve-kernel-2.6.32-32-pve: 2.6.32-136
pve-kernel-2.6.32-37-pve: 2.6.32-147
pve-kernel-2.6.32-34-pve: 2.6.32-140
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-2
pve-cluster: 3.0-16
qemu-server: 3.3-20
pve-firmware: 1.1-3
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-31
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.1-12
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

spirit · Mar 12, 2015

lwidomski said:

Yes. Error exists only with "Include RAM" options enabled

I can't test live migration on those pve, because this is only one node

Code:

root@pve3:~# pveversion -v
proxmox-ve-2.6.32: 3.3-147 (running kernel: 2.6.32-37-pve)
pve-manager: 3.4-1 (running version: 3.4-1/3f2d890e)
pve-kernel-2.6.32-32-pve: 2.6.32-136
pve-kernel-2.6.32-37-pve: 2.6.32-147
pve-kernel-2.6.32-34-pve: 2.6.32-140
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-2
pve-cluster: 3.0-16
qemu-server: 3.3-20
pve-firmware: 1.1-3
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-31
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.1-12
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

can you post your vmid.conf ?

do you have the same problem, if you don't choose windows as ostype ?
(maybe try linux for test ?)

I know they was a problem related to windows timer, but it was in qemu 2.2.0 (fixed in qemu 2.2.1)

(note that I can't reproduce it with qemu 2.1 and windows ostype)

lwidomski · Mar 13, 2015

Code:

boot: cdn
bootdisk: virtio0
cores: 1
ide2: none,media=cdrom
memory: 1024
name: WXP-test
net0: virtio=5A:4C:5F:F3:F2:4A,bridge=vmbr0
ostype: wxp
sockets: 1
virtio0: u00_proxmox:104/vm-104-disk-1.qcow2,format=qcow2,size=20G

With linux box (Oracle Linux) everything is OK.

Today i also test some other 3 Windows machines.
Wen i start snapshot it's failed, then i run command unlock on each one.

Code:

qm unlock 104 (etc.)

And after that snapshot with RAM start works.
Strenght, because the machine wasn't locked.

Holest can You try this and confirm my solutions?

Regards,
Widomski Łukasz

spirit · Mar 13, 2015

lwidomski said:
Code:

boot: cdn bootdisk: virtio0 cores: 1 ide2: none,media=cdrom memory: 1024 name: WXP-test net0: virtio=5A:4C:5F:F3:F2:4A,bridge=vmbr0 ostype: wxp sockets: 1 virtio0: u00_proxmox:104/vm-104-disk-1.qcow2,format=qcow2,size=20G

With linux box (Oracle Linux) everything is OK.

Today i also test some other 3 Windows machines.
Wen i start snapshot it's failed, then i run command unlock on each one.

Code:

qm unlock 104 (etc.)

And after that snapshot with RAM start works.
Strenght, because the machine wasn't locked.

Holest can You try this and confirm my solutions?

Regards,
Widomski Łukasz

Hi,
I'm finally able to reproduce it. (qemu 2.1 && qemu 2.2),
with a win2012r2 guest

Take a first snapshot,
it's hanging like you on.

then unlock, cancel the job,

and indeed taking a new snapshot is working.

It'll try to dig a little more today

spirit · Mar 13, 2015

I'm unable to reproduce it with qemu 2.2.1.

I'll discuss this bug on pve-devel mailing list.

(Edit : it's works fine with qemu 2.2 too)

spirit · Mar 13, 2015

Proxmox will provide an update to qemu 2.2 next week, which should fix this problem.

(If you can't wait, you can already get it in pvetest repository)

holest · Mar 13, 2015

Thanks spirit

silverado · May 11, 2015

Unfortunately for me the problem persists. As described above I have installed Proxmox manually on top of a fully encrypted Debian. This setup has worked flawlessly with versions lower than Proxmox 3.3. Since then my snapshots failed with the symptoms described above. The upgrade to qemu 2.2 didn't help. I can reproduce the error on multiple machines/installs.

For troubleshooting purposes I installed Proxmox normally and unencrypted on a different machine. I created several virtual machines there and could snapshot them without any problems.

Then I backupped the VMs and restored them on the encrypted Proxmox machine. I still could snapshot them there!

Now I created a new virtual machine on the encrypted Proxmox machine and with this virtual machine the snapshots fail again.

So for me it seems obvious that on an encrypted Proxmox machine something goes wrong with the creation of virtual machines that makes it impossible to take snapshots.

Any ideas?

spirit · May 12, 2015

silverado said:
VMs and restored them on the encrypted Proxmox machine. I still could snapshot them there!

Now I created a new virtual machine on the encrypted Proxmox machine and with this virtual machine the snapshots fail again.

can you post vmid.conf of restored vm and new vm ?

snapshot problem since 3.4

Member

Distinguished Member

Member

Member

Member

Member

Renowned Member

Member

New Member

Member

Distinguished Member

New Member

Distinguished Member

New Member

Distinguished Member

Distinguished Member

Distinguished Member

Member

Renowned Member

Distinguished Member