After backups run, vm's come back up with read-only filesystems

cdukes

Active Member
Sep 11, 2015
88
1
28
Raleigh, NC
www.logzilla.net
Code:
# pveversion
pve-manager/6.1-3/37248ce6 (running kernel: 5.3.10-1-pve)
I can't seem to figure out why this is happening, but every time our backups run, the vm's that were backed up come back up with read-only filesystems and I have to restore form the backup to get them working again.
This is only happening on cloud-init based images, not on full os images.
Also, not sure if it's relevant, these systems are part of an HA cluster. So maybe when the backups run, the cluster manager is starting the image before backup completes? (just a theory)
 
Last edited:

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
2,132
325
103
South Tyrol/Italy
Weird. That's only happening with cloudinit makes it even weirder. Do you use the QEMU guest agent in the VMs?

Also, nit sure if it's relevant, these systems are part of an HA cluster. So maybe when the backups run, the cluster manager is starting the image before backup completes? (just a theory)
Good for mentioning, but the VM is locked during the backup, so even if the HA manager tried to stop or start it, that operation would fail due to the lock. The only way the lock can be removed is once the backup was finished or if the node fails and gets fenced, then the HA removes some locks after recovering the VM/CT to another node, to ensure it can be started there again. As the whole node where the backup was made is fenced and the VM is started newly this should not cause this.


What backup method do you use? What's the underlying storage (for backup and VM)?
 

cdukes

Active Member
Sep 11, 2015
88
1
28
Raleigh, NC
www.logzilla.net
Weird. That's only happening with cloudinit makes it even weirder. Do you use the QEMU guest agent in the VMs?
Yes, all vm's have it.

What backup method do you use? What's the underlying storage (for backup and VM)?
LZO compression and Snapshot mode

What's the underlying storage (for backup and VM)?
Underlying storage for the VM's are ZFS on 4k block NVMe drives. (see attached screenshots)
 

Attachments

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!