Hi
Forgive my "Swenglish" since English is not my native language and I am from Sweden.
I have a backup job that has a problem.
I think my problems started when I got a power out but I am not sure when really.
I have an Ubuntu VM and when I start/reboot my Proxmox server physically it is trying to run a backup on that machine (201) and since the VM is not running and cannot be started is has a disk icon on it. Because of this no other backup job will run and the backup job in question is cannot be stopped. I cannot go into the VM CLI and I have tried to abort backup job on my PBS without any luck. When I start vias GUI i get invalid CRF token and if I do via qm start/delete/reboot/unlock VM from SSH I get can't lock file '/var/lock/qemu-server/lock-201.conf' - got timeout.
My PX version is 8.1.5 and PBS 3.1-4 and everything is updated.
I user a low power machine as a standalone PBS which ahs been workign perfectly.
The log for the VM backup job on my PBS says the following:
2024-03-24T22:50:35+01:00: starting new backup on datastore 'Backups' from ::ffff:192.168.10.99: "vm/201/2024-03-17T00:02:18Z"
2024-03-24T22:50:36+01:00: download 'index.json.blob' from previous backup.
2024-03-24T22:50:36+01:00: register chunks in 'drive-scsi0.img.fidx' from previous backup.
2024-03-24T22:50:36+01:00: download 'drive-scsi0.img.fidx' from previous backup.
2024-03-24T22:50:36+01:00: created new fixed index 1 ("vm/201/2024-03-17T00:02:18Z/drive-scsi0.img.fidx")
2024-03-24T22:50:36+01:00: add blob "/mnt/datastore/Backups/vm/201/2024-03-17T00:02:18Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-03-24T22:50:39+01:00: backup ended and finish failed: backup ended but finished flag is not set.
2024-03-24T22:50:39+01:00: removing unfinished backup
2024-03-24T22:50:39+01:00: TASK ERROR: backup ended but finished flag is not set.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
I have serched through internet forums and discussion to find a solution but everything I've tried turned out to be not working.
A little recap of what I have tried so far:
* Reboot PBS and Proxmos server - backup job keeps starting up again.
* Kill -9 PID - reactivates backup job so same status, vzdump process also
* Restart Corosync and pve processes - same status as above.
* Abort backupjob from within PBS - cannot be done
* Delete/unlock/start etc the whole VM - cannot be done since it is locked
* vzdump --stop - "stopping backup process 2957 failed" and so on.
I am now out of ideas and I cannot find a solution to this and so far I have tried all I could from what answers I could find online.
Anyone have an idea?
Would be grateful for any ideas!
Forgive my "Swenglish" since English is not my native language and I am from Sweden.
I have a backup job that has a problem.
I think my problems started when I got a power out but I am not sure when really.
I have an Ubuntu VM and when I start/reboot my Proxmox server physically it is trying to run a backup on that machine (201) and since the VM is not running and cannot be started is has a disk icon on it. Because of this no other backup job will run and the backup job in question is cannot be stopped. I cannot go into the VM CLI and I have tried to abort backup job on my PBS without any luck. When I start vias GUI i get invalid CRF token and if I do via qm start/delete/reboot/unlock VM from SSH I get can't lock file '/var/lock/qemu-server/lock-201.conf' - got timeout.
My PX version is 8.1.5 and PBS 3.1-4 and everything is updated.
I user a low power machine as a standalone PBS which ahs been workign perfectly.
The log for the VM backup job on my PBS says the following:
2024-03-24T22:50:35+01:00: starting new backup on datastore 'Backups' from ::ffff:192.168.10.99: "vm/201/2024-03-17T00:02:18Z"
2024-03-24T22:50:36+01:00: download 'index.json.blob' from previous backup.
2024-03-24T22:50:36+01:00: register chunks in 'drive-scsi0.img.fidx' from previous backup.
2024-03-24T22:50:36+01:00: download 'drive-scsi0.img.fidx' from previous backup.
2024-03-24T22:50:36+01:00: created new fixed index 1 ("vm/201/2024-03-17T00:02:18Z/drive-scsi0.img.fidx")
2024-03-24T22:50:36+01:00: add blob "/mnt/datastore/Backups/vm/201/2024-03-17T00:02:18Z/qemu-server.conf.blob" (363 bytes, comp: 363)
2024-03-24T22:50:39+01:00: backup ended and finish failed: backup ended but finished flag is not set.
2024-03-24T22:50:39+01:00: removing unfinished backup
2024-03-24T22:50:39+01:00: TASK ERROR: backup ended but finished flag is not set.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
2024-03-24T22:50:39+01:00: POST /fixed_chunk: 400 Bad Request: backup already marked as finished.
I have serched through internet forums and discussion to find a solution but everything I've tried turned out to be not working.
A little recap of what I have tried so far:
* Reboot PBS and Proxmos server - backup job keeps starting up again.
* Kill -9 PID - reactivates backup job so same status, vzdump process also
* Restart Corosync and pve processes - same status as above.
* Abort backupjob from within PBS - cannot be done
* Delete/unlock/start etc the whole VM - cannot be done since it is locked
* vzdump --stop - "stopping backup process 2957 failed" and so on.
I am now out of ideas and I cannot find a solution to this and so far I have tried all I could from what answers I could find online.
Anyone have an idea?
Would be grateful for any ideas!
