Tuxis launches free Proxmox Backup Server BETA service

There are chunks being marked bad for a reason unknown to me. I'm trying to find out why, but can't think of anything.

Please do remember that this is a beta product and service. Do not rely on it as your primary backup (yet!)!
 
Got this today

Code: 
INFO: starting new backup job: vzdump --all 1 --mailnotification failure --compress zstd --mailto root@i32.pl --storage pbs.tuxis.nl --mode snapshot --quiet 1
INFO: skip external VMs: 102, 104, 105, 107, 112
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2020-10-17 02:15:02
INFO: status = running
INFO: VM Name: azusa
INFO: include disk 'scsi0' 'local:100/vm-100-disk-0.qcow2' 128G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/100/2020-10-17T00:15:02Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: enabling encryption
INFO: issuing guest-agent 'fs-thaw' command
ERROR: VM 100 qmp command 'guest-fsfreeze-thaw' failed - got timeout
ERROR: VM 100 qmp command 'backup' failed - got timeout
ERROR: Backup of VM 100 failed - VM 100 qmp command 'backup' failed - got timeout
INFO: Failed at 2020-10-17 02:16:15

I assume this is related to PBS being down at the moment (as the offline backup has finished without issues before that), however the fact that it kills IO on the VM (Windows 2012 R2) is rather unexpected but not unknown it seems.

---

I also have to amend my previous statement, I had to disable PBS storage for pvestatd to come back to live despite proxmox-backup-client being long dead. Interesting, and it hasn't required that before, simply doing its job so long as no one tried to fetch list of backups from PBS.
 
Last edited:
Hi,

I still have to kill proxmox-backup-client each time Tuxis PBS is unavailable.
1603368842556.png
For example, today @ 08:00 CET

1603368898294.png

Nodes are up to date.

Thanks in advance
 
PBS is up and running since 17-10-2020. It does however have some big tasks to do, and it seems that the webservice becomes unresponsive every now and then while running those tasks (verify, garbage collect, prune). I don't know why. Maybe there is a limit on the amount of threads, maybe the process is waiting for IO.
 
  • Like
Reactions: Moayad
Yes,
Seems that verify does the job on every snapshots even if they are already in status OK.

It could takes hours with hundred (or more) of backups inside !
 
We've seen issues with verification of backups. We're not sure if they are persistant or that it is a leftover from the migration from ZFS to LVM/ext4. I would like to ask everyone to check their current backups and delete all backups that have failed verification. We need to establish if backups are still failing verification or if all is ok. So please login to https://pbs.tuxis.nl/ and clean up failed backups.
Click to expand...
Yes, current backups are still failing

chrome_2020-10-26_12-25-18.png

By default, we configure no pruning which means al your backups remain. Since the last email, we've seen some users configure pruning, but most people did not yet configure it. You can login to https://pbs.tuxis.nl/ and configure pruning using the manual.
Click to expand...
It's not required if it's used solely for PVE backups because it's being taken care by backup job.
 
@tuxis Ever since update to 0.9-4 new backups are not being verified anymore! This is essential because in case previous backup is not verified but is corrupted, all further backups will be corrupted also (client will not perform full backup again unless previous backup is missing or explicitly marked as failed)
 
Last edited:
Hmm. I see we have to manually configure it per datastore. Would be nice if we could configure a default somewhere (which is what I assumend what happened, and why I didn't configure it yet).
 
Shouldn't it be configurable by owner of the datastore, just like GC/prune schedule is? Perhaps there is new permission added?
 
eider said:
Shouldn't it be configurable by owner of the datastore, just like GC/prune schedule is? Perhaps there is new permission added?
Click to expand...

This is the case. You can see the 'Verify jobs' tab in your datastore.
 
I can not actually, just says "Forbidden (403) permission check failed".

So I can trigger manual verification, but not configure jobs.
 
I launched a for ds in `all datastores`; do proxmox-backup-manager verify $ds; done
 
tuxis said:
Cool. New bugs! :)
Click to expand...

new features actually ;)

the verify scheduling now has a lot more options in 0.9.4. in git master, there is a new permission Datastore.Verify (DatastoreAdmin has it by default) which allows verifying the whole datastore, irrespective of ownership, and allows setting up verification jobs. if a user has just Datastore.Backup, they can verify their own snapshots/groups or the whole datastore, but in the latter case the verification will be filtered and just include those groups which the user owns (so I can do a 1-click "verify everything that belongs to me in this datastore", even if I can't setup scheduled verification jobs on the datastore level).
 
  • Like
Reactions: TwiX
new features actually ;)

the verify scheduling now has a lot more options in 0.9.4. in git master, there is a new permission Datastore.Verify (DatastoreAdmin has it by default) which allows verifying the whole datastore, irrespective of ownership, and allows setting up verification jobs.
Click to expand...
Yes and no. The built binaries for proxmox-backup-server available for download are from day before Datastore.Verify was added as valid permission for DatastoreAdmin role (https://git.proxmox.com/?p=proxmox-...;hpb=b728a69e7df309eb598ad7d6c522247b92e07d5d), and PBS does not allow editing roles on its own (they're hardcoded)
 
Last edited:
  • Like
Reactions: eider
fabian said:
new features actually ;)

the verify scheduling now has a lot more options in 0.9.4. in git master, there is a new permission Datastore.Verify (DatastoreAdmin has it by default) which allows verifying the whole datastore, irrespective of ownership, and allows setting up verification jobs. if a user has just Datastore.Backup, they can verify their own snapshots/groups or the whole datastore, but in the latter case the verification will be filtered and just include those groups which the user owns (so I can do a 1-click "verify everything that belongs to me in this datastore", even if I can't setup scheduled verification jobs on the datastore level).
Click to expand...

Yes. Very cool. I do miss the overall datastore view. And the possibility to schedule the verify task via the CLI?
 
tuxis said:
Yes. Very cool. I do miss the overall datastore view. And the possibility to schedule the verify task via the CLI?
Click to expand...

the overall view might make a comeback in some fashion, and job config on the proxmox-manager CLI definitely would also be a good idea.

there are even more job related changes currently on list or not-yet-packaged, like the ability to configure sync jobs as semi-privileged users and setting an explicit backup owner for sync jobs as well.
 
  • Like
Reactions: tuxis
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back