I'm working with Veeam support on this, too. The standard Veeam PVE logs don't really give a clue about what's happening. The supporter asked me to set my log level to debug (in C:\Program Files\Veeam\Plugins\PVE\Service\appsettings.json). I've done that -- now I just need the failing startup of the Veeam worker to happen again, hopefully the logs will give more info.
Problem with worker VM start initiated by Veeam on PVE
- Thread starter Teletrend AG
- Start date
Thanks for sharing this info. I just edited the file - let's see...
And BTW: neweset Veeam B&R 12.3.1.1139 is still having the same issue.
Weve used Veeam with ESXi (6.7, 7.0 and 8.0) and its really really troublesome. Switching to PBS for our new PVE cluster made everything 100% better. The backup speed has gone up drastically. Backups just work! In Veeam we have to fiddle with the options every single week and it really takes a toll on us.
So: give PBS a try and never look back
So: give PBS a try and never look back
I opened a ticket with Veeam, and they demonstrated that they only send a POST API command to start the worker virtual machine. According to Veeam, the issue is on the Proxmox side.
Example : <== Request "Post" "https://pve-02:8006/api2/json/nodes/pve-02/qemu/119/status/start"
This request is accepted, but stucks starting the virtual machine proxmox side.
And then timeout with the message
Failed to prepare the worker pve-veeam-prod-worker02: Failed to power on the worker VM: start failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply.
Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.; W2K11 : An unknown Proxmox VE error has occurred; There are no available
workers in the cluster pve-02. Performance may be affected; Job finished with error at 3/24/2025 6:26:43 PM
Example : <== Request "Post" "https://pve-02:8006/api2/json/nodes/pve-02/qemu/119/status/start"
This request is accepted, but stucks starting the virtual machine proxmox side.
And then timeout with the message
Failed to prepare the worker pve-veeam-prod-worker02: Failed to power on the worker VM: start failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply.
Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.; W2K11 : An unknown Proxmox VE error has occurred; There are no available
workers in the cluster pve-02. Performance may be affected; Job finished with error at 3/24/2025 6:26:43 PM
I'm just wondering why the Veeam worker test never had failed so far (and I've tried it almost every time the backup failed).
Well, yes. I run my worker with 4 sockets / 4 GB RAM since ever. This because I allow two concurrent tasks only. In the worker setup, if you select this amount of concurrent tasks, it automatically defaults to that number of CPU and RAM.
Hey IT Dude,
Has this solution been working for you? To me, it seems like the only thing I haven't tried yet and could solve the problem. But before I start messing around with the configs, I'd like to ask you if this really solved the issue?
Best Regards,
Enex
Another thing might have fixed the issue for me is i noticed i had 2 jobs starting at the same time. Those 2 jobs had VMs on same host so it could be that 2 simultaneously start commands were given to same worker, resulting in hang.
I just noticed that since my change to 4 CPU i also migrated a VM to another host. So now those 2 jobs start at the same time but start different workers.
I just noticed that since my change to 4 CPU i also migrated a VM to another host. So now those 2 jobs start at the same time but start different workers.
Attachments
Last edited:
Ya so far it works solid. The worker just stays on so it never misses a beat.
Honestly, the config file is just opening it in notepad and changing a false to a true, and hit save. So you won't mess with much.
When I applied the 12.3.1.1139 patch it resets the config as well. So, keep that in mind in the future.
Last edited:
Thank you for the reply. I'll check it out and will update you guys if it'll work for my environments.
Update: Since changing the config a few days ago, it seems to work. I have updated before changing the config, therefore I do not know if it would have overwritten it.