API Internal Server Error [500], but requests complete?

Jan 8, 2021
12
0
1
33
Hi everyone,

We're in need for some help here please.

In short, some of the API requests return with 500, but complete anyway.

We're on proxmox-ve: 6.2-1

There are no errors in /var/log/syslog in fact here is one of the successful attempts:
Jan 7 15:47:45 xxx.xxx.xxx pvedaemon[42603]: <root@pam> update VM 0000: resize --disk scsi1 --size +700G

API endpoint request details:
/nodes/xxx.xxx.xxx/qemu/0000/resize


The issue is intermittent, but causes lots of trouble in this particular case as users keep on resizing vm storage having impression that it failed.

It would be very helpful to know how else can we debug this? I did not find any log files specific to API and there isn't anything helpful in /var/log/syslog nor /var/log/pve/tasks

Thanks!
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
8,158
1,594
164
the resize likely takes longer than the API request timeout, but as it has already been forwarded to the VM/storage layer, it will still proceed. what kind of storage is the disk on? how is the load situation when requests "fail"?

maybe we need to provide an async API endpoint as well (that spawns a real task worker that you can then poll for status/completion)...
 
  • Like
Reactions: Moayad
Jan 8, 2021
12
0
1
33
That makes sense, to answer your questions

1) Load isn't any different between when requests are successful and when they fail and general node load on average is not higher than 49%
2) ZFS

Yeah, think async endpoints would solve this. Any work around you could suggest in the interim?
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
8,158
1,594
164
no workaround unfortunately (except doing those resizes on the CLI where there is no timeout, but I assume your customers don't get that level of access to the PVE system ;))

I just updated https://bugzilla.proxmox.com/show_bug.cgi?id=2315 , feel free to subscribe there to get progress updates! shouldn't be too hard to fix..
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!