Possible bug after upgrading to 7.2: VM freeze if backing up large disks

Oh what a mess, upgraded yesterday one Ceph-Cluster and running into the same issue :(
Any Updates?

Code:
May 19 04:05:26 c01-n01 pvescheduler[4055863]: INFO: Starting Backup of VM 2002 (qemu)
May 19 04:05:59 c01-n01 pvescheduler[4055863]: INFO: Finished Backup of VM 2002 (00:00:33)
May 19 04:05:59 c01-n01 pvescheduler[4055863]: INFO: Starting Backup of VM 2003 (qemu)
May 19 04:06:38 c01-n01 pve-ha-lrm[4107092]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:06:38 c01-n01 pve-ha-lrm[4107092]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:06:43 c01-n01 pvestatd[3368]: status update time (7.812 seconds)
May 19 04:07:08 c01-n01 pve-ha-lrm[4110911]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:08 c01-n01 pve-ha-lrm[4110911]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:11 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:07:13 c01-n01 pvestatd[3368]: status update time (8.734 seconds)
May 19 04:07:21 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:07:24 c01-n01 pvestatd[3368]: status update time (8.978 seconds)
May 19 04:07:31 c01-n01 pvestatd[3368]: status update time (5.340 seconds)
May 19 04:07:48 c01-n01 pve-ha-lrm[4116129]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:48 c01-n01 pve-ha-lrm[4116129]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:07:54 c01-n01 pvestatd[3368]: status update time (8.808 seconds)
May 19 04:08:02 c01-n01 pvestatd[3368]: status update time (7.443 seconds)
May 19 04:08:13 c01-n01 pvestatd[3368]: status update time (8.428 seconds)
May 19 04:08:24 c01-n01 pvestatd[3368]: status update time (8.650 seconds)
May 19 04:08:31 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:08:34 c01-n01 pvestatd[3368]: status update time (8.806 seconds)
May 19 04:08:38 c01-n01 pve-ha-lrm[4122562]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:08:38 c01-n01 pve-ha-lrm[4122562]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:08:41 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:08:44 c01-n01 pvestatd[3368]: status update time (8.764 seconds)
May 19 04:08:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:08:54 c01-n01 pvestatd[3368]: status update time (8.900 seconds)
May 19 04:09:02 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:09:05 c01-n01 pvestatd[3368]: status update time (9.993 seconds)
May 19 04:09:09 c01-n01 pve-ha-lrm[4126654]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:09:09 c01-n01 pve-ha-lrm[4126654]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:09:12 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:09:15 c01-n01 pvestatd[3368]: status update time (10.657 seconds)
May 19 04:09:21 c01-n01 pvestatd[3368]: status update time (6.086 seconds)
May 19 04:09:31 c01-n01 pvestatd[3368]: status update time (5.248 seconds)
May 19 04:09:41 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:09:43 c01-n01 pvestatd[3368]: status update time (8.921 seconds)
May 19 04:09:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:09:54 c01-n01 pvestatd[3368]: status update time (8.768 seconds)
May 19 04:10:03 c01-n01 pvestatd[3368]: status update time (7.962 seconds)
May 19 04:10:13 c01-n01 pvestatd[3368]: status update time (7.421 seconds)
May 19 04:10:18 c01-n01 pve-ha-lrm[4135728]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:18 c01-n01 pve-ha-lrm[4135728]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:21 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:10:23 c01-n01 pvestatd[3368]: status update time (8.911 seconds)
May 19 04:10:32 c01-n01 pvestatd[3368]: status update time (6.804 seconds)
May 19 04:10:48 c01-n01 pve-ha-lrm[4139619]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:48 c01-n01 pve-ha-lrm[4139619]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:10:53 c01-n01 pvestatd[3368]: status update time (8.643 seconds)
May 19 04:11:02 c01-n01 pvestatd[3368]: status update time (7.291 seconds)
May 19 04:11:14 c01-n01 pvestatd[3368]: status update time (8.307 seconds)
May 19 04:11:18 c01-n01 pve-ha-lrm[4143537]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:18 c01-n01 pve-ha-lrm[4143537]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:21 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:11:23 c01-n01 pvestatd[3368]: status update time (8.659 seconds)
May 19 04:11:31 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:11:34 c01-n01 pvestatd[3368]: status update time (8.847 seconds)
May 19 04:11:38 c01-n01 pve-ha-lrm[4146088]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:38 c01-n01 pve-ha-lrm[4146088]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:41 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:11:44 c01-n01 pvestatd[3368]: status update time (8.949 seconds)
May 19 04:11:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:11:54 c01-n01 pvestatd[3368]: status update time (8.920 seconds)
May 19 04:11:58 c01-n01 pve-ha-lrm[4148672]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:58 c01-n01 pve-ha-lrm[4148672]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:12:01 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:12:04 c01-n01 pvestatd[3368]: status update time (8.935 seconds)
May 19 04:12:14 c01-n01 pvestatd[3368]: status update time (9.011 seconds)
May 19 04:12:21 c01-n01 pvestatd[3368]: status update time (6.167 seconds)
May 19 04:12:23 c01-n01 pvescheduler[4055863]: INFO: Finished Backup of VM 2003 (00:06:24)
 
Oh what a mess, upgraded yesterday one Ceph-Cluster and running into the same issue :(
Any Updates?

Code:
May 19 04:05:26 c01-n01 pvescheduler[4055863]: INFO: Starting Backup of VM 2002 (qemu)
May 19 04:05:59 c01-n01 pvescheduler[4055863]: INFO: Finished Backup of VM 2002 (00:00:33)
May 19 04:05:59 c01-n01 pvescheduler[4055863]: INFO: Starting Backup of VM 2003 (qemu)
May 19 04:06:38 c01-n01 pve-ha-lrm[4107092]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:06:38 c01-n01 pve-ha-lrm[4107092]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:06:43 c01-n01 pvestatd[3368]: status update time (7.812 seconds)
May 19 04:07:08 c01-n01 pve-ha-lrm[4110911]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:08 c01-n01 pve-ha-lrm[4110911]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:11 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:07:13 c01-n01 pvestatd[3368]: status update time (8.734 seconds)
May 19 04:07:21 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:07:24 c01-n01 pvestatd[3368]: status update time (8.978 seconds)
May 19 04:07:31 c01-n01 pvestatd[3368]: status update time (5.340 seconds)
May 19 04:07:48 c01-n01 pve-ha-lrm[4116129]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:48 c01-n01 pve-ha-lrm[4116129]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:07:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:07:54 c01-n01 pvestatd[3368]: status update time (8.808 seconds)
May 19 04:08:02 c01-n01 pvestatd[3368]: status update time (7.443 seconds)
May 19 04:08:13 c01-n01 pvestatd[3368]: status update time (8.428 seconds)
May 19 04:08:24 c01-n01 pvestatd[3368]: status update time (8.650 seconds)
May 19 04:08:31 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:08:34 c01-n01 pvestatd[3368]: status update time (8.806 seconds)
May 19 04:08:38 c01-n01 pve-ha-lrm[4122562]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:08:38 c01-n01 pve-ha-lrm[4122562]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:08:41 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:08:44 c01-n01 pvestatd[3368]: status update time (8.764 seconds)
May 19 04:08:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:08:54 c01-n01 pvestatd[3368]: status update time (8.900 seconds)
May 19 04:09:02 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:09:05 c01-n01 pvestatd[3368]: status update time (9.993 seconds)
May 19 04:09:09 c01-n01 pve-ha-lrm[4126654]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:09:09 c01-n01 pve-ha-lrm[4126654]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:09:12 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:09:15 c01-n01 pvestatd[3368]: status update time (10.657 seconds)
May 19 04:09:21 c01-n01 pvestatd[3368]: status update time (6.086 seconds)
May 19 04:09:31 c01-n01 pvestatd[3368]: status update time (5.248 seconds)
May 19 04:09:41 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:09:43 c01-n01 pvestatd[3368]: status update time (8.921 seconds)
May 19 04:09:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:09:54 c01-n01 pvestatd[3368]: status update time (8.768 seconds)
May 19 04:10:03 c01-n01 pvestatd[3368]: status update time (7.962 seconds)
May 19 04:10:13 c01-n01 pvestatd[3368]: status update time (7.421 seconds)
May 19 04:10:18 c01-n01 pve-ha-lrm[4135728]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:18 c01-n01 pve-ha-lrm[4135728]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:21 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:10:23 c01-n01 pvestatd[3368]: status update time (8.911 seconds)
May 19 04:10:32 c01-n01 pvestatd[3368]: status update time (6.804 seconds)
May 19 04:10:48 c01-n01 pve-ha-lrm[4139619]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:48 c01-n01 pve-ha-lrm[4139619]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:10:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:10:53 c01-n01 pvestatd[3368]: status update time (8.643 seconds)
May 19 04:11:02 c01-n01 pvestatd[3368]: status update time (7.291 seconds)
May 19 04:11:14 c01-n01 pvestatd[3368]: status update time (8.307 seconds)
May 19 04:11:18 c01-n01 pve-ha-lrm[4143537]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:18 c01-n01 pve-ha-lrm[4143537]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:21 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:11:23 c01-n01 pvestatd[3368]: status update time (8.659 seconds)
May 19 04:11:31 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - got timeout
May 19 04:11:34 c01-n01 pvestatd[3368]: status update time (8.847 seconds)
May 19 04:11:38 c01-n01 pve-ha-lrm[4146088]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:38 c01-n01 pve-ha-lrm[4146088]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:41 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:11:44 c01-n01 pvestatd[3368]: status update time (8.949 seconds)
May 19 04:11:51 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:11:54 c01-n01 pvestatd[3368]: status update time (8.920 seconds)
May 19 04:11:58 c01-n01 pve-ha-lrm[4148672]: VM 2003 qmp command failed - VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:11:58 c01-n01 pve-ha-lrm[4148672]: VM 2003 qmp command 'query-status' failed - got timeout
May 19 04:12:01 c01-n01 pvestatd[3368]: VM 2003 qmp command failed - VM 2003 qmp command 'query-proxmox-support' failed - unable to connect to VM 2003 qmp socket - timeout after 31 retries
May 19 04:12:04 c01-n01 pvestatd[3368]: status update time (8.935 seconds)
May 19 04:12:14 c01-n01 pvestatd[3368]: status update time (9.011 seconds)
May 19 04:12:21 c01-n01 pvestatd[3368]: status update time (6.167 seconds)
May 19 04:12:23 c01-n01 pvescheduler[4055863]: INFO: Finished Backup of VM 2003 (00:06:24)
Are you using KRBD? Did you start/stop or migrate VMs after changing to KRBD if not already?
 
  • Like
Reactions: fiona
Thank You Fabian,
sadly i can test the older pve-qemu-kvm coming weekend at earliest (service time window), since all vms need to be rebooted and i lose dirty map again.
If i understand it correctly, todays updates contain a regression patch for the "bdrv_co_block_status" issue,
No, that patch is not applied yet. And if setting krbd to 1 (and stop/start or migrating the VMs) didn't help, the patch is very unlikely to help (as that code is not used when krbd is enabled AFAICT).
i will try this one first, if it doesn't help, test the older pve-qemu-kvm versions as you suggested. i will provide all the requested logs and infos in a new thread, if issue persists.
Again, thanks!
 
  • Like
Reactions: itNGO
No KRBD.
VMs are migrated (it´s a Production Cluster)
Please switch to KRBD and migrate VMs to another node and back and check if issue persists.
 
Is switching to KRBD "live" possible? There is only one Ceph-Pool (3 Nodes)
Yes, it is not "really" live, cause you have to Migrate or STOP/START every VM to make it switch from RBD to KRBD. Did this yesterday on 7.2-4 for about 50 VMs without any problem. But to be sure... make it in the off-times... better save than sorry....
 
STOP/START the VMs is not possible, so:

1.) enabling KRBD on the Ceph-Pool while all VMs are running
2.) MIGRATING all VMs from one node to another node

Is this correct?
(sorry for this but it is a very critical 24/7 Cluster, so no downtime possible)
 
Not that it does really matter for me.... but a question.... when it is such an important system with critical 24/7 demand.... why did you upgrade it so soon after release of 7.2-x? :oops: It is not unusual that in the first weeks after a release there a minor and sometimes moderate bugs or issues, even in the so called Subscription Repository...

But yes... you can do this while all VMs are running. You can for additional test, create small VM... install an OS... switch to KRBD Live-Migrate it and check if all goes well. Then you can proceed with the other ones....
 
The package pve-qemu-kvm=6.2.0-8 which reverts the problematic change for QEMU's RBD driver, is now available on the pvetest repository.

You can test it by:
  1. Adding/Enabling the pvetest repository (can also be done in the Repostiories panel in the UI).
  2. apt update && apt install pve-qemu-kvm
  3. Removing/disabling the pvetest repository again.
  4. VMs need to be stopped/started or migrated to pick up the new version.
 
  • Like
Reactions: ITT
Not that it does really matter for me.... but a question.... when it is such an important system with critical 24/7 demand.... why did you upgrade it so soon after release of 7.2-x? :oops: It is not unusual that in the first weeks after a release there a minor and sometimes moderate bugs or issues, even in the so called Subscription Repository...

But yes... you can do this while all VMs are running. You can for additional test, create small VM... install an OS... switch to KRBD Live-Migrate it and check if all goes well. Then you can proceed with the other ones....
Because we needed the Namespaces for PBS.
Further, we evaluated v7.2 on 3 different PVE-Installations since it released in pvetest repository and we didn´t see such issues -> "you do it, you do it wrong" ;)

However, thanks for your help :)
 
Last edited:
The package pve-qemu-kvm=6.2.0-8 which reverts the problematic change for QEMU's RBD driver, is now available on the pvetest repository.

You can test it by:
  1. Adding/Enabling the pvetest repository (can also be done in the Repostiories panel in the UI).
  2. apt update && apt install pve-qemu-kvm
  3. Removing/disabling the pvetest repository again.
  4. VMs need to be stopped/started or migrated to pick up the new version.
hi, we ran last night in the same problem.
many vms stuck with qmp timeouts.
we have an rbd-mirror standby cluster, with journal replication. this cluster is now online and was working up to now.

we have one vm with the same problem.
should we test the new pve-qemu-kvm version?
must we change to krbd? can we do this while vms are online, or what have we to do with the vms?

its a big problem for us, with not so much sleep last night.

thanks for help,
ronny
 
should we test the new pve-qemu-kvm version?
must we change to krbd? can we do this while vms are online, or what have we to do with the vms?

its a big problem for us, with not so much sleep last night.
Some suggestions from the "official" side would be good...
 
Some suggestions from the "official" side would be good...
As you have a subscription, you can open a case/ticket and request an official statement about this....
 
should we test the new pve-qemu-kvm version?
must we change to krbd?
If it's the same problem as described in this thread, either approach should work around the issue.
can we do this while vms are online, or what have we to do with the vms?
If you use the new pve-qemu-kvm, there's no need to switch to krbd. You do need to stop/start or migrate the VMs for either workaround so they pick up the new version or new storage configuration.
 
some other question
do you all have these problems in combination with pbs?
i updated to 2.2 ... released yesterday :/

i wonder, how close and state of the art, we are
 
If it's the same problem as described in this thread, either approach should work around the issue.

If you use the new pve-qemu-kvm, there's no need to switch to krbd. You do need to stop/start or migrate the VMs for either workaround so they pick up the new version or new storage configuration.
thank you, maybe we will try tomorrow ... running out of power today
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!