PBS PVE Fail Backup

pakuzaz

Well-Known Member
Apr 27, 2020
466
29
48
38
Hi we have the following error message out


INFO: starting new backup job: vzdump 1** --remove 0 --node srv-*** --mode snapshot --storage PBS
INFO: Starting Backup of VM 1** (qemu)
INFO: Backup started at 2020-07-16 13:19:22
INFO: status = running
INFO: VM Name: srv-***
Disk 1
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1**/2020-07-16T11:19:22Z'
INFO: skipping guest-agent 'fs-freeze', agent configured but not running?
INFO: started backup task 'dad5cb91-b642-47b7-9912-146ee5708eab'
INFO: resuming VM again
INFO: status: 0% (124.0 MiB of 130.0 GiB), duration 3, read: 41.3 MiB/s, write: 41.3 MiB/s
INFO: status: 0% (1.3 GiB of 130.0 GiB), duration 37, read: 35.1 MiB/s, write: 35.1 MiB/s
ERROR: backup write data failed: command error: write_data upload error: pipelined request failed: HTTP Error 400 Bad Request: No such file or directory (os error 2)
INFO: aborting backup job
ERROR: Backup of VM 1** failed - backup write data failed: command error: write_data upload error: pipelined request failed: HTTP Error 400 Bad Request: No such file or directory (os error 2)
INFO: Failed at 2020-07-16 13:20:02
INFO: Backup job finished with errors
TASK ERROR: job errors
 
pveversion -v output and /etc/pve/storage.cfg contents would be interesting.
 
pveversion -v

Code:
proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.4.44-1-pve: 5.4.44-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.18-2-pve: 5.3.18-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve2
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-1
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-11
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-10
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1

Code:
pbs: PBS
        datastore backup1
        server 10.XX.X.XXX
        content backup
        fingerprint XX:ca:b1:4a:5d:72:XX:d9:21:19:7a:ee:cd:3d:XX:24:be:51:58:b9:4d:ec:XX:21:58:68:d6:43:cf:26:c7:1f
        maxfiles 0
        username backupuser@pam
 
and 'pvesm list PBS' works?
 
anz more details on the PBS end in the logs? e.g. journalctl -u proxmox-backup -u proxmox-backup-proxy
 
2020-07-16T13:19:25+02:00: starting new backup on datastore 'backup1': "vm/1XX/2020-07-16T11:19:22Z"
2020-07-16T13:19:25+02:00: GET /previous: 400 Bad Request: no previous backup
2020-07-16T13:19:25+02:00: created new fixed index 1 ("vm/1XX/2020-07-16T11:19:22Z/drive-sata0.img.fidx")
2020-07-16T13:19:25+02:00: add blob "/home/backup/vm/1XX/2020-07-16T11:19:22Z/qemu-server.conf.blob" (323 bytes, comp: 323)
2020-07-16T13:20:02+02:00: POST /fixed_chunk: 400 Bad Request: No such file or directory (os error 2)
2020-07-16T13:20:02+02:00: backup failed: connection error: bytes remaining on stream
2020-07-16T13:20:02+02:00: removing failed backup
2020-07-16T13:20:02+02:00: TASK ERROR: connection error: bytes remaining on stream
2020-07-16T13:20:02+02:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: broken pipe
 
did you manually modify the datastore in any way? such as rm -rf .chunks, or similar? is it just this one VM that is failing, or all? if you create a new datastore, does it work then?
 
I am having the same issue, I created the store and added it to the cluster by exactly following the instructions in the documentation.
 
could you post your storage.cfg on the PVE side and the datastore.cfg on the PBS side?
 
I ran into the same problem too. I have PVE and PBS connected via internet, not local network.

PVE 6.2-12 and PBS 0.9-1

Code:
Oct 26 10:28:32 pbs proxmox-backup-proxy[9156]: register worker
Oct 26 10:28:32 pbs proxmox-backup-proxy[9156]: FILE: "/var/log/proxmox-backup/tasks/C0/UPID:pbs:000023C4:002CD9C0:00000009:5F9696C0:backup:Azienda_vm_101:root@pam:"
Oct 26 10:28:32 pbs proxmox-backup-proxy[9156]: starting new backup on datastore 'Azienda': "vm/101/2020-10-26T09:28:31Z"
Oct 26 10:28:32 pbs proxmox-backup-proxy[9156]: GET /previous: 400 Bad Request: no valid previous backup
Oct 26 10:28:32 pbs proxmox-backup-proxy[9156]: created new fixed index 1 ("vm/101/2020-10-26T09:28:31Z/drive-scsi0.img.fidx")
Oct 26 10:28:32 pbs proxmox-backup-proxy[9156]: add blob "/BackUpAzienda/vm/101/2020-10-26T09:28:31Z/qemu-server.conf.blob" (298 bytes, comp: 298)
Oct 26 10:34:12 pbs proxmox-backup-proxy[9156]: error during snapshot file listing: 'unable to load blob '"/BackUpAzienda/vm/101/2020-10-26T09:28:31Z/index.json.blob"' - No such file or directory (os error 2)'
Oct 26 10:43:31 pbs proxmox-backup-proxy[9156]: error during snapshot file listing: 'unable to load blob '"/BackUpAzienda/vm/101/2020-10-26T09:28:31Z/index.json.blob"' - No such file or directory (os error 2)'

Bash:
root@pbs:~# ls -alh /BackUpAzienda/vm/101/2020-10-26T09\:28\:31Z/
total 56K
drwxr-xr-x 2 backup backup 4.0K Oct 26 10:28 .
drwxr-xr-x 3 backup backup 4.0K Oct 26 10:28 ..
-rw-r--r-- 1 backup backup 260K Oct 26 10:45 drive-scsi0.img.tmp_fidx
-rw-r--r-- 1 backup backup  298 Oct 26 10:28 qemu-server.conf.blob

My PVE storage.cfg:
Code:
dir: local
path /var/lib/vz
content iso,vztmpl,backup

lvmthin: local-lvm
thinpool data
vgname pve
content rootdir,images

zfspool: SDD
pool SDD
content rootdir,images
mountpoint /SDD
nodes proxmox

pbs: BackupRemoto
datastore Azienda
server DNSPBS.COM
content backup
fingerprint 84:53:de:62:f3:4e:19:7e:ab:a8:ea:8d:a4:46:36:66:37:df:e7:13:1f:e8:3a:e5:ea:37:95:8c:f7:c2:1a:0d
maxfiles 0
username root@pam

My datastore.cfg on the PBS:
Code:
datastore: Azienda
    comment
    path /BackUpAzienda

PVESM on PVE:
Code:
root@proxmox:~# pvesm list BackupRemoto
Volid                                           Format  Type      Size VMID                  
BackupRemoto:backup/vm/101/2020-10-26T09:28:31Z pbs-vm  backup    1 101
 
does the .chunks directory exist in the datastore path?
 
sorry, I missed that you are a new user. what does your PVE-side task log say?
 
sorry, I missed that you are a new user. what does your PVE-side task log say?
On PVE:
Code:
INFO: starting new backup job: vzdump 101 --storage BackupRemoto --node proxmox --mode stop --remove 0
INFO: Starting Backup of VM 101 (qemu)
INFO: Backup started at 2020-10-26 12:55:22
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: Issabel
INFO: include disk 'scsi0' 'local-lvm:vm-101-disk-0' 32G
INFO: creating Proxmox Backup Server archive 'vm/101/2020-10-26T11:55:21Z'
INFO: starting kvm to execute backup task
INFO: started backup task '8032f0cd-c465-4a0e-b470-7298297dd8e9'
INFO: scsi0: dirty-bitmap status: disabled (VM not running)
INFO:   0% (52.0 MiB of 32.0 GiB) in  3s, read: 17.3 MiB/s, write: 17.3 MiB/s
INFO:   0% (80.0 MiB of 32.0 GiB) in 17m 37s, read: 27.2 KiB/s, write: 27.2 KiB/s

BLOCKED FOR 13 HOURS AND MANUAL ABORT

Code:
ERROR: backup write data failed: command error: protocol canceled
INFO: aborting backup job
INFO: stopping kvm after backup task
ERROR: Backup of VM 101 failed - backup write data failed: command error: protocol canceled
INFO: Failed at 2020-10-26 13:13:02
INFO: Backup job finished with errors
TASK ERROR: job errors

ON PBS:
Code:
2020-10-25T20:56:28+01:00: starting new backup on datastore 'Azienda': "vm/102/2020-10-25T19:56:28Z"
2020-10-25T20:56:28+01:00: GET /previous: 400 Bad Request: no valid previous backup
2020-10-25T20:56:28+01:00: created new fixed index 1 ("vm/102/2020-10-25T19:56:28Z/drive-scsi0.img.fidx")
2020-10-25T20:56:28+01:00: add blob "/BackUpAzienda/vm/102/2020-10-25T19:56:28Z/qemu-server.conf.blob" (307 bytes, comp: 307)

BLOCKED FOR 13 HOURS AND MANUAL ABORT

Code:
2020-10-26T09:14:07+01:00: backup failed: task aborted
2020-10-26T09:14:07+01:00: removing failed backup
2020-10-26T09:14:07+01:00: TASK ERROR: task aborted
 
yeah, so if you abort a backup there is no manifest, but a snapshot dir, and the warnings about the manifest not being there are valid. pruning will clean it up.
 
Same problem here.
Proxmox:
Code:
proxmox-ve: 6.4-1 (running kernel: 5.4.114-1-pve)
pve-manager: 6.4-8 (running version: 6.4-8/185e14db)
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: not correctly installed
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.4-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-3
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.9-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-6
pve-cluster: 6.4-1
pve-container: 3.3-5
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-4
pve-firmware: 3.2-4
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1

PBS:
Code:
proxmox-backup: 1.0-4 (running kernel: 5.4.114-1-pve)
proxmox-backup-server: 1.1.9-1 (running version: 1.1.9)
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.98-1-pve: 5.4.98-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-5.4.60-1-pve: 5.4.60-2
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
ifupdown2: 3.0.0-1+pve3
libjs-extjs: 6.0.1-10
proxmox-backup-docs: 1.1.9-1
proxmox-backup-client: 1.1.9-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-6
pve-xtermjs: 4.7.0-3
smartmontools: 7.2-pve2
zfsutils-linux: 2.0.4-pve1

Only on two VMs on one of the nodes that are backupped to this PBS i get the error:
Code:
INFO: starting new backup job: vzdump 103 105 --node aura --mode snapshot --quiet 1 --mailnotification always --storage aura --all 0
INFO: Starting Backup of VM 103 (qemu)
INFO: Backup started at 2021-06-05 19:16:09
INFO: status = running
INFO: VM Name: dockerish
INFO: include disk 'scsi0' 'local-zfs:vm-103-disk-0' 40G
INFO: include disk 'scsi2' 'local-zfs:vm-103-disk-1' 50G
INFO: include disk 'scsi3' 'local-zfs:vm-103-disk-2' 500G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/103/2021-06-05T17:16:09Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '96f1a5e5-d139-4f61-a4e4-e4a21f69c4f4'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: scsi2: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: scsi3: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO:   0% (72.0 MiB of 590.0 GiB) in 3s, read: 24.0 MiB/s, write: 14.7 MiB/s
INFO:   0% (1.2 GiB of 590.0 GiB) in 18m 21s, read: 1.0 MiB/s, write: 958.7 KiB/s
ERROR: backup write data failed: command error: protocol canceled
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 103 failed - backup write data failed: command error: protocol canceled
INFO: Failed at 2021-06-05 19:34:33
INFO: Starting Backup of VM 105 (qemu)
INFO: Backup started at 2021-06-05 19:34:33
INFO: status = running
INFO: VM Name: zabbix
INFO: include disk 'scsi0' 'local-zfs:vm-105-disk-1' 50G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: pending configuration changes found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/105/2021-06-05T17:34:33Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'cee6d78f-8501-4823-ad53-529f1c70e499'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO:   0% (220.0 MiB of 50.0 GiB) in 3s, read: 73.3 MiB/s, write: 28.0 MiB/s
INFO:   1% (528.0 MiB of 50.0 GiB) in 10s, read: 44.0 MiB/s, write: 44.0 MiB/s
INFO:   2% (1.0 GiB of 50.0 GiB) in 1m, read: 10.0 MiB/s, write: 10.0 MiB/s
INFO:   3% (1.5 GiB of 50.0 GiB) in 2m, read: 8.8 MiB/s, write: 8.8 MiB/s
INFO:   4% (2.0 GiB of 50.0 GiB) in 2m 41s, read: 12.5 MiB/s, write: 12.5 MiB/s
INFO:   5% (2.5 GiB of 50.0 GiB) in 3m 1s, read: 25.2 MiB/s, write: 25.2 MiB/s
INFO:   6% (3.0 GiB of 50.0 GiB) in 3m 40s, read: 12.8 MiB/s, write: 10.6 MiB/s
INFO:   7% (3.5 GiB of 50.0 GiB) in 4m 32s, read: 9.8 MiB/s, write: 9.8 MiB/s
INFO:   8% (4.0 GiB of 50.0 GiB) in 5m 29s, read: 9.0 MiB/s, write: 9.0 MiB/s
INFO:   9% (4.5 GiB of 50.0 GiB) in 6m 38s, read: 7.5 MiB/s, write: 7.5 MiB/s
INFO:  10% (5.0 GiB of 50.0 GiB) in 7m 28s, read: 10.3 MiB/s, write: 7.9 MiB/s
INFO:  11% (5.5 GiB of 50.0 GiB) in 8m 29s, read: 8.3 MiB/s, write: 8.3 MiB/s
INFO:  12% (6.0 GiB of 50.0 GiB) in 9m 27s, read: 8.8 MiB/s, write: 8.8 MiB/s
INFO:  13% (6.5 GiB of 50.0 GiB) in 10m 45s, read: 6.6 MiB/s, write: 6.6 MiB/s
INFO:  14% (7.0 GiB of 50.0 GiB) in 11m 43s, read: 8.8 MiB/s, write: 6.8 MiB/s
INFO:  15% (7.5 GiB of 50.0 GiB) in 13m, read: 6.6 MiB/s, write: 6.6 MiB/s
INFO:  16% (8.0 GiB of 50.0 GiB) in 14m 14s, read: 6.9 MiB/s, write: 6.9 MiB/s
INFO:  17% (8.5 GiB of 50.0 GiB) in 15m 27s, read: 7.0 MiB/s, write: 7.0 MiB/s
INFO:  18% (9.0 GiB of 50.0 GiB) in 16m 41s, read: 7.0 MiB/s, write: 6.2 MiB/s
INFO:  19% (9.5 GiB of 50.0 GiB) in 17m 30s, read: 10.4 MiB/s, write: 10.4 MiB/s
INFO:  20% (10.0 GiB of 50.0 GiB) in 18m 39s, read: 7.5 MiB/s, write: 7.5 MiB/s
INFO:  21% (10.5 GiB of 50.0 GiB) in 19m 50s, read: 7.1 MiB/s, write: 7.0 MiB/s
INFO:  22% (11.0 GiB of 50.0 GiB) in 20m 50s, read: 8.9 MiB/s, write: 6.9 MiB/s
INFO:  23% (11.5 GiB of 50.0 GiB) in 21m 59s, read: 7.1 MiB/s, write: 7.1 MiB/s
INFO:  24% (12.0 GiB of 50.0 GiB) in 22m 58s, read: 8.6 MiB/s, write: 8.6 MiB/s
INFO:  25% (12.5 GiB of 50.0 GiB) in 24m 3s, read: 7.9 MiB/s, write: 7.9 MiB/s
INFO:  26% (13.0 GiB of 50.0 GiB) in 25m, read: 9.0 MiB/s, write: 9.0 MiB/s
INFO:  27% (13.5 GiB of 50.0 GiB) in 26m 5s, read: 7.9 MiB/s, write: 6.0 MiB/s
INFO:  28% (14.0 GiB of 50.0 GiB) in 27m 29s, read: 6.0 MiB/s, write: 6.0 MiB/s
INFO:  29% (14.5 GiB of 50.0 GiB) in 28m 45s, read: 6.8 MiB/s, write: 6.8 MiB/s
INFO:  30% (15.0 GiB of 50.0 GiB) in 30m, read: 6.8 MiB/s, write: 6.8 MiB/s
INFO:  31% (15.5 GiB of 50.0 GiB) in 31m 9s, read: 7.4 MiB/s, write: 5.7 MiB/s
INFO:  32% (16.0 GiB of 50.0 GiB) in 32m 49s, read: 5.2 MiB/s, write: 5.2 MiB/s
INFO:  32% (16.2 GiB of 50.0 GiB) in 49m 8s, read: 192.5 KiB/s, write: 192.5 KiB/s
ERROR: backup write data failed: command error: protocol canceled
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 105 failed - backup write data failed: command error: protocol canceled
INFO: Failed at 2021-06-05 20:23:43
INFO: Backup job finished with errors
TASK ERROR: job errors

On PBS i get:
Code:
2021-06-05T21:15:36+02:00: starting new backup on datastore 'aura': "vm/103/2021-06-05T19:15:28Z"
2021-06-05T21:15:36+02:00: GET /previous: 400 Bad Request: no valid previous backup
2021-06-05T21:15:36+02:00: created new fixed index 1 ("vm/103/2021-06-05T19:15:28Z/drive-scsi0.img.fidx")
2021-06-05T21:15:36+02:00: created new fixed index 2 ("vm/103/2021-06-05T19:15:28Z/drive-scsi2.img.fidx")
2021-06-05T21:15:36+02:00: created new fixed index 3 ("vm/103/2021-06-05T19:15:28Z/drive-scsi3.img.fidx")
2021-06-05T21:15:37+02:00: add blob "/media/pbs/aura/vm/103/2021-06-05T19:15:28Z/qemu-server.conf.blob" (1129 bytes, comp: 1129)
2021-06-05T21:53:03+02:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: broken pipe
2021-06-05T21:53:03+02:00: backup failed: connection error: Connection timed out (os error 110)
2021-06-05T21:53:03+02:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: broken pipe
2021-06-05T21:53:03+02:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: broken pipe
2021-06-05T21:53:03+02:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: broken pipe
2021-06-05T21:53:03+02:00: removing failed backup
2021-06-05T21:53:03+02:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: broken pipe
2021-06-05T21:53:03+02:00: TASK ERROR: connection error: Connection timed out (os error 110)

As mentioned, backup of all other VMs works just fine.
Already tried to remove all snapshots from PBS, run a Prune and backup again but always the same result.
 
are these VMs especially active? or in some other way different from the rest? the symptoms point to a network issue, but that might just be what shows up in the logs and the root cause is something else..
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!