VM shutdown, KVM: entry failed, hardware error 0x80000021

tzzz90

New Member
May 10, 2022
14
1
3
Hello Community,

The following problem exists, a Windows Server 2019 VM migrated to Proxmox crashes regularly. The following can be found in /var/log/syslog:

May 10 08:37:14 prox01 QEMU[1148]: KVM: entry failed, hardware error 0x80000021 May 10 08:37:14 prox01 QEMU[1148]: If you're running a guest on an Intel machine without unrestricted mode May 10 08:37:14 prox01 kernel: [87072.394575] set kvm_intel.dump_invalid_vmcs=1 to dump internal KVM state. May 10 08:37:14 prox01 QEMU[1148]: support, the failure can be most likely due to the guest entering an invalid May 10 08:37:14 prox01 QEMU[1148]: state for Intel VT. For example, the guest maybe running in big real mode May 10 08:37:14 prox01 QEMU[1148]: which is not supported on less recent Intel processors. May 10 08:37:14 prox01 QEMU[1148]: EAX=002b0c36 EBX=1cfd9180 ECX=00000000 EDX=00000000 May 10 08:37:14 prox01 QEMU[1148]: ESI=1cfe9200 EDI=33c19080 EBP=00000000 ESP=23db2660 May 10 08:37:14 prox01 QEMU[1148]: EIP=00008000 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=1 HLT=0 May 10 08:37:14 prox01 QEMU[1148]: ES =0000 00000000 ffffffff 00809300 May 10 08:37:14 prox01 QEMU[1148]: CS =c200 7ffc2000 ffffffff 00809300 May 10 08:37:14 prox01 QEMU[1148]: SS =0000 00000000 ffffffff 00809300 May 10 08:37:14 prox01 QEMU[1148]: DS =0000 00000000 ffffffff 00809300 May 10 08:37:14 prox01 QEMU[1148]: FS =0000 00000000 ffffffff 00809300 May 10 08:37:14 prox01 QEMU[1148]: GS =0000 00000000 ffffffff 00809300 May 10 08:37:14 prox01 QEMU[1148]: LDT=0000 00000000 000fffff 00000000 May 10 08:37:14 prox01 QEMU[1148]: TR =0040 1cfec000 00000067 00008b00 May 10 08:37:14 prox01 QEMU[1148]: GDT= 1cfedfb0 00000057 May 10 08:37:14 prox01 QEMU[1148]: IDT= 00000000 00000000 May 10 08:37:14 prox01 QEMU[1148]: CR0=00050032 CR2=ba0414d4 CR3=b9036002 CR4=00000000 May 10 08:37:14 prox01 QEMU[1148]: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 May 10 08:37:14 prox01 QEMU[1148]: DR6=00000000fffe0ff0 DR7=0000000000000400 May 10 08:37:14 prox01 QEMU[1148]: EFER=0000000000000000 May 10 08:37:14 prox01 QEMU[1148]: Code=kvm: ../hw/core/cpu-sysemu.c:77: cpu_asidx_from_attrs: Assertion `ret < cpu->num_ases && ret >= 0' failed.

pveversion

Code:
pve-manager/7.2-3/c743d6c1 (running kernel: 5.15.35-1-pve)

qm config

Code:
agent: 0
balloon: 0
bios: ovmf
boot:
cores: 8
cpu: host
efidisk0: local-lvm:vm-101-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
machine: pc-q35-5.0
memory: 16000
meta: creation-qemu=6.2.0,ctime=1651869005
name: Name
net0: e1000=5A:4E:0B:08:47:D9,bridge=vmbr0
numa: 0
onboot: 1
ostype: win10
scsi0: local-lvm:vm-101-disk-0,size=80G
scsi1: local-lvm:vm-101-disk-2,size=500G
scsihw: virtio-scsi-pci
smbios1: uuid=d5161f55-533b-41e3-ab1a-2b06ebd24a4a
sockets: 1
vmgenid: da29ce18-4168-4aca-a3c6-2b8b7254d88b


I can't really find a solution for this, hence the machine type q35-5.0, because there is no other network card.

CPU

Code:
description: CPU
          product: Intel(R) Xeon(R) E-2244G CPU @ 3.80GHz
          vendor: Intel Corp.
          physical id: 57
          bus info: cpu@0
          version: Intel(R) Xeon(R) E-2244G CPU @ 3.80GHz
          serial: To Be Filled By O.E.M.
          slot: CPU1
          size: 3231MHz
          capacity: 4005MHz
          width: 64 bits
          clock: 100MHz

This thread didnt helped me: https://forum.proxmox.com/threads/v...ntry-failed-hardware-error-0x80000021.109093/

Thank You

Best Regards,
tzzz
 
Same thing seems to happen to Windows 10 btw.
Problem apperaed today for me, but this wouldn't really line up with the last kernel update.
Noticed also may have happend the first time on tuesday, not sure, maybe someone just brought the Vm back up by hand, thinking he accedentily shut it down.
Not very reproducible right now, machine crashes every few hours.
Code:
May 10 22:30:17 tom QEMU[651818]: KVM: entry failed, hardware error 0x80000021
May 10 22:30:17 tom QEMU[651818]: If you're running a guest on an Intel machine without unrestricted mode
May 10 22:30:17 tom QEMU[651818]: support, the failure can be most likely due to the guest entering an invalid
May 10 22:30:17 tom QEMU[651818]: state for Intel VT. For example, the guest maybe running in big real mode
May 10 22:30:17 tom QEMU[651818]: which is not supported on less recent Intel processors.
May 10 22:30:17 tom QEMU[651818]: EAX=00000400 EBX=00000000 ECX=00000400 EDX=000fffff
May 10 22:30:17 tom QEMU[651818]: ESI=9d709f20 EDI=00000000 EBP=a5abeee0 ESP=0ac53fb0
May 10 22:30:17 tom QEMU[651818]: EIP=00008000 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=1 HLT=0
May 10 22:30:17 tom QEMU[651818]: ES =0000 00000000 ffffffff 00809300
May 10 22:30:17 tom QEMU[651818]: CS =8a00 7ff8a000 ffffffff 00809300
May 10 22:30:17 tom QEMU[651818]: SS =0000 00000000 ffffffff 00809300
May 10 22:30:17 tom QEMU[651818]: DS =0000 00000000 ffffffff 00809300
May 10 22:30:17 tom QEMU[651818]: FS =0000 00000000 ffffffff 00809300
May 10 22:30:17 tom QEMU[651818]: GS =0000 00000000 ffffffff 00809300
May 10 22:30:17 tom QEMU[651818]: LDT=0000 00000000 00000000 00000000
May 10 22:30:17 tom QEMU[651818]: TR =0040 0ac0f000 00000067 00008b00
May 10 22:30:17 tom QEMU[651818]: GDT=     0ac10fb0 00000057
May 10 22:30:17 tom QEMU[651818]: IDT=     00000000 00000000
May 10 22:30:17 tom QEMU[651818]: CR0=00050032 CR2=04f9efd4 CR3=001ad000 CR4=00000000
May 10 22:30:17 tom QEMU[651818]: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
May 10 22:30:17 tom QEMU[651818]: DR6=00000000ffff0ff0 DR7=0000000000000400
May 10 22:30:17 tom QEMU[651818]: EFER=0000000000000000
May 10 22:30:17 tom QEMU[651818]: Code=kvm: ../hw/core/cpu-sysemu.c:77: cpu_asidx_from_attrs: Assertion `ret < cpu->num_ases && ret >= 0' failed.

And the CPU is:
Code:
model name      : Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz
stepping        : 7
microcode       : 0x5003003
 
Last edited:
Try setting the CPU type to kvm64.
If Hyper-V is enabled, also try disabling it.

This error might be related to nested virtualization.
 
So to be honest I already did that, set host -> kvm64 and made sure HyperV is not installed/enabled.
I do have nested virtualisation on the host enabled though, but that's been on for months now, w.o. any issues.

In the Windows Event log there's unfortunatly nothing directly linked to this event.
The only thing I currently have running is replication (*/30), that would match most of the incedents I had.
These are the times this happened to me, I don't see the pattern, but this seems too much of a coincidence.
10.05 22:30
12.05 14:30
12.05 15:30
12.05 18:55
13.05 14:30

Code:
agent: 1
audio0: device=ich9-intel-hda,driver=none
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 20
cpu: kvm64
efidisk0: spool:vm-118-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:af:00,pcie=1
machine: pc-q35-6.2
memory: 131072
name: IRRELEVANT
net0: virtio=IRRELEVANT,bridge=vmbr0,firewall=1,tag=30
numa: 1
onboot: 1
ostype: win10
scsi0: spool:vm-118-disk-1,discard=on,iothread=1,queues=16,size=320G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=IRRELEVANT
sockets: 2
tablet: 1
tpmstate0: spool:vm-118-disk-2,size=4M,version=v2.0
vga: std,memory=512
vmgenid: IRRELEVANT

Just remembered, might be worth mentioning:
I passed trough a Nvidia GTX1050, but again, was fine for months.
 
Last edited:
How long does it take on average for the crash to appear?

I'm currently trying to reproduce it, but something you should know is that nested Hyper-V on KVM is still experimental.
Intel often enough works just fine, AMD still doesn't. But even with Intel you can always run into issues. Any Windows Update, Kernel Update, BIOS update, QEMU update can lead to new issues.

Do you get those issues on a new install, without cpu type 'host' and without Hyper-V, WSL2 etc. installed?

Does it work without PCI Passthrough?
 
So I was expecting the machine to crash again at 15:30 - but this time noting happened.
As I currently don't know what leads to the crash it's hard to tell, if any changes will fix anything.
Afaik as soon as I switch to KVM64 nested virtualisation is disabled anyway, right?

Furthermore tzzz posted the same error, but has no PCIe passtrough enabled according to the provided config, so I would guess that isn't the cause.

I do have another machine running just fine on the same host.
I guess if I roll back to the last kernel it would work fine, but then again, as soon as the current non-production kernel is integrated into the enterprise repo, shit might hit the fan big time.
 
If you can boot an older kernel (you can even pin one with proxmox-boot-tool), and you don't have those issues, it could help us narrow down the changes that lead to this.
 
Will do, for now I'm gonna keep an eye, if this continues happening. Next time it crashes I'm gonna switch to old kernel and see if it happens again.
Unfortunatly I can't manually cause this, therefor I logically can only prove that it's happening, not that it's not.

Does anybody have a guess/idea on how I could manually trigger this? Maybe some C code or something trying to access virtualisation/real mode inside the vm? Unfortunatly x86 is really outside my scope.

  • 5.15.30-2-pve also just crashed
  • 5.13.19-6-pve trying this now
 
Last edited:
Try setting the CPU type to kvm64.
If Hyper-V is enabled, also try disabling it.

This error might be related to nested virtualization.
I tried this, but again none effect. And there is no nested virtualization. Yesterday evening again a crash of the VM.
 
I didn't really know if this is the problem and if this is a proper solution to it.

VM: Win10 (CPU Type kvm64 / Machine Type Q35-6.2) PVE Kernel: PVE 5.15.35-3 CPU: Intel(R) Xeon(R) Silver 4210

I found out that My VM only stops when going into replication that runs every 30 mins.
Sometimes the qemu-guest-agents drops out of Service, maybe because it fails of Calling Volume Shadow Copy Service.

I found out that in my case there were a permission problems inside the VM.

Open DCOMCNFG.exe
Search for Component Services -> Computers -> My Computer

open Properties, goto COM-Security
Under Access Permission -> Edit Default

Add Network-Service and SYSTEM with Local Access Allow

this solved my problem until now, hope it will last.


unfortunately it happened again next Day at 10:30 am

Kernel Error
Code:
[Sun May 15 10:30:22 2022] set kvm_intel.dump_invalid_vmcs=1 to dump internal KVM state.

Replication Log
Code:
2022-05-15 10:30:18 118-0: start replication job
2022-05-15 10:30:18 118-0: guest => VM 118, running => 2043102
2022-05-15 10:30:18 118-0: volumes => spool:vm-118-disk-0,spool:vm-118-disk-1,spool:vm-118-disk-2
2022-05-15 10:30:19 118-0: freeze guest filesystem
2022-05-15 10:30:20 118-0: create snapshot '__replicate_118-0_1652603418__' on spool:vm-118-disk-0
2022-05-15 10:30:20 118-0: create snapshot '__replicate_118-0_1652603418__' on spool:vm-118-disk-1
2022-05-15 10:30:20 118-0: create snapshot '__replicate_118-0_1652603418__' on spool:vm-118-disk-2
2022-05-15 10:30:20 118-0: thaw guest filesystem
2022-05-15 10:30:20 118-0: using secure transmission, rate limit: none
2022-05-15 10:30:20 118-0: incremental sync 'spool:vm-118-disk-0' (__replicate_118-0_1652601617__ => __replicate_118-0_1652603418__)
2022-05-15 10:30:21 118-0: send from @__replicate_118-0_1652601617__ to spool/vm-118-disk-0@__replicate_118-0_1652603418__ estimated size is 624B
2022-05-15 10:30:21 118-0: total estimated size is 624B
2022-05-15 10:30:21 118-0: successfully imported 'spool:vm-118-disk-0'
2022-05-15 10:30:21 118-0: incremental sync 'spool:vm-118-disk-1' (__replicate_118-0_1652601617__ => __replicate_118-0_1652603418__)
2022-05-15 10:30:23 118-0: send from @__replicate_118-0_1652601617__ to spool/vm-118-disk-1@__replicate_118-0_1652603418__ estimated size is 258M
2022-05-15 10:30:23 118-0: total estimated size is 258M
2022-05-15 10:30:24 118-0: successfully imported 'spool:vm-118-disk-1'
2022-05-15 10:30:24 118-0: incremental sync 'spool:vm-118-disk-2' (__replicate_118-0_1652601617__ => __replicate_118-0_1652603418__)
2022-05-15 10:30:25 118-0: send from @__replicate_118-0_1652601617__ to spool/vm-118-disk-2@__replicate_118-0_1652603418__ estimated size is 624B
2022-05-15 10:30:25 118-0: total estimated size is 624B
2022-05-15 10:30:25 118-0: successfully imported 'spool:vm-118-disk-2'
2022-05-15 10:30:25 118-0: delete previous replication snapshot '__replicate_118-0_1652601617__' on spool:vm-118-disk-0
2022-05-15 10:30:25 118-0: delete previous replication snapshot '__replicate_118-0_1652601617__' on spool:vm-118-disk-1
2022-05-15 10:30:25 118-0: delete previous replication snapshot '__replicate_118-0_1652601617__' on spool:vm-118-disk-2
2022-05-15 10:30:26 118-0: (remote_finalize_local_job) delete stale replication snapshot '__replicate_118-0_1652601617__' on spool:vm-118-disk-0
2022-05-15 10:30:26 118-0: (remote_finalize_local_job) delete stale replication snapshot '__replicate_118-0_1652601617__' on spool:vm-118-disk-1
2022-05-15 10:30:26 118-0: (remote_finalize_local_job) delete stale replication snapshot '__replicate_118-0_1652601617__' on spool:vm-118-disk-2
2022-05-15 10:30:26 118-0: end replication job
 
Last edited:
Hello everyone! As far as I examined, the above problem appeared with 5.15 Proxmox kernel after update to Proxmox 7.2.
With enabled nested virtualization (cat /sys/module/kvm_intel/parameters/nested replies Y) on Proxmox host.
With Windows guests.
And only on Intel processors, In my case it is Intel(R) Xeon(R) CPU E3-1275 v6.

Have not faced any guest crashes on a number of AMD systems with enabled nested virtualization after update to Proxmox 7.2.
Also have not faced any guest crashes after booting the above E3-1275 v6 system with 5.13.19-6-pve kernel.
 
Hello!
I also had this issue receltly on of my newly transformed server with Xeon W-2125 (transission to Proxmox) with 7.2 installer. Wierd thing is, that i have 2 guest VMs with Windows 2022 (almost same config (just different cpu and ram ammounts) on the same Host machine, and ONLY the one with SQLserver is crashing, the one running filesharing services runs for more than week without problems. I have few servers with AMD EPYC servers which i upgraded to the same version, but non of the problems appear there. Today i upgraded an older HP server with Intel CPU, so we will see if the problem appears there also :-/

May 15 05:39:55 pve-2max QEMU[3138]: KVM: entry failed, hardware error 0x80000021
May 15 05:39:55 pve-2max kernel: [300493.195097] set kvm_intel.dump_invalid_vmcs=1 to dump internal KVM state.
May 15 05:39:55 pve-2max QEMU[3138]: If you're running a guest on an Intel machine without unrestricted mode
May 15 05:39:55 pve-2max QEMU[3138]: support, the failure can be most likely due to the guest entering an invalid
May 15 05:39:55 pve-2max QEMU[3138]: state for Intel VT. For example, the guest maybe running in big real mode
May 15 05:39:55 pve-2max QEMU[3138]: which is not supported on less recent Intel processors.
May 15 05:39:55 pve-2max QEMU[3138]: EAX=00002e39 EBX=be088180 ECX=00000001 EDX=00000000
May 15 05:39:55 pve-2max QEMU[3138]: ESI=8b9c9040 EDI=be0941c0 EBP=00000000 ESP=ed83f540
May 15 05:39:55 pve-2max QEMU[3138]: EIP=00008000 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=1 HLT=0
May 15 05:39:55 pve-2max QEMU[3138]: ES =0000 00000000 ffffffff 00809300
May 15 05:39:55 pve-2max QEMU[3138]: CS =be00 7ffbe000 ffffffff 00809300
May 15 05:39:55 pve-2max QEMU[3138]: SS =0000 00000000 ffffffff 00809300
May 15 05:39:55 pve-2max QEMU[3138]: DS =0000 00000000 ffffffff 00809300
May 15 05:39:55 pve-2max QEMU[3138]: FS =0000 00000000 ffffffff 00809300
May 15 05:39:55 pve-2max QEMU[3138]: GS =0000 00000000 ffffffff 00809300
May 15 05:39:55 pve-2max QEMU[3138]: LDT=0000 00000000 00000000 00000000
May 15 05:39:55 pve-2max QEMU[3138]: TR =0040 be097000 00000067 00008b00
May 15 05:39:55 pve-2max QEMU[3138]: GDT= be098fb0 00000057
May 15 05:39:55 pve-2max QEMU[3138]: IDT= 00000000 00000000
May 15 05:39:55 pve-2max QEMU[3138]: CR0=00050032 CR2=b77c1228 CR3=001ae002 CR4=00000000
May 15 05:39:55 pve-2max QEMU[3138]: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
May 15 05:39:55 pve-2max QEMU[3138]: DR6=00000000fffe0ff0 DR7=0000000000000400
May 15 05:39:55 pve-2max QEMU[3138]: EFER=0000000000000000
May 15 05:39:55 pve-2max QEMU[3138]: Code=kvm: ../hw/core/cpu-sysemu.c:77: cpu_asidx_from_attrs: Assertion `ret < cpu->num_ases && ret >= 0' failed.
May 15 05:39:56 pve-2max kernel: [300493.605172] zd144: p1 p2
May 15 05:39:56 pve-2max kernel: [300493.622364] vmbr0: port 2(tap100i0) entered disabled state
May 15 05:39:56 pve-2max kernel: [300493.622626] vmbr0: port 2(tap100i0) entered disabled state
May 15 05:39:56 pve-2max kernel: [300493.623192] zd48: p1 p2 p3 p4
May 15 05:39:56 pve-2max systemd[1]: 100.scope: Succeeded.
May 15 05:39:56 pve-2max systemd[1]: 100.scope: Consumed 4h 42min 42.850s CPU time.
May 15 05:39:56 pve-2max qmeventd[2873239]: Starting cleanup for 100
May 15 05:39:56 pve-2max qmeventd[2873239]: Finished cleanup for 100
 
Will do, for now I'm gonna keep an eye, if this continues happening. Next time it crashes I'm gonna switch to old kernel and see if it happens again.
Unfortunatly I can't manually cause this, therefor I logically can only prove that it's happening, not that it's not.

Does anybody have a guess/idea on how I could manually trigger this? Maybe some C code or something trying to access virtualisation/real mode inside the vm? Unfortunatly x86 is really outside my scope.

  • 5.15.30-2-pve also just crashed
  • 5.13.19-6-pve trying this now
Did changing the kernel help?
 
Did changing the kernel help?
5.13.19-6-pve kernel works absolutely stable in my case.
Also I have not faced any guest crashes with 5.15.35-1-pve kernel if nested virtualization on Proxmox host is disabled
(cat /sys/module/kvm_intel/parameters/nested replies N).
But I have not tested for a long time, only 24h, as I need nested for some VMs. If someone don`t need it, I suggest to check this option.
To toggle nested virtualization /etc/modprobe.d/kvm-intel.conf can be used.
set options kvm-intel nested=0 to disable.
 
I tried reproducing it here on an Intel i7-6700K by letting 2 VMs (Win10, Win11) run over the weekend with nested virtualization, CPU `host` and running VMs inside (Ubuntu on Hyper-V) but I couldn't.
They were still running today without any issues.

It looks like all the CPUs with these issues are newer than Skylake?
 
I tried reproducing it here on an Intel i7-6700K by letting 2 VMs (Win10, Win11) run over the weekend with nested virtualization, CPU `host` and running VMs inside (Ubuntu on Hyper-V) but I couldn't.
They were still running today without any issues.

It looks like all the CPUs with these issues are newer than Skylake?
on my affected RIG is Intel Xeon W-2125 which is Skylake. I'm gonna test turning OFF the nested option through modprobe options as @apkoval suggested. The problematic VM crushes once per 1-2days arround 5 am
 
Last edited:
So I collected some data over the weekend and for me there seems to be no kernel version related issue.
It crashes on all versions.
May 12 17:31:425.15.35-1-pve
May 12 18:55:07KVM: entry failed, hardware error 0x80000021
May 12 21:43:175.15.35-1-pve
May 13 14:30:14KVM: entry failed, hardware error 0x80000021
May 13 18:25:335.15.35-1-pve
May 13 19:00:15KVM: entry failed, hardware error 0x80000021
May 13 20:06:075.15.35-1-pve
May 13 22:26:505.15.30-2-pve
May 13 22:38:42KVM: entry failed, hardware error 0x80000021
May 13 22:41:585.13.19-6-pve
May 14 00:04:40KVM: entry failed, hardware error 0x80000021
May 14 13:34:195.15.35-1-pve
May 15 10:30:20KVM: entry failed, hardware error 0x80000021
May 15 11:00:20KVM: entry failed, hardware error 0x80000021
May 15 21:24:415.13.19-6-pve
 
So I collected some data over the weekend and for me there seems to be no kernel version related issue.
It crashes on all versions.
May 12 17:31:425.15.35-1-pve
May 12 18:55:07KVM: entry failed, hardware error 0x80000021
May 12 21:43:175.15.35-1-pve
May 13 14:30:14KVM: entry failed, hardware error 0x80000021
May 13 18:25:335.15.35-1-pve
May 13 19:00:15KVM: entry failed, hardware error 0x80000021
May 13 20:06:075.15.35-1-pve
May 13 22:26:505.15.30-2-pve
May 13 22:38:42KVM: entry failed, hardware error 0x80000021
May 13 22:41:585.13.19-6-pve
May 14 00:04:40KVM: entry failed, hardware error 0x80000021
May 14 13:34:195.15.35-1-pve
May 15 10:30:20KVM: entry failed, hardware error 0x80000021
May 15 11:00:20KVM: entry failed, hardware error 0x80000021
May 15 21:24:415.13.19-6-pve
It is very strange to hear about problems on 5.13.19-6-pve kernel.
Maybe it is also CPU specific.
I can confirm that VMs do not crash with this Kernel on a number of hosts with Xeon E3-1275 v6 and i9-9900K CPUs.

Near a month ago I have tried running some test versions of 5.15 pve kernel on Proxmox 7.1 for a number of days, and there were same Windows VM crashes. After rolling back to 5.13 branch there were no single crash until updating to Proxmox 7.2 with stock 5.15 kernel.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!