local node address: cannot use IP '*' it must be configured exactly once on local node!

mcpdev80

Member
Oct 2, 2019
9
5
23
Germany
Hi @all,

I've been trying to create a cluster of 3 nodes for several days now. I always get the same error message:

detected the following error(s):
* local node address: cannot use IP '178.*.*.*', it must be configured exactly once on local node!

TASK ERROR: Check if node may join a cluster failed!

Here is my config:
3 Server from Hetzner with diffrent IP networks and gateways:
I create two vSwitches:

10.0.100.10 master
10.0.100.11 slave0
10.0.100.12 slave1

10.0.200.10 master
10.0.200.11 slave0
10.0.200.12 slave1

interfaces:

auto vmbr0
iface vmbr0 inet static
address *.*.*.*
netmask 255.255.255.255
gateway *.*.*.*
bridge-ports enp6s0
bridge-stp off
bridge-fd 0
pointopoint *.*.*.*

iface vmbr0 inet6 auto
post-up echo 2048 > /sys/class/net/vmbr0/bridge/hash_max
post-up echo 1 > /sys/class/net/vmbr0/bridge/multicast_snooping
post-up echo 0 > /proc/sys/net/ipv6/conf/vmbr0/accept_ra

up ip route add -net *.*.*.* netmask 255.255.255.224 gw *.*.*.* vmbr0
up sysctl -w net.ipv4.ip_forward=1
up sysctl -w net.ipv4.conf.enp6s0.send_redirects=0
up sysctl -w net.ipv6.conf.all.forwarding=1
up ip route add 192.168.0.0/16 via 148.251.84.126 dev vmbr0
up ip route add 172.16.0.0/12 via 148.251.84.126 dev vmbr0
up ip route add 10.0.0.0/8 via 148.251.84.126 dev vmbr0


# vlan raw device
auto enp6s0.4000
iface enp6s0.4000 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4000
iface vmbr4000 inet static
bridge_ports enp6s0.4000
bridge_stp off
bridge_fd 0
address 10.0.100.10
netmask 24

# vlan raw device
auto enp6s0.4001
iface enp6s0.4001 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4001
iface vmbr4001 inet static
bridge_ports enp6s0.4001
bridge_stp off
bridge_fd 0
address 10.0.200.10
netmask 24

hosts:

127.0.0.1 localhost.localdomain localhost
*.*.*.* master.*.* master

10.0.100.10 master-cs.*.* master-cs
10.0.100.11 slave0-cs.*.* slave0-cs
10.0.100.12 slave1-cs.*.* slave1-cs

#
# IPv6
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
* master.*.* master


Here my commands:
pvecm create NameCluster --link0 10.0.100.10,priority=20 --link1 10.0.200.10,priority=15

pvecm add <IPMaster> --link0 10.0.100.11,priority=20 --link1 10.0.200.11,priority=15

pvecm add <IPMaster> --link0 10.0.100.12,priority=20 --link1 10.0.200.12,priority=15

Situation:
Cluster Create: OK
Add Node: Error
* local node address: cannot use IP '<IPFromNode>', it must be configured exactly once on local node!
 
Last edited:
what does "ip address show to IP/32 up" output? (replace "IP" with the IP in question..)
 
Hi fabian,

thank you for your response.

Here the information you want:

First Server: IP 6.10.87.55 Netmask 255.255.255.224 | IP 6.10.87.61 Netmask 255.255.255.224
Second Server: IP 163.53.20.33 Netmask 255.255.255.192
Third Server: IP 55.35.31.182 Netmask 255.255.255.192

interfaces for master:

auto lo
iface lo inet loopback

iface lo inet6 loopback

iface enp6s0 inet manual

iface enp6s0 inet6 manual

auto vmbr0
iface vmbr0 inet static
address 6.10.87.55
netmask 255.255.255.255
gateway 6.10.87.33
bridge-ports enp6s0
bridge-stp off
bridge-fd 0
pointopoint 6.10.87.33

iface vmbr0 inet6 auto
post-up echo 2048 > /sys/class/net/vmbr0/bridge/hash_max
post-up echo 1 > /sys/class/net/vmbr0/bridge/multicast_snooping
post-up echo 0 > /proc/sys/net/ipv6/conf/vmbr0/accept_ra

up ip route add -net 6.10.87.33 netmask 255.255.255.224 gw 6.10.87.33 vmbr0
up sysctl -w net.ipv4.ip_forward=1
up sysctl -w net.ipv4.conf.enp6s0.send_redirects=0
up sysctl -w net.ipv6.conf.all.forwarding=1
up ip route add 192.168.0.0/16 via 6.10.87.61 dev vmbr0
up ip route add 172.16.0.0/12 via 6.10.87.61 dev vmbr0
up ip route add 10.0.0.0/8 via 6.10.87.61 dev vmbr0


# vlan raw device
auto enp6s0.4000
iface enp6s0.4000 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4000
iface vmbr4000 inet static
bridge_ports enp6s0.4000
bridge_stp off
bridge_fd 0
address 10.0.100.10
netmask 24

# vlan raw device
auto enp6s0.4001
iface enp6s0.4001 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4001
iface vmbr4001 inet static
bridge_ports enp6s0.4001
bridge_stp off
bridge_fd 0
address 10.0.200.10
netmask 24



interfaces for slave0:

auto lo
iface lo inet loopback

iface lo inet6 loopback

iface enp6s0 inet manual

iface enp6s0 inet6 manual

auto vmbr0
iface vmbr0 inet static
address 163.53.20.33
netmask 255.255.255.255
gateway 163.53.20.1
bridge-ports enp6s0
bridge-stp off
bridge-fd 0
pointopoint 163.53.20.1

iface vmbr0 inet6 auto
post-up echo 2048 > /sys/class/net/vmbr0/bridge/hash_max
post-up echo 1 > /sys/class/net/vmbr0/bridge/multicast_snooping
post-up echo 0 > /proc/sys/net/ipv6/conf/vmbr0/accept_ra

up sysctl -w net.ipv4.ip_forward=1
up sysctl -w net.ipv4.conf.enp6s0.send_redirects=0
up sysctl -w net.ipv6.conf.all.forwarding=1


# vlan raw device
auto enp6s0.4000
iface enp6s0.4000 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4000
iface vmbr4000 inet static
bridge_ports enp6s0.4000
bridge_stp off
bridge_fd 0
address 10.0.100.11
netmask 24

# vlan raw device
auto enp6s0.4001
iface enp6s0.4001 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4001
iface vmbr4001 inet static
bridge_ports enp6s0.4001
bridge_stp off
bridge_fd 0
address 10.0.200.11
netmask 24


interfaces for slave1:

auto lo
iface lo inet loopback

iface lo inet6 loopback

iface enp6s0 inet manual

iface enp6s0 inet6 manual

auto vmbr0
iface vmbr0 inet static
address 55.35.31.182
netmask 255.255.255.255
gateway 55.35.31.129
bridge-ports enp6s0
bridge-stp off
bridge-fd 0
pointopoint 55.35.31.129

iface vmbr0 inet6 auto
post-up echo 2048 > /sys/class/net/vmbr0/bridge/hash_max
post-up echo 1 > /sys/class/net/vmbr0/bridge/multicast_snooping
post-up echo 0 > /proc/sys/net/ipv6/conf/vmbr0/accept_ra

up sysctl -w net.ipv4.ip_forward=1
up sysctl -w net.ipv4.conf.enp6s0.send_redirects=0
up sysctl -w net.ipv6.conf.all.forwarding=1


# vlan raw device
auto enp6s0.4000
iface enp6s0.4000 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4000
iface vmbr4000 inet static
bridge_ports enp6s0.4000
bridge_stp off
bridge_fd 0
address 10.0.100.11
netmask 24

# vlan raw device
auto enp6s0.4001
iface enp6s0.4001 inet static
vlan-raw-device enp6s0
mtu 1400

# vlan
auto vmbr4001
iface vmbr4001 inet static
bridge_ports enp6s0.4001
bridge_stp off
bridge_fd 0
address 10.0.200.11
netmask 24
 
Last edited:
please post the actual output of the command I posted ;)
 
Hi sorry i didn't get it ;-)

on master:

root@master ~ # ip address show to 6.10.87.55 /32 up
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
inet 6.10.87.55 peer 6.10.87.33/32 scope global enp6s0
valid_lft forever preferred_lft forever
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 6.10.87.55 peer 6.10.87.33/32 brd 6.10.87.55 scope global vmbr0
valid_lft forever preferred_lft forever
4: enp6s0.4000@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4000 state UP group default qlen 1000
inet 6.10.87.55 peer 6.10.87.33/32 scope global enp6s0.4000
valid_lft forever preferred_lft forever
5: vmbr4000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 6.10.87.55 peer 6.10.87.33/32 scope global vmbr4000
valid_lft forever preferred_lft forever
6: enp6s0.4001@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4001 state UP group default qlen 1000
inet 6.10.87.55 peer 6.10.87.33/32 scope global enp6s0.4001
valid_lft forever preferred_lft forever
7: vmbr4001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 6.10.87.55 peer6.10.87.33/32 scope global vmbr4001
valid_lft forever preferred_lft forever


on slave0:

root@slave0 ~ # ip address show to 163.53.20.33 /32 up
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
inet 163.53.20.33 peer 163.53.20.1/32 scope global enp6s0
valid_lft forever preferred_lft forever
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 163.53.20.33 peer 163.53.20.1/32 scope global vmbr0
valid_lft forever preferred_lft forever
4: enp6s0.4000@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4000 state UP group default qlen 1000
inet 163.53.20.33 peer 163.53.20.1/32 scope global enp6s0.4000
valid_lft forever preferred_lft forever
5: vmbr4000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 163.53.20.33 peer 163.53.20.1/32 scope global vmbr4000
valid_lft forever preferred_lft forever
6: enp6s0.4001@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4001 state UP group default qlen 1000
inet 163.53.20.33 peer 163.53.20.1/32 scope global enp6s0.4001
valid_lft forever preferred_lft forever
7: vmbr4001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 163.53.20.33 peer 163.53.20.1/32 scope global vmbr4001
valid_lft forever preferred_lft forever

on slave1:

root@slave1 ~ # ip address show to 55.35.31.182/32 up
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
inet 55.35.31.182 peer 55.35.31.129/32 scope global enp6s0
valid_lft forever preferred_lft forever
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 55.35.31.182 peer 55.35.31.129/32 brd 55.35.31.182 scope global vmbr0
valid_lft forever preferred_lft forever
4: enp6s0.4000@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4000 state UP group default qlen 1000
inet 55.35.31.182 peer55.35.31.129/32 scope global enp6s0.4000
valid_lft forever preferred_lft forever
5: vmbr4000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 55.35.31.182 peer 55.35.31.129/32 scope global vmbr4000
valid_lft forever preferred_lft forever
6: enp6s0.4001@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4001 state UP group default qlen 1000
inet 55.35.31.182 peer 55.35.31.129/32 scope global enp6s0.4001
valid_lft forever preferred_lft forever
7: vmbr4001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 55.35.31.182 peer 55.35.31.129/32 scope global vmbr4001
valid_lft forever preferred_lft forever
 
did you change IPs? the original post said '178.*.*.*' ;) if that address is still valid, the same output for that address would also be helpful, in addition to a complete "ip address show" and "ip r show" output from the first node.
 
Hi fabian,

I veiled the IP's, as they would be publicly available. ;-)

Here the outputs:

root@master ~ # ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether 3c:32:3d:b2:a2:f1 brd ff:ff:ff:ff:ff:ff
inet 6.10.87.55 peer 6.10.87.33/32 scope global enp6s0
valid_lft forever preferred_lft forever
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3c:32:3d:b2:a2:f1 brd ff:ff:ff:ff:ff:ff
inet 6.10.87.55 peer 6.10.87.33/32 brd 6.10.87.55 scope global vmbr0
valid_lft forever preferred_lft forever
4: enp6s0.4000@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4000 state UP group default qlen 1000
link/ether 3c:32:3d:b2:a2:f1 brd ff:ff:ff:ff:ff:ff
inet 6.10.87.55 peer 6.10.87.33/32 scope global enp6s0.4000
valid_lft forever preferred_lft forever
5: vmbr4000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3c:32:3d:b2:a2:f1 brd ff:ff:ff:ff:ff:ff
inet 10.0.100.10/24 brd 10.0.100.255 scope global vmbr4000
valid_lft forever preferred_lft forever
inet 6.10.87.55 peer 6.10.87.33/32 scope global vmbr4000
valid_lft forever preferred_lft forever
6: enp6s0.4001@enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4001 state UP group default qlen 1000
link/ether 3c:32:3d:b2:a2:f1 brd ff:ff:ff:ff:ff:ff
inet 6.10.87.55 peer 6.10.87.33/32 scope global enp6s0.4001
valid_lft forever preferred_lft forever
7: vmbr4001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3c:32:3d:b2:a2:f1 brd ff:ff:ff:ff:ff:ff
inet 6.10.87.55 peer 6.10.87.33/32 scope global vmbr4001
valid_lft forever preferred_lft forever
inet 10.0.200.10/24 brd 10.0.200.255 scope global vmbr4001
valid_lft forever preferred_lft forever


root@master ~ # ip r show
default via 6.10.87.33 dev vmbr0 onlink
default via 6.10.87.33 dev enp6s0 proto static
default via 6.10.87.33 dev vmbr0 proto static
default via 6.10.87.33 dev enp6s0.4000 proto static
default via 6.10.87.33 dev enp6s0.4001 proto static
default via 6.10.87.33 dev vmbr4000 proto static
default via 6.10.87.33 dev vmbr4001 proto static
6.10.87.33 dev vmbr0 proto kernel scope link src 6.10.87.55
6.10.87.33 dev enp6s0 proto kernel scope link src 6.10.87.55
6.10.87.33 dev enp6s0.4000 proto kernel scope link src 6.10.87.55
6.10.87.33 dev enp6s0.4001 proto kernel scope link src 6.10.87.55
6.10.87.33 dev vmbr4000 proto kernel scope link src 6.10.87.55
6.10.87.33 dev vmbr4001 proto kernel scope link src 6.10.87.55
10.0.100.0/24 dev vmbr4000 proto kernel scope link src 10.0.100.10
10.0.200.0/24 dev vmbr4001 proto kernel scope link src 10.0.200.10
 
According to the "ip address show" you posted, your local IP (6.10.87.55) is indeed configured on multiple interfaces (on vmbr0, as well as the parent physical interface and all VLAN interfaces), which is why the check fails. Is that intentional?

I tried to reproduce your setup as best as I could, but never quite got that behaviour. Are you sure you don't have some other network setup going on? Maybe a script somewhere messing your configuration up, or some files in "/etc/network/interfaces.d/"?

For reference, here is how your config looks on my machine:

Code:
auto lo
iface lo inet loopback

iface ens18 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.26.225
        netmask 255.255.240.0
        gateway 192.168.16.1
        bridge_ports ens18
        bridge_stp off
        bridge_fd 0

iface ens19 inet manual

auto vmbr1
iface vmbr1 inet static
        address 6.10.87.55
        netmask 255.255.255.255
        gateway 6.10.87.33
        bridge-ports ens19
        bridge-stp off
        bridge-fd 0
        pointopoint 6.10.87.33

post-up echo 2048 > /sys/class/net/vmbr1/bridge/hash_max
post-up echo 1 > /sys/class/net/vmbr1/bridge/multicast_snooping

up ip route add -net 6.10.87.33 netmask 255.255.255.224 gw 6.10.87.33 vmbr1
up sysctl -w net.ipv4.ip_forward=1
up sysctl -w net.ipv4.conf.ens19.send_redirects=0
up sysctl -w net.ipv6.conf.all.forwarding=1

# vlan raw device
auto ens19.4000
iface ens19.4000 inet static
        vlan-raw-device ens19
        mtu 1400

# vlan
auto vmbr4000
iface vmbr4000 inet static
        bridge_ports ens19.4000
        bridge_stp off
        bridge_fd 0
        address 10.0.100.10
        netmask 24

Code:
~ root@test1
❯ ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether fe:0e:bb:00:bc:3b brd ff:ff:ff:ff:ff:ff
3: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UP group default qlen 1000
    link/ether 7a:e9:6f:86:55:12 brd ff:ff:ff:ff:ff:ff
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:0e:bb:00:bc:3b brd ff:ff:ff:ff:ff:ff
    inet 192.168.26.225/20 brd 192.168.31.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::fc0e:bbff:fe00:bc3b/64 scope link
       valid_lft forever preferred_lft forever
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:e9:6f:86:55:12 brd ff:ff:ff:ff:ff:ff
    inet 6.10.87.55 peer 6.10.87.33/32 brd 6.10.87.55 scope global vmbr1
       valid_lft forever preferred_lft forever
    inet6 fe80::78e9:6fff:fe86:5512/64 scope link
       valid_lft forever preferred_lft forever
6: ens19.4000@ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr4000 state UP group default qlen 1000
    link/ether 7a:e9:6f:86:55:12 brd ff:ff:ff:ff:ff:ff
7: vmbr4000: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 7a:e9:6f:86:55:12 brd ff:ff:ff:ff:ff:ff
    inet 10.0.100.10/24 brd 10.0.100.255 scope global vmbr4000
       valid_lft forever preferred_lft forever
    inet6 fe80::78e9:6fff:fe86:5512/64 scope link
       valid_lft forever preferred_lft forever

~ root@test1
❯ ip address show to 6.10.87.55/32 up
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 6.10.87.55 peer 6.10.87.33/32 brd 6.10.87.55 scope global vmbr1
       valid_lft forever preferred_lft forever
 
Hi Stefan,

Thank you for your support.

At Hetzner unfortunately everything is running on one network port, so this is intended.

I checked the directory /etc/network/interfaces.d and the directory is empty.

Best regards,
Marcel
 
So just to confirm: Your setup works this way? VMs running on any vmbr can access everything they are supposed to?

I'm still confused why your interfaces are all assigned the IPs they are. Again, if I run your "/etc/network/interfaces" I only get the 6.10.87.55 IP on the vmbr1 setup. Ignoring the vmbr0 interface in what I posted, that's just for me to access the machine, I'm effectively also using only a single network port.
 
I do not currently have any VMs up and running, the first thing I wanted to do is have a working cluster.

However, it still fails at the moment, said error.
The 3 servers are completely new and work perfectly otherwise.
 
Hi,

i found the problem!

If you install the Proxmox Server over Hetzner, the network is configurated over /etc/systemd/network.
My problem was that I didn't deactivate these settings.

Thank you very much for your help!!!

Marcel
 
  • Like
Reactions: fabian

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!