PVE 8 to 9 Unable to reach Container(s)

J

jaapmarcus

Member
Proxmox Subscriber
Jul 2, 2021
4
0
21
41
After upgrade from PVE 8 to 9 I am unable to reach both containers / or ping from the containers externally

On my host:
ip a
```
root@server01 /etc/network # ip a


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000


link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00


inet 127.0.0.1/8 scope host lo


valid_lft forever preferred_lft forever


inet6 ::1/128 scope host noprefixroute


valid_lft forever preferred_lft forever


2: nic0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000


link/ether a8:a1:59:3b:1a:27 brd ff:ff:ff:ff:ff:ff


altname enxa8a1593b1a27


inet 5.xxx.xxx.213/27 scope global nic0


valid_lft forever preferred_lft forever


inet6 2a01:xxxx:160:6450::1/128 scope global


valid_lft forever preferred_lft forever


inet6 fe80::aaa1:59ff:fe3b:1a27/64 scope link proto kernel_ll


valid_lft forever preferred_lft forever


3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000


link/ether 6a:f5:19:4e:f4:01 brd ff:ff:ff:ff:ff:ff


inet 5.xxx.xxx.213/32 scope global vmbr0


valid_lft forever preferred_lft forever


inet6 2a01:xxx:160:6450::2/64 scope global


valid_lft forever preferred_lft forever


inet6 fe80::7873:4ff:feec:9fbe/64 scope link proto kernel_ll


valid_lft forever preferred_lft forever


4: veth100i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000


link/ether fe:16:98:60:30:ce brd ff:ff:ff:ff:ff:ff link-netnsid 0


5: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000


link/ether c6:c5:27:a4:85:b4 brd ff:ff:ff:ff:ff:ff


6: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000


link/ether 6a:f5:19:4e:f4:01 brd ff:ff:ff:ff:ff:ff


7: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000


link/ether c6:c5:27:a4:85:b4 brd ff:ff:ff:ff:ff:ff


8: veth101i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000


link/ether fe:d9:8a:c5:3b:58 brd ff:ff:ff:ff:ff:ff link-netnsid 1


9: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000


link/ether 66:5d:eb:52:29:f3 brd ff:ff:ff:ff:ff:ff


10: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000


link/ether 86:d7:d7:ca:52:1b brd ff:ff:ff:ff:ff:ff


11: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000


link/ether 66:5d:eb:52:29:f3 brd ff:ff:ff:ff:ff:ff
```

/etc/network/interfaces/
```
auto lo

iface lo inet loopback

iface lo inet6 loopback

auto nic0

iface nic0 inet static

address 5.xxx.xxx.213/27
gateway 5.xxx.xxx.193

iface nic0 inet6 static
address 2a01:xxx:160:6450::1/128
gateway fe80::1


auto vmbr0
iface vmbr0 inet static
address 5.xxx.xxx.213/32
bridge-ports none
bridge-stp off
bridge-fd 0
up ip route add 5.xxx.xxx.209/32 dev vmbr0
up ip route add 5.xxx.xxx.220/32 dev vmbr0
up ip route add 5..xxx.xxx.251/32 dev vmbr0


iface vmbr0 inet6 static
address 2a01:xxx:160:6450::2/64


```

VM:
```
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether bc:24:11:e8:a0:87 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 5.xxx.xxx.220/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 2a01:xxxx:160:6450::7/128 scope global
valid_lft forever preferred_lft forever
inet6 fe80::be24:11ff:fee8:a087/64 scope link
valid_lft forever preferred_lft forever
```

I can access the host without any issues but not the VM's

any idea what is going wrong?
 
Seems like you have a routed setup for your VMs.

is IP forwarding and proxy_arp active?

Code: 
cat /proc/sys/net/ipv4/ip_forward
cat /proc/sys/net/ipv4/conf/nic0/proxy_arp


Is the /32 configured on the vmbr0 on the same subnet as nic0? is this a Hetzner Server by chacne? If yes, why dont you just configure the /27 Subnet on the bridge itself, remove the additional /32 routes from the configuration and use a bridged setup. you might need to register the MAC addresses of your VM in the Control Panel, if you're using Hetzner.
 
Last edited:
shanreich said:
Seems like you have a routed setup for your VMs.

is IP forwarding and proxy_arp active?

Code: 
cat /proc/sys/net/ipv4/ip_forward
cat /proc/sys/net/ipv4/conf/nic0/proxy_arp


Is the /32 configured on the vmbr0 on the same subnet as nic0? is this a Hetzner Server by chacne? If yes, why dont you just configure the /27 Subnet on the bridge itself, remove the additional /32 routes from the configuration and use a bridged setup. you might need to register the MAC addresses of your VM in the Control Panel, if you're using Hetzner.
Click to expand...
Yes it is Hetzner:

```
root@server01 /etc/network # cat /proc/sys/net/ipv4/ip_forward
0


root@server01 /etc/network # cat /proc/sys/net/ipv4/conf/nic0/proxy_arp
0
````

It might be overwritten by the upgrade
 
Does activating them fix the problem? Setting them in the procfs is not persistent, so it's possible they were only set temporarily and the settings got lost on reboot, since there are no post-up directives in /etc/network/interfaces (although it would be possible to persist them via other means).

Can you try if enabling both fixes the setup?
Code: 
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/nic0/proxy_arp

If that fixes the problem, persist the settings by adding the following lines to your /etc/network/interfaces file in the nic0 section:

Code: 
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/nic0/proxy_arp


Otherwise, set them to 0 again:\
Code: 
echo 0 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/nic0/proxy_arp
 
shanreich said:
Does activating them fix the problem? Setting them in the procfs is not persistent, so it's possible they were only set temporarily and the settings got lost on reboot, since there are no post-up directives in /etc/network/interfaces (although it would be possible to persist them via other means).

Can you try if enabling both fixes the setup?
Code: 
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/nic0/proxy_arp

If that fixes the problem, persist the settings by adding the following lines to your /etc/network/interfaces file in the nic0 section:

Code: 
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/nic0/proxy_arp


Otherwise, set them to 0 again:\
Code: 
echo 0 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/nic0/proxy_arp
Click to expand...
Yes it fixes the issue.
 
You must log in or register to reply here.
Top Bottom