Cannot get networking to work

[micha11]

Hi,

I have a new proxmox installation and try to get the first vm running. Unfortunately I cannot get the networking to run.

Suggested goal:
Proxmox running on 10.10.13.10/24
VMs running on 10.10.15.0/24 and 10.10.14.0/24

• I set up a 10.10.15.0/24 net and assigned an ip to a normal laptop - network is running (e.g. internet router etc. - gateways are listening on 10.10.x.254)
• Internet access from proxmox vm is also working

As a vom on 10.10.15.x did not reach the network I tried to set up a vm on the 10.,10.13.0 network (using vmbr11 - also tried with vmbr13) - but even this vm is not able to ping any host (nor reach the debian installation mirror)

Here is my /etc/network/interfaces:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet static
        address 10.10.13.10/24
        gateway 10.10.13.254
        subnetmask 255.255.255.0

iface eno2 inet manual

auto vmbr11
iface vmbr11 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr14
iface vmbr14 inet static
        address 10.10.14.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr15
iface vmbr15 inet static
        address 10.10.15.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr13
iface vmbr13 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

ip route on proxmox host:

>ip route
default via 10.10.13.254 dev eno1 proto kernel onlink
10.10.13.0/24 dev eno1 proto kernel scope link src 10.10.13.10
10.10.14.0/24 dev vmbr14 proto kernel scope link src 10.10.14.1
10.10.15.0/24 dev vmbr15 proto kernel scope link src 10.10.15.1

>ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether ac:1f:6b:ee:da:8a brd ff:ff:ff:ff:ff:ff
    inet 10.10.13.10/24 scope global eno1
       valid_lft forever preferred_lft forever
    inet6 fe80::ae1f:6bff:feee:da8a/64 scope link
       valid_lft forever preferred_lft forever
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether ac:1f:6b:ee:da:8b brd ff:ff:ff:ff:ff:ff
4: vmbr11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 8a:ee:89:36:16:4d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::88ee:89ff:fe36:164d/64 scope link
       valid_lft forever preferred_lft forever
5: vmbr14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether e2:4b:15:21:ae:90 brd ff:ff:ff:ff:ff:ff
    inet 10.10.14.1/24 scope global vmbr14
       valid_lft forever preferred_lft forever
    inet6 fe80::e04b:15ff:fe21:ae90/64 scope link
       valid_lft forever preferred_lft forever
6: vmbr15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 62:a5:b3:d8:cc:f0 brd ff:ff:ff:ff:ff:ff
    inet 10.10.15.1/24 scope global vmbr15
       valid_lft forever preferred_lft forever
    inet6 fe80::60a5:b3ff:fed8:ccf0/64 scope link
       valid_lft forever preferred_lft forever
7: vmbr13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 66:62:7c:bd:c8:6a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::c825:feff:fec9:9547/64 scope link
       valid_lft forever preferred_lft forever
8: tap102i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr13 state UNKNOWN group default qlen 1000
    link/ether 66:62:7c:bd:c8:6a brd ff:ff:ff:ff:ff:ff

Ethernet Card is a Intel I210

It seems that I am missing something but cannot get what it is.

Could someone please help me?

Regards

Michael

 

[bobmc]

The most obvious problem with your config is that the 'isolated' networks have no gateway defined.

I would suggest that the easiest way to make this work would be to install a vm with pfsense or similar, and let that act as a network router for the various networks you are running.

 

[micha11]

Hi,

the isolated networks have gateways in the hardware-router defined. These are working (as checked with a laptop).

To eliminate the routing issue I tried to set up a vm on the same network as the proxmox host is running (using vmbr13). That should work - or what exactly is missing there? I tried the real gateway (10.10.13.254 as well as the proxmox host 10.10.13.10 as gateway inside the vm - neither worked)

Regards Michael

 

[micha11]

Add: I cannot set a gateway on the vmbr-interfaces as proxmox rejects that because of a defined geateway on eno1

 

[bobmc]

Try adding to your /etc/network/interfaces

post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s '10.10.14.0/24' -o eno1 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s '10.10.14.0/24' -o eno1 -j MASQUERADE post-up iptables -t nat -A POSTROUTING -s '10.10.15.0/24' -o eno1 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s '10.10.15.0/24' -o eno1 -j MASQUERADE

 

[micha11]

As expected this is not the solution as this just adds routes for the two other networks (which may be needed later on). This has no effect one the 10.10.13.99-vm

Currently I am trying to get a vm on the 10.10.13.0/24 net working (= same net as the proxmox host). To keep things simple I would like to first fix this simple setup before going on to the multi-network stuff.

 

[bobmc]

Can the proxmox host itself reach the internet?

 

[micha11]

yes the proxmox server itself is completly "online" and can download files :

root@proxmox:/tmp# wget https://forum.proxmox.com/styles/uix/images/Proxmox_logo_standard_hex_500px.png
--2022-01-06 17:40:26--  https://forum.proxmox.com/styles/uix/images/Proxmox_logo_standard_hex_500px.png
Auflösen des Hostnamens forum.proxmox.com (forum.proxmox.com)… 79.133.36.245, 2a01:7e0:0:424::4
Verbindungsaufbau zu forum.proxmox.com (forum.proxmox.com)|79.133.36.245|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: 9825 (9,6K) [image/png]
Wird in »Proxmox_logo_standard_hex_500px.png« gespeichert.

Proxmox_logo_standard_hex_500px.png                  100%[=====================================================================================================================>]   9,59K  --.-KB/s    in 0,001s

2022-01-06 17:40:26 (12,7 MB/s) - »Proxmox_logo_standard_hex_500px.png« gespeichert [9825/9825]

 

[bobmc]

I think that because you have assigned a static IP to eno1 and this is the only physical nic you have connected you will struggle to get VM's on the same network.

Normally, vmbr0 is bridged to eno1 and the static IP and gateway of the host is assigned to this, then you simply create VM's and Containers on vmbr0 and all works quite simply and easily

 

[micha11]

If I remove the static ip from eno1 and move it to the vmbr I cannot access the host anymore from outside (the ui of proxmox)

 

[micha11]

I activated the second nic (eno2) and assigned the vmbr to this one - this works!

The setup seems currently to be sufficient
Thanks!