[SOLVED] [WORKAROUND FOUND] Routing issues probably self-inflicted. Amateur needs help. Only for patient users.

AndrzejL

New Member
Nov 8, 2020
5
1
1
Hello everyone.

Ugh... I'm afraid to even ask the question... Just to be fair - I am an amateur - I don't even know what I don't know...

Ok so I am setting up proxmox but its "complicated".

- Normally proxmox runs with LAN device as a main network (internet) interface - in my case its wireless.

- Wireless network runs as 192.168.1.10 wlp3s0.
- There is a LAN interface present enp0s25 its set 192.168.0.1
- There is a vmbr0 which is also bridged with wlp3s0 device running with 192.168.2.10

- I have post-up rules in my /etc/network/interfaces that are setting up the masquerading and enable ip forwarding - the rules look like this

Code:
post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
post-up   iptables -t nat -A POSTROUTING -s '192.168.0.0/24' -o wlp3s0 -j MASQUERADE
post-up   iptables -t nat -A POSTROUTING -s '192.168.2.0/24' -o wlp3s0 -j MASQUERADE

Thanks to those I have internet on both LAN and VM machines. Before I added those - I had neither.

Well... the devices from the enp0s25 side can talk to the vmbr0 devices so for example if I ssh from 192.168.0.104 to 192.168.2.12 vm - it works. I can also connect to the wlp3s0 proxmox ssh on 192.168.1.10

Wireless devices on my network running 192.168.1.x IP can access wlp3s0 ssh on 192.168.1.10 - that's the proxmox server BUT they cannot access any of the vmbr0 devices - as in the vms

So for example laptop connected wirelessly as 192.168.1.40 can ssh to the wlp3s0 192.168.1.10 (proxmox server) but it cannot ssh to the 192.168.2.12 - vm.
In the same time desktop connected via cable as 192.168.0.104 can ssh to both of the above mentioned ssh servers.

I am assuming that I need to add routing rule so that the wlp3s0 192.168.1.10 device lets the requests from 192.168.1.x devices to the vmbr0 192.168.2.x devices correct?

I am not even sure if its my proxmox routing that's the issue or my modem / router from the ISP that's blocking the connections between the sub-networks

What I would like to achieve is have proxmox access internet via wlan and then spread that internet to the virtual machines using vmbr0 device and to the LAN machines on enp0s25 - so that all the networks 192.168.0.x 192.168.1.x and 192.168.2.x are talking to each other.

Before you ask me - its a home server, not work / school related. I am not looking for someone to do my homework, I am trying to learn / get it going for home purposes

Thanks in advance for any help provided.

Kind regards.

Andrzej

Some information / config files@

Code:
root@andrzejl:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.254   0.0.0.0         UG    0      0        0 wlp3s0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 enp0s25
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 wlp3s0
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 vmbr0

Code:
root@andrzejl:~# ping google.com
PING google.com (74.125.193.138) 56(84) bytes of data.
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=1 ttl=57 time=13.4 ms
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=2 ttl=57 time=14.0 ms
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=3 ttl=57 time=13.8 ms
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=4 ttl=57 time=14.2 ms
^C
--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 13.357/13.837/14.156/0.332 ms

Code:
root@andrzejl:~# iwconfig
tap101i0  no wireless extensions.

wlp3s0    IEEE 802.11  ESSID:"redacted"
          Mode:Managed  Frequency:5.5 GHz  Access Point: 24:1F:A0:70:F7:60
          Bit Rate=43.3 Mb/s   Tx-Power=17 dBm
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Encryption key:off
          Power Management:off
          Link Quality=43/70  Signal level=-67 dBm
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:19   Missed beacon:0

lo        no wireless extensions.

vmbr0     no wireless extensions.

enp0s25   no wireless extensions.

Code:
root@andrzejl:~# ifconfig
enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.1  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::1a03:73ff:fe46:a74d  prefixlen 64  scopeid 0x20<link>
        ether 18:03:73:46:a7:4d  txqueuelen 1000  (Ethernet)
        RX packets 42075  bytes 5270878 (5.0 MiB)
        RX errors 0  dropped 8690  overruns 0  frame 0
        TX packets 65663  bytes 77232845 (73.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xe1b00000-e1b20000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 956  bytes 301514 (294.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 956  bytes 301514 (294.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap101i0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether be:06:01:d2:d0:fc  txqueuelen 1000  (Ethernet)
        RX packets 7840  bytes 44776910 (42.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 17452  bytes 1326286 (1.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vmbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.10  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::5038:4aff:fee5:eccb  prefixlen 64  scopeid 0x20<link>
        ether be:06:01:d2:d0:fc  txqueuelen 1000  (Ethernet)
        RX packets 7840  bytes 44667150 (42.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 17460  bytes 1326994 (1.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.10  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 50:b7:c3:43:d5:0c  txqueuelen 1000  (Ethernet)
        RX packets 32313  bytes 30434760 (29.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 22235  bytes 3321427 (3.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Code:
root@andrzejl:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

allow-hotplug wlp3s0
iface wlp3s0 inet dhcp
        wpa-ssid redacted
        wpa-psk redacted

auto vmbr0
iface vmbr0 inet static
        address  192.168.2.10
        netmask  255.255.255.0
        bridge-ports wlp3s0
        bridge-stp off
        bridge-fd 0

allow-hotplug enp0s25
iface enp0s25 inet static
        address  192.168.0.1
        netmask  255.255.255.0
        bridge-ports wlp3s0
        bridge-stp off
        bridge-fd 0

        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '192.168.0.0/24' -o wlp3s0 -j MASQUERADE
        post-up   iptables -t nat -A POSTROUTING -s '192.168.2.0/24' -o wlp3s0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '192.168.0.0/24' -o wlp3s0 -j MASQUERADE
        post-down iptables -t nat -A POSTROUTING -s '192.168.2.0/24' -o wlp3s0 -j MASQUERADE

Code:
root@andrzejl:~# cat /etc/wpa_supplicant/wpa_supplicant.conf
ctrl_interface=/var/run/wpa_supplicant
ap_scan=1
update_config=1
network={
        ssid="redacted"
        psk=redacted
}

Code:
root@andrzejl:~# brctl show
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.be0601d2d0fc       no              tap101i0

Code:
root@andrzejl:~# brctl show vmbr0
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.be0601d2d0fc       no              tap101i0
 
Last edited:

AndrzejL

New Member
Nov 8, 2020
5
1
1
Hi all

Just to add:

1604854141056.png

My router cannot ping the vmbr0 device either. I think there would be two reasons. Either the router is not great - wouldn't surprised me - and it basically cannot route the packets properly OR and I am hoping this is the case - the wlp3s0 (proxmox wireless device) is not routing packets properly - this would possibly be an easy fix for someone who know what they are doing (diagnostics / iptablesfoo).

Regards.

Andrzej
 

AndrzejL

New Member
Nov 8, 2020
5
1
1
I ran this on the proxmox machine

Code:
iptables -A FORWARD -i wlp3s0 -o vmbr0 -s 192.168.1.0/24 -d 192.168.2.0/24 -j ACCEPT
iptables -A FORWARD -i vmbr0 -o wlp3s0 -s 192.168.2.0/24 -d 192.168.1.0/24 -j ACCEPT
iptables -A FORWARD -i wlp3s0 -o vmbr0 -s 192.168.1.10 -d 192.168.2.0/24 -j ACCEPT
iptables -A FORWARD -i vmbr0 -o wlp3s0 -s 192.168.2.0/24 -d 192.168.1.10 -j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.2.0/24 -o wlp3s0 -j SNAT --to 192.168.1.1

Code:
root@andrzejl:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A FORWARD -s 192.168.1.0/24 -d 192.168.2.0/24 -i wlp3s0 -o vmbr0 -j ACCEPT
-A FORWARD -s 192.168.2.0/24 -d 192.168.1.0/24 -i vmbr0 -o wlp3s0 -j ACCEPT
-A FORWARD -s 192.168.1.10/32 -d 192.168.2.0/24 -i wlp3s0 -o vmbr0 -j ACCEPT
-A FORWARD -s 192.168.2.0/24 -d 192.168.1.10/32 -i vmbr0 -o wlp3s0 -j ACCEPT

Code:
root@andrzejl:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination


Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  192.168.1.0/24       192.168.2.0/24
ACCEPT     all  --  192.168.2.0/24       192.168.1.0/24
ACCEPT     all  --  andrzejl.eu          192.168.2.0/24
ACCEPT     all  --  192.168.2.0/24       andrzejl.eu


Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
root@andrzejl:~#



Didn't help... Any takers?

Cheers.

Andrzej
 
Last edited:

spirit

Famous Member
Apr 2, 2010
4,536
335
103
www.odiso.com
you can't add ip address on a interface enslave in in bridge. It's curious than dhcp is able to give it an ip, but I don't think it's work.

about enp0s25, I don't known what you're trying to dot, but it's not a bridge, so you can't add bridge options to this interfaces.

why don't you do a simple

Code:
auto vmbr0
iface vmbr0 inet static
        address  192.168.2.10
        netmask  255.255.255.0
        bridge-ports wlp3s0 enp0s25
        bridge-stp off
        bridge-fd 0

or with dhcp


Code:
auto vmbr0
iface vmbr0 inet dhcp
        bridge-ports wlp3s0 enp0s25
        bridge-stp off
        bridge-fd 0
 

AndrzejL

New Member
Nov 8, 2020
5
1
1
you can't add ip address on a interface enslave in in bridge. It's curious than dhcp is able to give it an ip, but I don't think it's work.

about enp0s25, I don't known what you're trying to dot, but it's not a bridge, so you can't add bridge options to this interfaces.

why don't you do a simple

Code:
auto vmbr0
iface vmbr0 inet static
        address  192.168.2.10
        netmask  255.255.255.0
        bridge-ports wlp3s0 enp0s25
        bridge-stp off
        bridge-fd 0

or with dhcp


Code:
auto vmbr0
iface vmbr0 inet dhcp
        bridge-ports wlp3s0 enp0s25
        bridge-stp off
        bridge-fd 0

Hi

For some reason your method did not work. After reboot I've had no internet on the enp0s25 device. Possibly on vmbr0 too.

My config at the moment is insane but it works and I've figured out my problem with port forwarding. Well its a workaround but it works so...

Code:
grep -v '#' /etc/network/interfaces

Code:
auto lo
iface lo inet loopback


allow-hotplug wlp3s0
iface wlp3s0 inet dhcp
        wpa-ssid redacted
        wpa-psk redacted


auto vmbr0
iface vmbr0 inet static
        address  192.168.2.10
        netmask  255.255.255.0
        bridge-ports wlp3s0
        bridge-stp off
        bridge-fd 0


allow-hotplug enp0s25
iface enp0s25 inet static
        address  192.168.0.1
        netmask  255.255.255.0
        bridge-ports wlp3s0
        bridge-stp off
        bridge-fd 0
        
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -o wlp3s0 -j MASQUERADE
        post-up iptables -t nat -A PREROUTING -p tcp --dport 50505 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.12
        post-up iptables -A FORWARD -p tcp --dport 50505 -d 192.168.2.12 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 50506 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.13
        post-up iptables -A FORWARD -p tcp --dport 50506 -d 192.168.2.13 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 50507 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.14
        post-up iptables -A FORWARD -p tcp --dport 50507 -d 192.168.2.14 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 631 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.13
        post-up iptables -A FORWARD -p tcp --dport 631 -d 192.168.2.13 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 65432 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.12
        post-up iptables -A FORWARD -p tcp --dport 65432 -d 192.168.2.12 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p udp --dport 1900 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.12
        post-up iptables -A FORWARD -p udp --dport 1900 -d 192.168.2.12 -j ACCEPT
        post-up iptables -i wlp3s0 -I INPUT -p udp -s 192.168.1.12/32 --dport 137 -j ACCEPT
        post-up iptables -i vmbr0 -I INPUT -p udp -s 192.168.2.12/32 --dport 137 -j ACCEPT
        post-up iptables -i enp0s25 -I INPUT -p udp -s 192.168.0.12/32 --dport 137 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 6566 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.13
        post-up iptables -t nat -A PREROUTING -p udp --dport 6566 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.13
        post-up iptables -A FORWARD -p tcp --dport 6566 -d 192.168.2.13 -j ACCEPT
        post-up iptables -A FORWARD -p udp --dport 6566 -d 192.168.2.13 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 1024:1200 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.13
        post-up iptables -t nat -A PREROUTING -p udp --dport 1024:1200 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.13
        post-up iptables -A FORWARD -p tcp --dport 1024:1200 -d 192.168.2.13 -j ACCEPT
        post-up iptables -A FORWARD -p udp --dport 1024:1200 -d 192.168.2.13 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 22000 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.14
        post-up iptables -A FORWARD -p tcp --dport 22000 -d 192.168.2.14 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p udp --dport 21207 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.14
        post-up iptables -A FORWARD -p udp --dport 21207 -d 192.168.2.14 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 8384 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.14
        post-up iptables -A FORWARD -p tcp --dport 8384 -d 192.168.2.14 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p udp --dport 8384 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.14
        post-up iptables -A FORWARD -p udp --dport 8384 -d 192.168.2.14 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 30303 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 50505 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 110 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 110 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 143 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 143 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 465 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 465 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 587 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 587 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 993 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 993 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 995 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 995 -d 192.168.2.50 -j ACCEPT
        post-up iptables -t nat -A PREROUTING -p tcp --dport 4242 -d 192.168.1.10 -j DNAT --to-destination 192.168.2.50
        post-up iptables -A FORWARD -p tcp --dport 4242 -d 192.168.2.50 -j ACCEPT

Now I am having trouble with multicast but that's a story for another place and time.

Regards.

Andrzej
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!