This is 4th try after getting no responses from Stack Exchange sites [1, 2, 3] and I've been fighting with this issue for ~2 weeks. I really hope someone can help me with this issue.
---
I have a setup like this (this diagram is also available here):
I'm running a single PVE host in my homelab with single NIC port. Inside that PVE host, I configured a Ubuntu VM to be a virtual router (I'll this as 'Router VM') for all my VMs. My home network is 192.168.1.0/24 and my VMs network is 10.0.0.0/24. The Router VM is running isc-dhcp-server so it can provide dynamic IP leases to my VMs.
Note: I'm aware of free router software like pfSense, OpenWRT & etc. But I want to take this opportunity to build the router from scratch so I can learn how everything works especially on routing/bridging/iptables side of things because that's my objective with this homelab - to learn deeper about virtualization and networking.
Network configurations in my Proxmox GUI:
Due to physical space issue, I have to place my PVE host in different place (instead of next to my home router) and connects it to a WiFi Repeater via a network cable. This WiFi Repeater is then connected to my home WiFi AP. When I'm connected to my WiFi AP and WiFi Repeater, I'm able to ping the PVE host and Router VM just fine.
Problem: when I ping from my laptop to my VMs while I'm connected to WiFi AP, the ping timed out. But it worked fine when I'm connected to the WiFi Repeater. I have tried two repeaters from different brands but the problem persist. I really wonder what's wrong here. I started to think if this is a misconfiguration issue on my Router VM/PVE host and not repeaters issue.
I tried adding static route/next hop in my home router and also in my laptop by following this guide. Unfortunately, that didn't help.
Both PVE host and Router VM has IP forwarding enabled:
And below are their configurations:
1.
2.
3.
4.
Note: I didn't have any custom iptables rules for Router VM in this PVE host. All of these were added by Tailscale when I configured it for PVE host remote access. I believe Tailscale isn't the cause/issue here because my problem was already there before I installed Tailscale.
5.
6.
7.
8.
9.
10.
11.
12.
What should I add/update so I can ping the VMs when I'm connected to my WiFi AP (not WiFi Repeater)? Is this iptables issue or repeaters issue in general?
---
I have a setup like this (this diagram is also available here):
I'm running a single PVE host in my homelab with single NIC port. Inside that PVE host, I configured a Ubuntu VM to be a virtual router (I'll this as 'Router VM') for all my VMs. My home network is 192.168.1.0/24 and my VMs network is 10.0.0.0/24. The Router VM is running isc-dhcp-server so it can provide dynamic IP leases to my VMs.
Note: I'm aware of free router software like pfSense, OpenWRT & etc. But I want to take this opportunity to build the router from scratch so I can learn how everything works especially on routing/bridging/iptables side of things because that's my objective with this homelab - to learn deeper about virtualization and networking.
Network configurations in my Proxmox GUI:
Due to physical space issue, I have to place my PVE host in different place (instead of next to my home router) and connects it to a WiFi Repeater via a network cable. This WiFi Repeater is then connected to my home WiFi AP. When I'm connected to my WiFi AP and WiFi Repeater, I'm able to ping the PVE host and Router VM just fine.
Problem: when I ping from my laptop to my VMs while I'm connected to WiFi AP, the ping timed out. But it worked fine when I'm connected to the WiFi Repeater. I have tried two repeaters from different brands but the problem persist. I really wonder what's wrong here. I started to think if this is a misconfiguration issue on my Router VM/PVE host and not repeaters issue.
I tried adding static route/next hop in my home router and also in my laptop by following this guide. Unfortunately, that didn't help.
Both PVE host and Router VM has IP forwarding enabled:
Code:
$ cat /proc/sys/net/ipv4/ip_forward
1And below are their configurations:
1.
/etc/network/interfaces in PVE host
Code:
auto lo
iface lo inet loopback
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.1.5/24
gateway 192.168.1.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 02.
ip a in PVE host
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether fc:4d:d4:f3:fe:d3 brd ff:ff:ff:ff:ff:ff
altname enp0s25
3: wlp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether ba:65:34:a3:09:60 brd ff:ff:ff:ff:ff:ff permaddr 28:c2:dd:a5:46:53
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether fc:4d:d4:f3:fe:d3 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.5/24 scope global vmbr0
valid_lft forever preferred_lft forever
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8e:7e:21:7e:f9:e1 brd ff:ff:ff:ff:ff:ff
6: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 100.111.63.28/32 scope global tailscale0
valid_lft forever preferred_lft forever
7: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
link/ether 62:27:fd:32:26:0c brd ff:ff:ff:ff:ff:ff
8: tap100i1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN group default qlen 1000
link/ether 62:c8:a9:96:dc:7e brd ff:ff:ff:ff:ff:ff
9: tap101i1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN group default qlen 1000
link/ether fa:35:bb:a9:7d:b8 brd ff:ff:ff:ff:ff:ff
10: tap102i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN group default qlen 1000
link/ether 46:9c:a4:fd:81:27 brd ff:ff:ff:ff:ff:ff
11: tap103i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr1 state UNKNOWN group default qlen 1000
link/ether 12:90:ee:36:27:7a brd ff:ff:ff:ff:ff:ff3.
iptables -S in PVE host
Code:
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N ts-forward
-N ts-input
-A INPUT -j ts-input
-A FORWARD -j ts-forward
-A ts-forward -i tailscale0 -j MARK --set-xmark 0x40000/0xff0000
-A ts-forward -m mark --mark 0x40000/0xff0000 -j ACCEPT
-A ts-forward -s 100.64.0.0/10 -o tailscale0 -j DROP
-A ts-forward -o tailscale0 -j ACCEPT
-A ts-input -s 100.111.63.28/32 -i lo -j ACCEPT
-A ts-input -s 100.115.92.0/23 ! -i tailscale0 -j RETURN
-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP4.
iptables -S -t nat in PVE host
Code:
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N ts-postrouting
-A POSTROUTING -j ts-postrouting
-A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADENote: I didn't have any custom iptables rules for Router VM in this PVE host. All of these were added by Tailscale when I configured it for PVE host remote access. I believe Tailscale isn't the cause/issue here because my problem was already there before I installed Tailscale.
5.
ip a in Router VM
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 2a:06:12:a9:18:98 brd ff:ff:ff:ff:ff:ff
altname enp0s18
inet 192.168.1.6/24 brd 192.168.1.255 scope global ens18
valid_lft forever preferred_lft forever
inet6 fe80::2806:12ff:fea9:1898/64 scope link
valid_lft forever preferred_lft forever
3: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether ca:bc:10:17:e0:2f brd ff:ff:ff:ff:ff:ff
altname enp0s19
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether da:56:23:09:09:72 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/24 brd 10.0.0.255 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::d856:23ff:fe09:972/64 scope link
valid_lft forever preferred_lft forever6.
/etc/netplan/00-installer-config.yaml in Router VM
Code:
network:
version: 2
renderer: networkd
ethernets:
ens18:
dhcp4: no
addresses: [192.168.1.6/24]
gateway4: 192.168.1.1
nameservers:
addresses: [8.8.8.8,8.8.4.4]7.
/etc/systemd/network/uplink.network in Router VM
Code:
[Match]
Name=ens19
[Network]
Bridge=br08.
/etc/systemd/network/br0.netdev in Router VM
Code:
[NetDev]
Name=br0
Kind=bridge9.
/etc/systemd/network/br0.network in Router VM
Code:
[Match]
Name=br0
[Network]
DNS=8.8.8.8
Address=10.0.0.1/24
Gateway=10.0.0.110.
iptables -S in Router VM
Code:
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -s 127.0.0.0/8 -d 127.0.0.0/8 -i lo -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A INPUT -m state --state ESTABLISHED -j ACCEPT
-A INPUT -p udp -m udp --dport 33434:33523 -j REJECT --reject-with icmp-port-unreachable
-A INPUT -i br0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i br0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i br0 -p udp -m udp --dport 67:68 -j ACCEPT
-A INPUT -i ens18 -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -j DROP
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.0.0.0/24 -d 192.168.1.0/24 -i br0 -o ens18 -j DROP
-A FORWARD -i br0 -o ens18 -j ACCEPT
-A FORWARD -d 10.0.0.0/24 -p icmp -j ACCEPT
-A FORWARD -d 10.0.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A FORWARD -d 10.0.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A FORWARD -j DROP11.
iptables -S -t nat in Router VM
Code:
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -o ens18 -j MASQUERADE12.
/etc/dhcp/dhcpd.conf in Router VM
Code:
option domain-name "example.org";
option domain-name-servers ns1.example.org, ns2.example.org;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style none;
subnet 10.0.0.0 netmask 255.255.255.0 {
range 10.0.0.35 10.0.0.39;
option routers 10.0.0.1;
option domain-name-servers 8.8.8.8;
option broadcast-address 10.0.0.255;
host ubuntu-desktop {
hardware ethernet 5A:5B:74:01:69:34;
fixed-address 10.0.0.3;
}
}What should I add/update so I can ping the VMs when I'm connected to my WiFi AP (not WiFi Repeater)? Is this iptables issue or repeaters issue in general?
