Hello,
I have some networking issues that I would need help for.
I added a diagram of my network configuration.
My goal is to use VM A on Proxmox 2 as a reverse proxy for every VM on both my Proxmox servers.
My main router is my Freebox, on 192.168.100.254/24.
Proxmox 1 is using the subnet 192.168.100.0/24 for the VMs. 192.168.1.200/24 is its IP for the LAN.
Proxmox 2 (hostname miniprox) is using the subnet 192.168.101.0/24 for the VMs. 192.168.1.201/24 is its IP for the LAN.
Currently, VM A is able to connect to some services but not all. For example, docker containers on 192.168.100.205/24 are OK, as are the docker containers on 192.168.100.11/24. But it can't access the main service on http://192.168.100.205:80 (502 Bad Gateway).
It also can't access VM 1 (if it matters, it's a LXC container) at all.
It can however ping all the VMs, even those it can't access with the reverse proxy.
The reverse proxy I use is nginx proxy manager, in a docker container.
VM 3 is the reverse proxy I want to replace with VM A. VM 3 works fine for every service.
Here are my Proxmox servers interfaces configuration :
Proxmox 1 :
Proxmox 2 :
On my VM A :
On my VM 1 :
I tried to diagnose the network using tcpdump but I can't find how to fix the issue, even when seing the RESET flags.
Here is the tcpdump when trying to access VM1.domain.tld (pointing to 192.168.101.12), the reverse proxy then trying to redirect to 192.168.100.3 :
On VM A :
On Proxmox 2 :
On Proxmox 1 :
On VM 1 :
Here is the iptables I'm using for forwarding :
There is also a weird behaviour where I can't access anymore http://192.168.100.205 when redirecting all 80/443 traffic from my Freebox to my Proxmox 2 server. I don't know if it matters in all of this.
Thanks a lot for your help
I have some networking issues that I would need help for.
I added a diagram of my network configuration.
My goal is to use VM A on Proxmox 2 as a reverse proxy for every VM on both my Proxmox servers.
My main router is my Freebox, on 192.168.100.254/24.
Proxmox 1 is using the subnet 192.168.100.0/24 for the VMs. 192.168.1.200/24 is its IP for the LAN.
Proxmox 2 (hostname miniprox) is using the subnet 192.168.101.0/24 for the VMs. 192.168.1.201/24 is its IP for the LAN.
Currently, VM A is able to connect to some services but not all. For example, docker containers on 192.168.100.205/24 are OK, as are the docker containers on 192.168.100.11/24. But it can't access the main service on http://192.168.100.205:80 (502 Bad Gateway).
It also can't access VM 1 (if it matters, it's a LXC container) at all.
It can however ping all the VMs, even those it can't access with the reverse proxy.
The reverse proxy I use is nginx proxy manager, in a docker container.
VM 3 is the reverse proxy I want to replace with VM A. VM 3 works fine for every service.
Here are my Proxmox servers interfaces configuration :
Proxmox 1 :
Code:
root@proxmox:~# ip a
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr0 state UP group default qlen 1000
link/ether 18:c0:4d:5e:4d:77 brd ff:ff:ff:ff:ff:ff
altname enx18c04d5e4d77
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 18:c0:4d:5e:4d:77 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.200/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::1ac0:4dff:fe5e:4d77/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
4: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 82:e1:5b:00:6b:35 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.1/24 scope global vmbr2
valid_lft forever preferred_lft forever
inet6 fe80::78dd:a8ff:fe30:8975/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
root@proxmox:~# ip route
default via 192.168.1.254 dev vmbr0 proto kernel onlink
192.168.1.0/24 dev vmbr0 proto kernel scope link src 192.168.1.200
192.168.100.0/24 dev vmbr2 proto kernel scope link src 192.168.100.1
192.168.100.2 via 192.168.1.201 dev vmbr0
192.168.101.0/24 via 192.168.1.201 dev vmbr0Proxmox 2 :
Code:
root@miniprox:~# ip a
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr0 state UP group default qlen 1000
link/ether e0:51:d8:1b:fe:1f brd ff:ff:ff:ff:ff:ff
altname enxe051d81bfe1f
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether e0:51:d8:1b:fe:1f brd ff:ff:ff:ff:ff:ff
inet 192.168.1.201/24 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::e251:d8ff:fe1b:fe1f/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
6: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 86:6c:a1:42:a8:94 brd ff:ff:ff:ff:ff:ff
inet 192.168.101.1/24 scope global vmbr1
valid_lft forever preferred_lft forever
inet6 fe80::4807:f7ff:fe31:b67e/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
7: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether e2:83:36:9d:3b:aa brd ff:ff:ff:ff:ff:ff
inet 192.168.100.2/24 scope global vmbr2
valid_lft forever preferred_lft forever
inet6 fe80::444b:84ff:fe76:29ae/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
root@miniprox:~# ip route
default via 192.168.1.254 dev vmbr0 proto kernel onlink
192.168.1.0/24 dev vmbr0 proto kernel scope link src 192.168.1.201
192.168.100.0/24 via 192.168.1.200 dev vmbr0
192.168.101.0/24 dev vmbr1 proto kernel scope link src 192.168.101.1On my VM A :
Code:
root@docker-miniprox:~# ip a
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 02:a8:ab:af:f1:89 brd ff:ff:ff:ff:ff:ff
altname enp0s18
inet 192.168.101.12/24 brd 192.168.101.255 scope global ens18
valid_lft forever preferred_lft forever
inet6 fe80::a8:abff:feaf:f189/64 scope link
valid_lft forever preferred_lft forever
[docker containers IP removed]
root@docker-miniprox:~# ip route
default via 192.168.101.1 dev ens18 proto static
192.168.101.0/24 dev ens18 proto kernel scope link src 192.168.101.12
[docker containers routes removed]On my VM 1 :
Code:
root@VM1:~# ip a
2: eth0@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 76:6c:5c:f1:5f:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.100.3/24 brd 192.168.100.255 scope global eth0
valid_lft forever preferred_lft forever
root@VM1:~# ip route
default via 192.168.100.1 dev eth0 onlink
192.168.100.0/24 dev eth0 proto kernel scope link src 192.168.100.3I tried to diagnose the network using tcpdump but I can't find how to fix the issue, even when seing the RESET flags.
Here is the tcpdump when trying to access VM1.domain.tld (pointing to 192.168.101.12), the reverse proxy then trying to redirect to 192.168.100.3 :
On VM A :
Code:
root@docker-miniprox:~# tcpdump -i any host 192.168.100.3 -n
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
21:37:32.077853 vethea66fca P IP 172.18.0.2.56660 > 192.168.100.3.80: Flags [S], seq 3742118872, win 64240, options [mss 1460,sackOK,TS val 2736377073 ecr 0,nop,wscale 7], length 0
21:37:32.077854 br-01956c7401fd In IP 172.18.0.2.56660 > 192.168.100.3.80: Flags [S], seq 3742118872, win 64240, options [mss 1460,sackOK,TS val 2736377073 ecr 0,nop,wscale 7], length 0
21:37:32.077865 ens18 Out IP 192.168.101.12.56660 > 192.168.100.3.80: Flags [S], seq 3742118872, win 64240, options [mss 1460,sackOK,TS val 2736377073 ecr 0,nop,wscale 7], length 0
21:37:32.078190 ens18 In IP 192.168.100.3.80 > 192.168.101.12.56660: Flags [R.], seq 0, ack 3742118873, win 0, length 0
21:37:32.078196 br-01956c7401fd Out IP 192.168.100.3.80 > 172.18.0.2.56660: Flags [R.], seq 0, ack 3742118873, win 0, length 0
21:37:32.078197 vethea66fca Out IP 192.168.100.3.80 > 172.18.0.2.56660: Flags [R.], seq 0, ack 1, win 0, length 0On Proxmox 2 :
Code:
root@miniprox:~# tcpdump -i any host 192.168.100.3 -n
tcpdump: WARNING: any: That device doesn't support promiscuous mode
(Promiscuous mode not supported on the "any" device)
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
22:39:09.363419 tap103i0 P IP 192.168.101.12.56634 > 192.168.100.3.80: Flags [S], seq 2398949181, win 64240, options [mss 1460,sackOK,TS val 2736474355 ecr 0,nop,wscale 7], length 0
22:39:09.363420 vmbr1 In IP 192.168.101.12.56634 > 192.168.100.3.80: Flags [S], seq 2398949181, win 64240, options [mss 1460,sackOK,TS val 2736474355 ecr 0,nop,wscale 7], length 0
22:39:09.363433 vmbr0 Out IP 192.168.1.201.56634 > 192.168.100.3.80: Flags [S], seq 2398949181, win 64240, options [mss 1460,sackOK,TS val 2736474355 ecr 0,nop,wscale 7], length 0
22:39:09.363434 enp1s0 Out IP 192.168.1.201.56634 > 192.168.100.3.80: Flags [S], seq 2398949181, win 64240, options [mss 1460,sackOK,TS val 2736474355 ecr 0,nop,wscale 7], length 0
22:39:09.363592 enp1s0 In IP 192.168.100.3.80 > 192.168.1.201.56634: Flags [R.], seq 0, ack 2398949182, win 0, length 0
22:39:09.363594 vmbr0 In IP 192.168.100.3.80 > 192.168.1.201.56634: Flags [R.], seq 0, ack 1, win 0, length 0
22:39:09.363600 vmbr1 Out IP 192.168.100.3.80 > 192.168.101.12.56634: Flags [R.], seq 0, ack 2398949182, win 0, length 0
22:39:09.363601 tap103i0 Out IP 192.168.100.3.80 > 192.168.101.12.56634: Flags [R.], seq 0, ack 1, win 0, length 0On Proxmox 1 :
Code:
root@proxmox:~# tcpdump -i any host 192.168.100.3 -n
tcpdump: WARNING: any: That device doesn't support promiscuous mode
(Promiscuous mode not supported on the "any" device)
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
22:39:29.544300 enp6s0 In IP 192.168.1.201.56286 > 192.168.100.3.80: Flags [S], seq 1913467726, win 64240, options [mss 1460,sackOK,TS val 2736494535 ecr 0,nop,wscale 7], length 0
22:39:29.544300 vmbr0 In IP 192.168.1.201.56286 > 192.168.100.3.80: Flags [S], seq 1913467726, win 64240, options [mss 1460,sackOK,TS val 2736494535 ecr 0,nop,wscale 7], length 0
22:39:29.544437 vmbr0 Out IP 192.168.100.3.80 > 192.168.1.201.56286: Flags [R.], seq 0, ack 1913467727, win 0, length 0
22:39:29.544439 enp6s0 Out IP 192.168.100.3.80 > 192.168.1.201.56286: Flags [R.], seq 0, ack 1, win 0, length 0On VM 1 :
Code:
root@VM1:~# tcpdump -i any host 192.168.100.3 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernelHere is the iptables I'm using for forwarding :
Code:
root@miniprox:~# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp -- anywhere anywhere tcp dpt:http to:192.168.101.12:80
DNAT tcp -- anywhere anywhere tcp dpt:https to:192.168.101.12:443
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE all -- 192.168.101.0/24 !192.168.101.0/24There is also a weird behaviour where I can't access anymore http://192.168.100.205 when redirecting all 80/443 traffic from my Freebox to my Proxmox 2 server. I don't know if it matters in all of this.
Thanks a lot for your help
Attachments
Last edited: