Networking question

kobuki

Renowned Member
Dec 30, 2008
473
27
93
I have a simple Proxmox installation, using 2 network interfaces, one for the public internet, and one for the LAN. Let's assume 44.55.66.77 is public ip of the host, and 10.1.1.1 is a the host LAN address. vmbr0 is the public bridge, vmbr1 is the LAN bridge, mastering eth0 and eth1, respectively.

I fire up a KVM Linux VE, bridging it to vmbr1, and do a simple SNAT to have internet access in it. The VE's static IP address on the LAN is 10.1.1.2. All is fine, internet working, I can access the host and the internet.

Now, the hard part (for me). I want to create a LAN on the 172.5.0.0 private subnet the same way the 10.1.1.0 is working in the above scenario, and want to assign 172.5.0.2 to the VE instead of 10.1.1.2, while keeping the 10.1.1.0 subnet for other VEs. I'm inexperienced with bridges, so I don't know how to achieve this... Please help. Maybe I have to create a third bridge? Adding IP aliases (and setting up NAT/forwards) to vmbr1 or eth1 doesn't work.
 
Last edited:
You have assigned IP to your bridges because you want to reach Proxmox server itself through that IP. On the VM side, the IP of the bridge has no influence at all (in fact, I've 4 bridges and only vmbr0 with an IP, since I have to reach my proxmox server through that IP).
Think a bridge like being a physical hub where you "plug" eth0 of your VMs. Each VM can have it's IP and subnet. So you can use vmbr1 to "plug" (or "bridge" as you say) new KVM VM that will be configured internally as 172.5.0.x (or whatever you like). Then SNAT as you do for the other subnet.
Beware that if you put your VM adapter in promosquous mode you can sniff all the traffic that passes through that bridge (vmbr1 in your case), so you could create a security hole (like would happen with a physical hub).
 
Thanks for your quick answer. Yes, I'm aware that the virtual bridges more or less behave like a hub. Howewer I was making a small mistake in my config - it was a physical machine converted to a KVM VE, and I just set up the gateway wrong. But the forums did well as a 'Rubber Duck' :) Thanks again and sorry for the inconvenience to other readers as well.
 
Okay. Now everything is working, except one thing. I cannot for the life of mine find out why I can't reach the forwarded port 2200 from within the HN and from within the NATted VE at 172.30.0.1, using the public IP address. The public address below is the 77.88.99.224, and a simple thing (running in the HN) such as:

Code:
vs1:~# ssh -p 2200 username@77.88.99.224
doesn't work. Using the 172.30.0.1 LAN address or ssh'ing from any public address outside this HN works (the ssh port is moved to 2200 on that particular VE). Using the usual NAT loopback SNATting like the one below just doesn't work. I've spent a considerable amount of time working this out to no avail. I cannot find out what the problem is.

Code:
iptables -I POSTROUTING -t nat -p tcp --destination 77.88.99.224 --dport 2200 -j SNAT --to-source 172.30.0.254
Can you suggest a solution?

Code:
vs1:~# iptables -A PREROUTING -t nat -p tcp -i vmbr0 -d 77.88.99.224 --dport 2200 -j DNAT --to-destination 172.30.0.1:2200
vs1:~# iptables -I FORWARD -d 172.30.0.1 -p tcp --dport 2200 -j ACCEPT
vs1:~# iptables -A POSTROUTING -tnat -s 172.30.0.1/32 -d ! 172.30.0.1/32 -o vmbr0 -j SNAT --to-source 77.88.99.224
vs1:~# iptables -A FORWARD -i vmbr0 --match state --state RELATED,ESTABLISHED --dest 172.30.0.1/32 -j ACCEPT

vs1:~# iptables -L -v -n
Chain INPUT (policy ACCEPT 32832 packets, 4984K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 81162 packets, 33M bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            172.30.0.1          tcp dpt:2200
   13   988 ACCEPT     all  --  vmbr0  *       0.0.0.0/0            172.30.0.1          state RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT 28282 packets, 5244K bytes)
 pkts bytes target     prot opt in     out     source               destination

vs1:~# iptables -L -v -n -t nat
Chain PREROUTING (policy ACCEPT 24648 packets, 2138K bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       tcp  --  vmbr0  *       0.0.0.0/0            77.88.99.224         tcp dpt:2200 to:172.30.0.1:2200

Chain OUTPUT (policy ACCEPT 1353 packets, 89960 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 24995 packets, 2132K bytes)
 pkts bytes target     prot opt in     out     source               destination
   13   988 SNAT       all  --  *      vmbr0   172.30.0.1          !172.30.0.1          to:77.88.99.224

vs1:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:17:be:7a:d2
          inet6 addr: fe80::216:17ff:febe:7ad2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:300695974 errors:251 dropped:0 overruns:0 frame:251
          TX packets:89951554 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:395760297396 (368.5 GiB)  TX bytes:40907267832 (38.0 GiB)
          Interrupt:46 Base address:0x6000

eth1      Link encap:Ethernet  HWaddr 00:16:17:bc:4b:d9
          inet6 addr: fe80::216:17ff:febc:4bd9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:320298 errors:0 dropped:0 overruns:0 frame:0
          TX packets:110630 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:41444776 (39.5 MiB)  TX bytes:36487397 (34.7 MiB)
          Interrupt:47 Base address:0x8000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:4446881 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4446881 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7996380708 (7.4 GiB)  TX bytes:7996380708 (7.4 GiB)

tap101i1d0 Link encap:Ethernet  HWaddr f6:41:a2:f8:ed:67
          inet6 addr: fe80::f441:a2ff:fef8:ed67/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:20471526 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28056037 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:23451679677 (21.8 GiB)  TX bytes:25132334223 (23.4 GiB)

tap102i0d0 Link encap:Ethernet  HWaddr c2:24:c8:18:1a:bf
          inet6 addr: fe80::c024:c8ff:fe18:1abf/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:2226959 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5085102 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:2590970732 (2.4 GiB)  TX bytes:2830363826 (2.6 GiB)

vmbr0     Link encap:Ethernet  HWaddr 00:16:17:be:7a:d2
          inet addr:77.88.99.224  Bcast:77.88.99.255  Mask:255.255.255.0
          inet6 addr: fe80::216:17ff:febe:7ad2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:64008122 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35896008 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:240875010626 (224.3 GiB)  TX bytes:24901987770 (23.1 GiB)

vmbr1     Link encap:Ethernet  HWaddr 00:16:17:bc:4b:d9
          inet addr:10.42.1.2  Bcast:10.42.1.255  Mask:255.255.255.0
          inet6 addr: fe80::216:17ff:febc:4bd9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20620676 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28072424 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:23184569148 (21.5 GiB)  TX bytes:25143218555 (23.4 GiB)

vmbr1:1   Link encap:Ethernet  HWaddr 00:16:17:bc:4b:d9
          inet addr:172.30.0.254  Bcast:172.30.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

vs1:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.30.0.0      0.0.0.0         255.255.255.0   U     0      0        0 vmbr1
10.42.1.0       0.0.0.0         255.255.255.0   U     0      0        0 vmbr1
77.88.99.0       0.0.0.0         255.255.255.0   U     0      0        0 vmbr0
0.0.0.0         77.88.99.254     0.0.0.0         UG    0      0        0 vmbr0
 
Last edited:
Anyone has an idea? I'm still stuck with this. The system is working so every service I want to publish can be accessed from the internet, but it's very inconvenient and makes testing the system a lot harder. Any guide using NAT and bridges might be useful, although I haven't been able to find one. Maybe it's a bridge problem or even a bug?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!