Routed Block Public IP /29

[KingLlama]

So, i'm running into a issue with setting up. I have eno1 plugged into that is going into the datacenters switch. vmbr is set with the routed block and the gateway being .142 but as I am looking at this is its showing .143 in which is incorrect but on proxmox gui it shows .142. Im trying to route out from one vm to the public internet with vmbr1 to bridge to vmbr0 to reach out to the world but having trouble.


root@fiendserver:/# ip route
default via xxx.71.205.142 dev vmbr0 onlink
192.168.10.0/24 dev vmbr1 proto kernel scope link src 192.168.10.100
207.71.205.136/29 dev vmbr0 proto kernel scope link src xxx.71.205.138
root@fiendserver:/#


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
link/ether 00:25:64:fd:2f:2d brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:25:64:fd:2f:2f brd ff:ff:ff:ff:ff:ff
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:25:64:fd:2f:31 brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:25:64:fd:2f:33 brd ff:ff:ff:ff:ff:ff
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:25:64:fd:2f:2d brd ff:ff:ff:ff:ff:ff
inet xxx.71.205.138/29 brd xxx.71.205.143 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::225:64ff:fefd:2f2d/64 scope link
valid_lft forever preferred_lft forever
7: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 06:42:f8:95:94:d5 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.100/24 brd 192.168.10.255 scope global vmbr1
valid_lft forever preferred_lft forever
inet6 fe80::b4c4:c6ff:fe23:131c/64 scope link
valid_lft forever preferred_lft forever
8: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000
link/ether 6a:6f:41:39:b6:6e brd ff:ff:ff:ff:ff:ff
9: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 7a:15:6a:92:42:13 brd ff:ff:ff:ff:ff:ff
10: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr1 state UP group default qlen 1000
link/ether 06:42:f8:95:94:d5 brd ff:ff:ff:ff:ff:ff
11: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
link/ether 7a:15:6a:92:42:13 brd ff:ff:ff:ff:ff:ff
root@fiendserver:/#

 

[rauxon]

From those output, you have configured:

• vmbr0 x.71.205.143/29 (network x.71.205.138/29)
  Which looks to be a typo - a /29 network would be x.71.205.128
• Default route via x.71.205.142

So - proxmox is on .143, and the gateways is on .142. Given that the network is a /29, you will not be able to access a gateway on .24, since that is not on the local network IP range. If you can just list the various network details (e.g. desired IP address for proxmox, gateway address, network address/netmask), one of us can tell you what to enter into the system to get it to work.

You cannot just 'bridge' between vmbr0 and vmbr1 to route given that they are on different IP ranges. Bridging is a layer 2 feature (e.g. equivalent to a network cable or switch), and you need L3 (e.g. router). Proxmox can do it, but I much prefer using a container to keep things cleaner at the proxmox layer.

If you do want to do it with just the proxmox instance, you are going to need to enable routing (edit /etc/sysctl.conf) and almost definitely configure NAT (I use /etc/rc.local, but there are probably better ways). I am specifically not giving details here, because if you want to do that you really need to learn enough about networking not to break things yourself. If you do it remotely you can quite easily lose your access and need to get console access to fix it.

 

[KingLlama]

From those output, you have configured:

• vmbr0 x.71.205.143/29 (network x.71.205.138/29)
  Which looks to be a typo - a /29 network would be x.71.205.128
• Default route via x.71.205.142

So - proxmox is on .143, and the gateways is on .142. Given that the network is a /29, you will not be able to access a gateway on .24, since that is not on the local network IP range. If you can just list the various network details (e.g. desired IP address for proxmox, gateway address, network address/netmask), one of us can tell you what to enter into the system to get it to work.

You cannot just 'bridge' between vmbr0 and vmbr1 to route given that they are on different IP ranges. Bridging is a layer 2 feature (e.g. equivalent to a network cable or switch), and you need L3 (e.g. router). Proxmox can do it, but I much prefer using a container to keep things cleaner at the proxmox layer.

If you do want to do it with just the proxmox instance, you are going to need to enable routing (edit /etc/sysctl.conf) and almost definitely configure NAT (I use /etc/rc.local, but there are probably better ways). I am specifically not giving details here, because if you want to do that you really need to learn enough about networking not to break things yourself. If you do it remotely you can quite easily lose your access and need to get console access to fix it.

That was a typo in and it is fixed. So, What I am trying to figure out which is a better to route these to the internet so that I can set them up as testing environments for multiple things. I thought these would work for natting but it seems that
iptables -t nat -A PREROUTING -p tcp -d xxx.71.205.138 --dport 80 -i vmbr0 -j DNAT --to-destination 192.168.10.160:80
iptables -t nat -A PREROUTING -p tcp -d xxx.71.205.138 --dport 443 -i vmbr0 -j DNAT --to-destination 192.168.10.160:443
iptables -A POSTROUTING -t nat -s 192.168.10.0/24 -j MASQUERADE

https://youtu.be/Q5l7VH6b5r4 I followed this tutorial so that it can reach out to the internet but it didn't save.

 

[rauxon]

I would not use proxmox directly for this - exposing your hypervisor directly is not ideal, and you can reverse the setup so that one of your VMs acts as the firewall/etc and keep proxmox on the inside. The video you referenced seems to talk about a single external IP rather than a routed /28 - the video and the answer below should both work with either, but will need some smaller changes... The details of which are networking rather than proxmox

Option 1 : Proxmox as Router/firewall:


This setup uses a bridge internally for all VMs/containers, and an external network connection to the server. Setup:

1. vmbr0 with the public IP address, and default route via primary network
2. vmbr1 with an internal IP address (192.168.10.1/24 for example)
3. All VMs/containers are put on vmbr1

Option 2 : A container/VM as firewall:

This setup uses a bridge internally for all VMs/containers and Proxmox itself, and the external connection is logically to a VM/container. This means that all traffic to/from proxmox goes through the VM/container and you can protect all your proxmox services from random people on the internet. Setup:

1. vmbr0 with an internal IP address
2. vmbr1 with no IP address
3. All VMs/containers are put on vmbr0, and the firewall/router VM/container is also attached to vmbr1

Whichever way you setup, the configuration required is the same:

• iptables rules - I don't specify the destination IP address (relying on just the incoming interface/port to specify the traffic)
• ip_forwarding enabled - this appears to be missed in the video: /etc/sysctl.conf uncomment the 'net.ipv4.ip_forward=1' line

I would strongly recommend Option 2 - it takes a little longer to setup, but protects your proxmox installation from the internet as well as giving you more control. There are other options around the port forwarding (haproxy for example), and you can also then do VPN setup to the firewall to allow you to access things directly rather than needing to go through port forwards when you want to.

Setting up a full network is not really part of this forum though - since proxmox or not, it is standard linux networking.