[SOLVED] Proxmox with Ionos. Multiple public IPs and private as well.

[doctick]

Hello, I have proxmox running on a dedicated server. The main proxmox ip is a public non vlan ending in 65. I have 2 other public addresses that are vlan tagged at 1000. I created a linux vlan eth0.1000 and then created a linux bridge vmbr1 turned on vlan tagging and pointed bridged port to eth0.1000.

ips
x.x.x.65 untagged main IP for machine and proxmox
x.x.x.228 tagged vlan 1000
x.x.x.161 tagged vlan 1000
all 3 ips gateway is 10.255.255.1

if I type ip r from the node shell I get

default via 10.255.255.1 dev eth0 proto kernel onlink
10.10.10.0/24 dev vmbr0 proto kernel scope link src 10.10.10.1

the bridge vmbr0 works fine as a nat network
the bridge vmbr1 tagged is not working.

Thanks for your help
Kevin

 

[_gabriel]

3 ips gateway is 10.255.255.1

it can't work, only one default gateway is possible in a OS.
do not set additionnal public ip on vmbr1 but directly in your guest VM/CT which has vmbr1.

 

[doctick]

I am able to add a vlan tagged ip to vmbr1 and see it from the internet when i try to add a public IP to a vm it does not work. I can ping 127.0.0.1 and the public ip but ip r shows nothing and ip neighbor shows failed.

I have open icmp through firewall so I can ping through the firewall. when the vm boots the ping goes from unreachable to request timed out.

in the vm I have configured like normal with the public ip x.x.x.109/32 and the route as 10.255.255.1 I am wondering if there is something also I need to do for routing.

Thanks

Kevin

 

[doctick]

Here is a copy of my interface file.

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

#iface eth0 inet6 manual
# pre-up sleep 5
# up dhclient -6 -nw -v eth0
# down dhclient -6 -r -v eth0

iface eth1 inet manual

auto eth0.1002
iface eth0.1002 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet static
address x.x.x.161/32
gateway 10.255.255.1
bridge-ports eth0.1002
bridge-stp off
bridge-fd 0
#Public IP access

auto vmbr2
iface vmbr2 inet manual
bridge-ports eth0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Tagged vlan

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr1 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr1 -j MASQUERADE

The 10.10.10.0/24 network is not working in this way. I have to use vlan tagged network which is put in vmbr1

Thank you

Kevin

 

[_gabriel]

you don't need NAT from 161 public ip
if 1 guest use the 161 ip, juste remove ip from vmbr1 interface and set iface vmbr1 inet manual

edit: you lost me, where are others ip in your interfaces ? + put content into CODE TAG

 

[doctick]

Thank you Gabriel.

I have vms that will use the private ip as well.

I am trying to get the public ips
working right now.

thank you
Kevin

 

[doctick]

ok, here are some updates. I have fixed the private network. here is my interfaces file

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
bridge-ports eth0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Tagged VLAN

auto vmbr1.1002
iface vmbr1.1002 inet static
address x.x.x.228/32
gateway 10.255.255.1

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr1.1002 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr1.1002 -j MASQUERADE

I create a new vm and give it vmbr1 and tag 1002 I add the public address x.x.x.161 with gateway 10.255.255.1 and I get no connection to the world

from the vm I can ping the local 127.0.0.1 and the ip x.x.x161. I cannot ping the gateway 10.255.255.1 I cannot ping 8.8.8.8 They both show network unreachable. my setup in the vm is
network:
version: 2
ethernets:
ens18:
addresses:
- x.x.x.161/32
routes:
- to: default
via: 10.255.255.1
nameservers:
addresses:
- 8.8.8.8

ip r returns nothing
ip neighbor returns nothing.

Thank you

Kevin

 

[_gabriel]

VLAN is 1002 but you said VLAN 1000 in your first post.
Are VLAN provided by ionos ?

 

[doctick]

Sorry, my provider changed to plan 1002

Yes, by ionos

Kevin

 

[_gabriel]

Where is defined your ip .65 ?

 

[doctick]

Not using .65. IONOS said cannot use non vlan and vlan at the same time.

Something with their firewall

Thank you

Kevin

 

[_gabriel]

isn't registering MAC to public ip required ?

 

[doctick]

No, they have a public internet you can create which vlan tags ips, you can have multiple items using ip. That is the setup I am using. The standard ip is not used and only the tagged ips are used in setup. The main console and private vms work fine. The additional public ips are what is not working.

Thank you.
Kevin

 

[_gabriel]

It should work in (my) theory, but I'm not a VLAN user :-/
Have you tried attach vmbr1.1002 to VM without VLAN tag in VM's NIC ?

 

[doctick]

I point the vms to vmbr1 with tagging turned on. Vmbr1.1002 is a vlan device just under vmbr1. In theory it should work.

Thank you
Kevi

 

[doctick]

Here is how to make it work.

Original Config.

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0


I made these changes to setup the vlan tag and vmbr1 for the tagged traffic from the vm

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

auto eth0.1002
iface eth0.1002 inet manual

post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

auto vmbr0
iface vmbr0 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0

auto vmbr1
iface vmbr1 inet static
address x.x.x.228/32
gateway 10.255.255.1
bridge-ports eth0.1002
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Tagged VLAN

This sets up proxmox to work.

Then inside of vm, I am using ubuntu 22.04, you set up netplan like this

network:
ethernets:
ens18:
addresses:
- x.x.x.161/32
nameservers:
addresses:
- 8.8.8.8
search: []
routes:
- to: default
via: 10.255.255.1
metric: 100 #This is needed
on-link: true #This is needed
version: 2

when setting up the vm use vmbr1 for public ip and vmbr0 for private ip

Thank you

Kevin

 

[dekkert]

For my understanding: Is the VLAN tag used to make access private? What would happen if someone attempted to hack using VLAN tags?

I have a similar setup on a dedicated server, but I manage private access through a WireGuard VPN. There is no access to the PVE server or the VMs from the public IP address.