IPv4 Subnetting not working behind fritzbox.

[pascalku]

Hey there,

I am spinning up a local proxmox virtual environment behind my Fritz!Box 7490. I want to use this to create private development environments.

I followed the tutorial to configure the network interface (see https://pve.proxmox.com/wiki/Network_Configuration). Now I got the following configuration:

auto lo
iface lo inet loopback

# primary network interface
auto enp2s0
iface enp2s0 inet static
        address 192.168.178.131
        netmask 255.255.0.0
        gateway 192.168.0.1
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/eno1/proxy_arp

# vm network interface
auto vmbr0
iface vmbr0 inet static
        address 192.168.93.65
        netmask 255.255.255.224
        bridge_ports none
        bridge_stp off
        bridge_fd 0

If I now set the address 192.168.93.69/32 in a VM, it is not reachable on the port (set during the installation of Ubuntu, for example). The virtual machine is not only not accessible from the network, but also not from the host system.

Virtual machine (nano /etc/pve/qemu-server/100.conf):

bootdisk: scsi0
cores: 1
ide2: local:iso/ubuntu-18.04.4-live-server-amd64.iso,media=cdrom
memory: 4096
name: s010203
net0: virtio=6A:ED:78:D3:BF:86,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-100-disk-0,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=14ad7975-37ee-4e95-8cd8-c723f638e077
sockets: 2
vmgenid: 67f9b74d-e72d-4569-810c-01389dc50c31

Anyone an idea?

 

[t.lamprecht]

Hi,

If I now set the address 192.168.93.69/32 in a VM

a bit guestimating as it's late, but how can the VM route if it gets a /32 IP? Shouldn't it get a /27 ?

But anyway, as you say fritzbox and you use LAN addresses in your post for the VMs, I'd guess that you rather want the NAT Masquerading setup.

The routed is for when you have multiple public IP which you must route through the host NIC due to a "one MAC only" in some hosted setups limitation.

 

[BobhWasatch]

If I now set the address 192.168.93.69/32 in a VM, it is not reachable on the port (set during the installation of Ubuntu, for example). The virtual machine is not only not accessible from the network, but also not from the host system.

Why are you setting a /32 on the VM? The internal subnet for VM's is a /27 per your settings. For that matter, why not just use a /24 and make everything simpler? It is a private address space after all.

 

[pascalku]

Now I switched to a /24 subnet for testing purposes and specified it in the virtual machine. From the host system it is now also pingable but not from the local network?

 

[BobhWasatch]

That sounds right. Nothing on your local network knows a route to the VM's. The VM's can send packets out but nobody can reply.

I don't know exactly what you are trying to do or why, but the simplest thing is to have your Proxmox host and the VM's all on the same bridge and same subnet.

Don't assign an IP to enp2s0, instead add it to vmbr0 by changing "bridge-ports none" to "bridge-ports enp2s0". Then assign VM's addresses on that same subnet either statically or with DHCP. Now it is one flat bridged network and everything can see everything else.

If you are trying to have an isolated private subnet inside Proxmox, it gets more complicated depending on exactly what you want to do.

The next simplest thing is to set up NAT on your existing Proxmox configuration. There are HOWTO's around on how to do it. That will give you a private isolated subnet inside Proxmox. The VM's won't be reachable from outside the Proxmox host, although they can connect out and responses will come back.

Making pings to the VM's from the LAN happen with a multi-subnet scenario will likely involve reconfiguring your network to a hierarchical setup (e.g. having your main router route several subnets on different ports or VLAN's) and learning a lot of details about how IP routing works. Some people do this so services can be firewalled. or because the network is large and splitting it up makes it easier to manage But it is a significant amount of work.

 

[apoc]

I have also noticed that your network uses a /16 subnet mask (netmask 255.255.0.0) which is pretty unusual for a home setup and also violates RFC private ip-addresses. This means that ultimately you will likely not be able to access some websites in the internet in the long run.
Even though this seems to fit with the gateway you have provided...

What exactly do you want to achieve?
Apologies this is not meant as an offense but to me it seems you are "messing around with super- and subnets but have no idea how IPv4 actually works".

 

[pascalku]

In general, I want to have a separate network for my virtual machines. In the datacenter a server setup will soon get a 27 network. I would like to test this setup at home before placing the server in the datacenter. Therefore I installed Proxmox on an old machine and want to reach the virtual machines created on the host within the network.

 

[BobhWasatch]

I would like to test this setup at home before placing the server in the datacenter.

Ok, that makes sense. I presume that the VM's will be offering services to the Internet then? That your provider will be routing the whole /27 to your Ethernet port? If that is the case you should go with the simplest setup. Put the host's Ethernet and all the VM's on the same bridge. Assign one address from your /27 (not the first or last, those are special) to the bridge to get access to the host. Assign addresses to the VM's from the same /27.

For the home test, just assign addresses from your home LAN instead of the /27. Use the same netmask as your LAN, otherwise you will have problems. When you set up the real server you'll just have to adjust IP addresses and netmasks. BTW, your /16 is unusual but I don't think it violates any rules. All of 192.168.0.0/16 is private address space.

If this will be on the public Internet you might also want to think about a firewall policy. You can set up firewalls on each VM or you can use the Proxmox one to manage them all. Of course the setup on your test network will be slightly different than the real one but at least you can get a handle on how it works.

 

[pascalku]

Hey @BobhWasatch!

Thanks for the detailed answer.

So, I now just setup the host with the following configuration:

auto lo
iface lo inet loopback

auto enp2s0
iface enp2s0 inet static
    address 192.168.178.131
    netmask 255.255.0.0
    gateway 192.168.0.1

Applied the settings with a reboot and now I am able to spin up virtual machines with IP manually selected ip addresses? But now there is no network selectable in vm setup.

 

[pascalku]

Update to the post above:

I now read the documentation (linked in my first post) again and build a bridged configuration like this:

auto lo
iface lo inet loopback

auto enp2s0
iface enp2s0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.178.131
    netmask 255.255.0.0
    gateway 192.168.0.1

 

[apoc]

unusual but I don't think it violates any rules

You are right. My mistake. I went somewhat off to soon. And completely messed up my argument about not knowing anything about IPv4

Still having trouble to get the point in a /16 subnet at home. Because no one needs that many IP-adresses. It also limits the options for setting things up because you just have eaten the whole address space. Yes, you can use 172.16. and 10.x addresses as well, but in the end it is just shooting with BIG guns on a small target.

My approach would be the following (if it needs to be a /27 setup):

• Setup a /24 network for your Fritzbox as it is default.
• Add a virtual (internal) network / bridge for your VMs, you plan to use on a /27 network
• add a virtual router instance to your Proxmox. One leg on the Fritzbox, one on the internal /27 network
  • you can simply activate ip forwarding to make it a router instance.
• Either on the Fritzbox or on your workstation, add a route which indicates where to find the /27 network (e.g. the virtual router instance)

That's it.

Otherwise, if you plan just to use the same IP-addresses (but different subnet masks) why not just using a /24 ip segment as you are doing it with the /16 currently?

 

[pascalku]

Is it possible to spend static addresses from outside the vms? So root users of my virtual machines cannot manipulate my network?

 

[t.lamprecht]

You can always limit IP address usage by disallowing all but one IP address as source for outgoing packets in the VMs firewall rules.

 

[pascalku]

But this is again on the side of the VM? So I am not able to tell the virtual machine that it can only use one list of IPs and everything else will not work no matter what is set inside the virtual machine?

 

[BobhWasatch]

I think the t.lamprecht was referring to the Proxmox firewall. If you read the docs you will see there is the ability to apply rules for the whole system and rules that apply to individual VM's. These are outside control of a root user on the VM. See the "ipfilter-net" rules here:

https://pve.proxmox.com/wiki/Firewall

VM's can also have their own host-based firewall that the root user on the VM manages. That is outside the control of Proxmox.