Network Configuration

[SoCalPilot]

First-time Proxmox (VE 8.2) user here, doing my best to do my research before asking for help. I need a little help getting my first Windows VM online and believe my problem is with the networking setup as configured by the hosting provider that did the initial network configuration. I can access my main configuration panel via my primary public IP, however, I can't get any VMs to reach the Internet even after installing the VirtIO drivers.

Here are the network specs I was provided...

IP Range (CIDR):    23.227.168.192/29
Subnet:            255.255.255.248
Gateway IP:        23.227.168.193
Usable IP:        23.227.168.194
Usable IP:        23.227.168.195
Usable IP:        23.227.168.196
Usable IP:        23.227.168.197
Usable IP:        23.227.168.198

And here is the contents of /etc/network/interfaces

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2

auto vmbr0
iface vmbr0 inet static
        address 23.227.168.194/29
        gateway 23.227.168.193
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

From what I'm reading I possibly need to set MASQUERADE under /etc/network/interfaces but this part throws me in a loop and I don't want to start randomly editing my interfaces file in a random trial and error type. Unfortunately, my web host was unable to help further and pointed me to this forum. I'd be willing to PayPal someone for their time as I respect everyone's time is valuable and something simple to you might take me days of trial and error to figure out.

 

[sw-omit]

If you look in the windows-vm itself, do you at least see the network-adaptor itself? (Use the "old" control panel for this)
And if you configure the following static settings, does it then not work?
IP: 23.227.168.195 (note the 195, not 194, just to be clear)
Mask: 255.255.255.248
Gateway 23.227.168.193
DNS 1: 8.8.8.8
DNS 2: 1.1.1.1

What are your network-adaptor settings in Proxmox for this VM?

 

[SoCalPilot]

If you look in the windows-vm itself, do you at least see the network-adaptor itself? (Use the "old" control panel for this)

Yes, I do see an adapter connected, just says no internet access. I have tried all the adapters including the Intel E1000, Intel E1000E, VirtIO, Realteck RTL8139 & WMware vmxnet3 and they all pretty much do the same thing, show connected with no internet access.

And if you configure the following static settings, does it then not work?
IP: 23.227.168.195 (note the 195, not 194, just to be clear)
Mask: 255.255.255.248
Gateway 23.227.168.193
DNS 1: 8.8.8.8
DNS 2: 1.1.1.1

I swear I tried this before and it didn't work, however, I tried again and that did the job, the VM immediately went online. Thank you very much for the help. PM me your PayPal and I'll buy you lunch.

Final question .... Since I have 5 usable IPs I am assuming I can perform the same steps on each additional VM I create and just select a unique free IP? And let's say I have 5 VMs using all 5 available IPs, what happens if I spin up a 6th VM and reuse the same static IP as set in another VM, will that create some kind of IP conflict on the network?

 

[sw-omit]

Yes, you have 5 usable IP's, but one of them is in use for your Proxmox itself, so you only have 4 for your VM's.
If you re-use the IP, then you'll create an IP-Address conflict [1], which means that either one or both of the devices will just not work, possibly at random even.
If you need more then these 4 IP's, you'll need a router(-vm) to distribute the traffic to multiple internal services, or buy more IP's
If you're planning to have more then 4 VM's, it might be better to already plan for it now, as it not only will make your network probably more secure, but since you don't have all the IP's in use yet, it's easier to set up.

The only (main) thing you need to be aware of, is that you can't have the same port on the same IP in use by different servers behind it serving different content (without extra configuration or at all).
What I mean with that for example, if you have 10 website-servers, all using 80 and 443, without that extra configuration you can only use port 80 and 443 on those 4 (or 5 if you put proxmox behind the router too) IP's, but you CAN for example on 23.227.168.195 route port 80 to VM 1, and port 81 to VM 2, and 23.227.168.196 port 80 to VM 3

As for that lunch, no real need for something like this simple a tip (at least simple for me, working in networking daily), but maybe if you need a hand with the router-VM setup I'll consider it
I would personally suggest using OPNSense btw, although others prefer PFSense or even Proxmox's own built-in network-management.

[1] https://en.wikipedia.org/wiki/IP_address#Addressing_conflicts

 

[sw-omit]

Oh, and in case that wasn't clear btw, with a router-VM, you can have multiple IP's assigned to it, so you would only need 1 VM to distribute traffic to how many VM's you put behind it, and usually 1-2 cores and a couple GB of ram is plenty for smaller setups.

 

[SoCalPilot]

Additional question but posting to this thread as it's likely related to the same configuration issue...

I'm trying to install TrueNAS into a new VM on ProxMox and I'm able to get to the console setup page which initially says to access my web interface at http://0.0.0.0 so I chose to configure network interfaces I setup a static IPv4 that is currently unused and then it correctly shows on the console to access the web interface at this new static IP I've setup however, I can't access it from any browser outside of the ProxMox server BUT if I go into my Windows VM and try to access the IP it works as expected so basically I can access from another VM on the ProxMox server, but not externally which leads me to believe that the networking configuration isn't configured right?

I thought that assigning a free static IP would get it working as it did for my Windows VM (Although I haven't tried to access it externally yet, I know it is online from using the console).

I can post screenshots, just didn't want to post a bunch of useless stuff and waste someone's time but will gladly share whatever is needed. Luckily this is not a production server and there is no sensitive information to protect.

 

[alexskysilk]

Here are the network specs I was provided...

This is usually assigned to your router, which in turn can/should be set up to NAT internally. There is almost no usecase when you want your hypervisor/VMs facing directly out to the internet- here be dragons.

 

[SoCalPilot]

This is usually assigned to your router, which in turn can/should be set up to NAT internally. There is almost no usecase when you want your hypervisor/VMs facing directly out to the internet- here be dragons.

Question, to get these VMs available externally must I set up a VM with a router first or is there a way to avoid that step? I understand the security risk, although for a dummy dev server with no sensitive content, being hacked isnt really a concern for now.

 

[alexskysilk]

Question, to get these VMs available externally must I set up a VM with a router first

It all depends on how your ISP is delivering the IPs to you. You probably have some sort of device at the head end of your network thats provided by your ISP. This device can be set to pass through the IPs, or it could be set up as a NAT.

IF its delivering the IPs directly, all you need to do is simply plug in a cable from your host to a modem's LAN port, and assign it to a vmbr. You can now have 5 separate mac addresses (it doesnt matter if its a physical or logical mac) that can have an individual IP in the range assigned. the device may be serving them as dhcp anyway, so you can just set your guests to dhcp and the first 5 will receive ips.

IF its providing NAT services, it likely has a dhcp server on its LAN side, in which case just assign all your guests internally as dhcp and they will receive an internal IP.

 

[SoCalPilot]

It all depends on how your ISP is delivering the IPs to you.

Not sure if this makes a difference but this is a bare metal server from a dedicated web hosting provider. I'm assuming the public IPs are being served directly into the machine.

I didnt think I needed a router VM as I also have another dedicated server running CentOS & cPanel and Im able to easily create a new "site" and assign it one of the dedicated IPs assigned to that server without issue although I understand Im probably not doing a proper comparison.

 

[alexskysilk]

Not sure if this makes a difference but this is a bare metal server from a dedicated web hosting provider.

then its option 1

There are two ways to deal with this:
1. create a dedicated router vm. map the uplink provided by your colo to the vm as eth0, and add a second virtual nic attached to vmbr0. the router will respond to all 5 IPs, and you can NAT traffic to any logical internal address based on whatever rules you require; this is the most flexible and secure method.
2. map the uplink to vmbr0. you can have 5 individual mac addresses, each assigned one of your addresses in your range. this can be your bridge itself or vms.

 

[alexskysilk]

I didnt think I needed a router VM as I also have another dedicated server running CentOS & cPanel and Im able to easily create a new "site" and assign it one of the dedicated IPs assigned to that server without issue although I understand Im probably not doing a proper comparison.

You dont need one, if you intend to use your host as the "router" as you do with your cpanel machine. PVE is a different use case.

 

[SoCalPilot]

2. map the uplink to vmbr0. you can have 5 individual mac addresses, each assigned one of your addresses in your range. this can be your bridge itself or vms.

I thought that is what I was doing by using this setup as the hosting provider configured for me...

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2

auto vmbr0
iface vmbr0 inet static
address 23.227.168.194/29
gateway 23.227.168.193
bridge-ports bond0
bridge-stp off
bridge-fd 0

source /etc/network/interfaces.d/*

Then in the VM I setup vmbr0 as the bridge, but that doesnt appear to be working.

 

[alexskysilk]

please post the vmid.conf for the vm in question, along with the guest network config.

 

[SoCalPilot]

please post the vmid.conf for the vm in question, along with the guest network config.

I believe you mean this for the VM...

But Im not sure exactly what you mean by guest network config, guessing its not this...

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2

auto vmbr0
iface vmbr0 inet static
        address 23.227.168.194/29
        gateway 23.227.168.193
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

Or this....?

If not, please give me additional pointer on what your looking for.

 

[alexskysilk]

thats your host's network config, I need to see the guest (eg, the vm)

 

[SoCalPilot]

thats your host's network config, I need to see the guest (eg, the vm)

Sorry. Would this be what your looking for...

Side Note: I tried a bunch of different "Models" and rebooted after each attempt thinking that would solve the issue with no success.

 

[alexskysilk]

You mentioned that your guest has no connectivity.

what you arent providing is your GUEST'S network configuration. as in, whatever the equivalent is for /etc/network/interfaces in your guest's operating system. It is not possible for us to continue troubleshooting the guest without being inside it.

 

[SoCalPilot]

You mentioned that your guest has no connectivity.

what you arent providing is your GUEST'S network configuration. as in, whatever the equivalent is for /etc/network/interfaces in your guest's operating system. It is not possible for us to continue troubleshooting the guest without being inside it.

Sorry about that. I checked and there was no network file in /etc/network/interfaces and according to a Google search its located in some DB file (All TrueNAS config is stored in SQL tables in /data/freenas-v1.db, and can only be altered via the UI or midclt API calls). Since I can access the VM from another VM on the same host server I was able to take this screenshot:

 

[alexskysilk]

I dont see a gateway defined anywhere.

can you ping the gateway?