SolusVM like networking

[Jlourenco]

Hello,

First of all, let me get something clear, my level of network knowledge is very low... I'm a programmer that runs its own servers but I've only recently started dealing with more "complex" networks.
Having this said, currently I own a server with SolusVM on it with something like 20 VMs on it.
Each VM is isolated, have its own IP address, its responsible for it self, can use its own iptables rules and most importantly for me the isolation is not configured at the VM level meaning, I can give access of one VM to a person knowing they wont find a way to another VM.

Recently I acquired another server which will have another 20 IP's or so and I was trying to change from SolusVM to some other virtualiser.
Proxmox seems good and I've installed it on my local server lab to run some tests, so far I'm pretty pleased with it and I actually prefer it over SolusVM. There's only one thing that is still worrying me. I haven't figured out how to configure the network to be like the SolusVM one.
I haven't received the new server yet but once I do, I will have a very limited time to install the Proxmox and configure the basics of it. Testing multiple IP's not easy to do on local since I only have one IP anyway :/


Having all of this said... can please someone give me some guidance, I'm not afraid of learning or having to read but so far I'm not really sure what I actually need to search for to achieve this configuration.


Thanks,
Joao

 

[Dominic]

Hi!

Proxmox VE can also do firewall "above" the VM level using iptables. With an own IP address for each VM, getting them connected to the internet should be relatively straightforward.

Your most valuable resource in this case might be the network chapter of our reference documentation and the firewall chapter, too.

If you have enough computing power locally, you can run Proxmox VE inside Proxmox VE. Depending on your local lab this might help testing the network setup.

Best,
Dominic

 

[Jlourenco]

Hello,

So I've got the server, and I've been trying to setup the network side of it and it's not really working out :/

I've got one NIC with one IP and one IP Block /27.

What I was looking for was a way to at the proxmox level define which public ip is given to each machine, and then on boot the machine would just get that public ip.

So far I have the following configuration:

/etc/sysctl.conf:
net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1


/etc/network/interfaces:
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp196s0
auto enp196s0
iface enp196s0 inet dhcp


# Setup VM networks
auto vmbr0
iface vmbr0 inet static
address 138.xxx.yyy.33
netmask 255.255.255.224
gateway 138.xxx.yyy.63
bridge_ports none
bridge_stp off
bridge_fd 0


My test vm is using the brigde and its not being able to access the Internet for some reason.

Any idea what I did wrong here?
I'm a single person freelancer without much funds but depending on the cost I would be up to paying someone to configure the network side of this for me :/

Thanks,
João Lourenço

 

[Dominic]

Hello!

Is your server at a hosting provider? You might want to try both the routed and the bridged setup.

Why do you use one interface with DHCP? Could you connect your Proxmox VE host to the internet like this?

At a start, I'd recommend sticking as far as possible to the provided documentation. For example, you have net.ipv6.conf.all.forwarding in sysctl.conf but I don't see the proxy_arp line from the routed configuration anywhere. I'd try something like this as starting point:

auto lo
iface lo inet loopback

auto enp196s0
iface enp196s0 inet static
        address  YOUR_SINGLE_IP_HERE
        netmask  255.255.255.255
        gateway  138.xxx.yyy.63
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp196s0/proxy_arp


auto vmbr0
iface vmbr0 inet static
        address  138.xxx.yyy.33
        netmask  255.255.255.224
        bridge_ports none
        bridge_stp off
        bridge_fd 0

What I was looking for was a way to at the proxmox level define which public ip is given to each machine, and then on boot the machine would just get that public ip.

Seems like you are looking for something like Cloud-Init:

Using Cloud-Init, configuration of network devices and ssh keys on the hypervisor side is possible.

However, I would probably do this manually to begin with.

 

[Jlourenco]

Hi!

Thanks for the quick answer, I didn't give out all the information by mistake, indeed I've followed or tried to follow the documentation but it didn't exactly work :/

Yes the server is hosted at hetzner facilities which is making it a bit more troublesome, it's in a part of their network that is still a bit oldish and doesn't offer the possibility of using VNC therefore every-time I miss configure the network I have to ask a Lara to get connected and pay for it hourly.

I've been trying to make the IP static and failed miserably, here are what I when trough:

With the IP set to dhcp I ran ifconfig to check the netmask and broadcast used. (for the primary IP)

enp196s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet MY_SINGLE_IP_HERE  netmask 255.255.255.192  broadcast xxx.yyy.zzz.127

Then I tried to apply the config like u stated:

auto enp196s0
iface enp196s0 inet static
        address  MY_SINGLE_IP_HERE
        netmask  255.255.255.192
        gateway  xxx.yyy.zzz.127
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp196s0/proxy_arp

Doing this resulted in a loss of internet access.
Tbh I've tried a couple of variations of this specific IP and it always failed which is why I gave up and just left it with dhcp.
Also tried to use the post-up section with dhcp but also failed, so I guess they are not so compatible :/

Cloud-init, I saw something about it but I wasn't sure about it, I can research a bit more around that too but indeed I believe getting the network interfaces properly defined should probably be the first thing.

Thanks

 

[Dominic]

With the IP set to dhcp I ran ifconfig to check the netmask and broadcast used. (for the primary IP)

Did you get some instructions for network configuration when you ordered your server? The broadcast address is not the same as the (default) gateway. You can see your current default gateway with ip route.

 

[Jlourenco]

Oh nice tipppp

Okay good thanks!

I got it like this now:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto enp196s0
iface enp196s0 inet static
        address xxx.yyy.zzz.122
        netmask 255.255.255.192
        gateway xxx.yyy.zzz.65
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp106s0/proxy_ar

auto vmbr0
iface vmbr0 inet static
        address xxy.yyz.zzx.33
        netmask 255.255.255.224
        gateway xxy.yyz.zzx.63
        bridge-ports none
        bridge-stp off
        bridge-fd 0

Even with this, when I give to my test VM the vmbr0 network interface it's not being able to access the internet.
Is this all there is to the host and now I just have to check the VM configuration?

Thanks

 

[Dominic]

Is this all there is to the host and now I just have to check the VM configuration?

That means your host is now working?

A host configuration can be as short as this, so if you got the addresses correct (which is a little difficult to check from here) that could/should be it.

 

[Jlourenco]

Sry, yes indeed the host is working, just not the VM's yet. They don't have internet access :/

 

[Dominic]

For the guests you will also have to experiment with their own network configuration. You can use tcpdump on a device to check if ping reaches them.

 

[Jlourenco]

Still haven't found a proper way of setting this up...
I'm trying to avoid as much as I can configuration on the guest but without it and using the "Routed configuration" from the docs it doesn't even have access to the internet.
I see there's an option for buying a "STANDARD" subscription which includes remote support, would this support include this kind of configuration?
I just wanted to get the server up and running and instead I'm just loosing time and time and to be honest I wanted to leave solusvm but if with proxmox I can't even get the network configured I don't think I have much of a choice :/

 

[Dominic]

Hi!

I'm trying to avoid as much as I can configuration on the guest

You will not be able to completely avoid this.

You can try to ping from your guest to an ip address like 8.8.8.8. Then check on the interfaces of the host if those ICMP packages appear using tcpdump. If yes, then you probably have to add routes (ip route add) on the host to the guests.


The Proxmox VE Subscription Agreement mentions

We do not support modified packages, (...) system and network design (...) and high availability design.

Eventually contacting a Proxmox VE reseller might be an idea.