[lxc/#676] Feature Request: Physical NIC assignment for LXC containers in Proxmox 4.0

B

batonac

Renowned Member
Aug 2, 2015
15
5
68
York, PA, USA
avu.nu
Hi All,

I've just joined the forum, and looking forward to interactions here.

I'm hoping this is an acceptable to place to submit/discuss a feature request. I'm playing with PVE 4.0 and especially with the LXC integration. In my configuration, I'd really like to be able to connect one of my containers to a physical NIC on the host (without bridging). In a standard LXC config, this is a simple as:
Code: 
lxc.network.type = phys
lxc.network.link = eth0
lxc.network.name = wan0
I'm hoping such a simple config means it wouldn't be hard to integrate into Proxmox. I'm also hoping there's not some precedent against this, as I feel it could be a really useful feature.

BTW, thanks for such a great product. PVE is awesome.
 
Re: [lxc/#676] Feature Request: Physical NIC assignment for LXC containers in Proxmox

Wow, thanks. That was fast. I appreciate your prompt response. Looking forward to this development!
 
bmwt said:
any update on this? Even a variable for 100.conf would be great, no need for gui support just something we can stick in the config would be great.

thanks!

-brendan
Click to expand...

Ditto regarding this.

I've been trying to run ntopng in a container (from a wswitch port copied from my WAN port) for the longest time, and I have not been able to get the Physical NIC -> VMBR bridge -> LXC container configuration to all be properly promiscuous no matter what I do.

I read in the LXC docs about the "phys" option in standard LXC, and thought this might help.

In the meantime, until this feature is added to the gui, can I just edit my config line below:

net1: name=eth1,bridge=vmbr3,hwaddr=66:64:33:30:32:62,type=veth
Click to expand...

to

net1: name=eth1,link=eth3,type=phys
Click to expand...

Would this work? If not, is there another way to accomplish this in the meantime?

Thanks,
Matt
 
that won't work, the netX values only allow veth network definitions.

what you can do though is set "lxc.network.XX" (all the network options from "man lxc.container.conf" are accepted and passed through to LXC by proxmox) - but please note that these interfaces are then only managed by LXC and not integrated in any way into the proxmox tools (i.e., not included in the firewall, no visibility in the GUI, no setup in the container, no checks whatsoever). in other words, this is an advanced feature that is not really supported by us, but is there for situation where you absolutely need the power and flexibility that it offers.
 
fabian said:
that won't work, the netX values only allow veth network definitions.

what you can do though is set "lxc.network.XX" (all the network options from "man lxc.container.conf" are accepted and passed through to LXC by proxmox) - but please note that these interfaces are then only managed by LXC and not integrated in any way into the proxmox tools (i.e., not included in the firewall, no visibility in the GUI, no setup in the container, no checks whatsoever). in other words, this is an advanced feature that is not really supported by us, but is there for situation where you absolutely need the power and flexibility that it offers.
Click to expand...

Fabian, thank you.

Would these go as separate lines in the same "/etc/pve/lxc/xxx.conf" file?

Thanks,
Matt
 
yes, you can check "man lxc.container.conf" for the details (network devices are grouped automatically, so the order is important). PVE has a whitelist for lxc options that are passed through, all of them are copied as-is to the generated lxc configuration file.
 
Well,

I tried the following config:

Code: 
# cat /etc/pve/lxc/150.conf 
arch: amd64
cpulimit: 2
cpuunits: 1024
hostname: ntopng
memory: 2048
net0: name=eth0,bridge=vmbr0,gw=xxx.xxx.xxx.xxx,hwaddr=66:35:62:65:38:62,ip=xxx.xxx.xxx.xxx/xx,type=veth
ostype: ubuntu
rootfs: volume=container:subvol-150-disk-1,size=32G
swap: 512
lxc.network.type = phys
lxc.network.link = eth3
lxc.network.name = eth1

This appeared to work at first, but upon rebooting the container, it doesn't appear to return the eth3 device to the host. It goes missing somehow. eth3 just disappears. This results in the container failing to boot back up again.
 
you could try starting the container in foreground mode with debug output ("lxc-start -n ID -F -ldebug -o pathtologfile") and then shutting it down. this should give some output about the network devices as well.
 
Yeah, the problem is not that I don't understand why the container won't start.

Let me clarify.

The container started just fine the first time, and used the physical interface I configured properly.

It failed to start subsequent times, because once restarted, the physical NIC i told it to use (eth3) disappeared from the host. So now the container will no longer start because the NIC on the host I am telling it to use, no longer exists, and this results in an error.

Code: 
# ifconfig eth3
eth3: error fetching interface information: Device not found

So clearly, a container that is being told to use eth3 as its physical NIC is not going to be able to run if the eth3 device has disappeared.

The part I am trying to understand is why eth3 disappeared after shutting down the container. I feel it must have something to do with how LXC takes control of an Ethernet interface when you assign it with phys. It must have somehow failed to properly return the interface to its original state on the host.

This might be a bug. Once I get to a point where I can reboot my server and get eth3 back (I cant right now, other stuff is running) I am going to see if I can reproduce it.
 
So, yes, I just tested, and this is totally reproducible with latest Enterprise packages in 4.2 and an Ubuntu 14.04 LXC container

Steps to reproduce:

1.) Create LXC Container, with the following lines in the config file:
Code: 
lxc.network.type = phys
lxc.network.link = *an available host ethernet device*  (for me, eth3)
lxc.network.name = *name of said device in container* (for me, eth1)

2.) Check Host Ethernet device (eth3) before: (from host console)
Code: 
# ifconfig eth3
eth3      Link encap:Ethernet  HWaddr 00:1b:21:88:ab:cb  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:42 Memory:faee0000-faf00000

3.) Start Container.

4.) Verify properly working physically forwarded Ethernet device inside container (for me, eth1)

5.) Issue reboot command in container.

Container shuts down as expected, but fails to restart. Attempting to manually start container results in an error.

6.) Check Host Ethernet device (eth3) after: (from host console)
Code: 
# ifconfig eth3
eth3: error fetching interface information: Device not found

I have no idea if this is a upstream LXC issue or something related to the Proxmox implementation, but I wanted to make it known.
 
mattlach said:
So, yes, I just tested, and this is totally reproducible with latest Enterprise packages in 4.2 and an Ubuntu 14.04 LXC container

Steps to reproduce:

1.) Create LXC Container, with the following lines in the config file:
Code: 
lxc.network.type = phys
lxc.network.link = *an available host ethernet device*  (for me, eth3)
lxc.network.name = *name of said device in container* (for me, eth1)

2.) Check Host Ethernet device (eth3) before: (from host console)
Code: 
# ifconfig eth3
eth3      Link encap:Ethernet  HWaddr 00:1b:21:88:ab:cb 
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:42 Memory:faee0000-faf00000

3.) Start Container.

4.) Verify properly working physically forwarded Ethernet device inside container (for me, eth1)

5.) Issue reboot command in container.

Container shuts down as expected, but fails to restart. Attempting to manually start container results in an error.

6.) Check Host Ethernet device (eth3) after: (from host console)
Code: 
# ifconfig eth3
eth3: error fetching interface information: Device not found

I have no idea if this is a upstream LXC issue or something related to the Proxmox implementation, but I wanted to make it known.
Click to expand...

I will look into it. I actually meant to run this test in foreground mode (i.e., the first working boot) to see what LXC logs on shutdown/cleanup. But since it looks reproducible, I will do it myself.
 
fabian said:
I will look into it. I actually meant to run this test in foreground mode (i.e., the first working boot) to see what LXC logs on shutdown/cleanup. But since it looks reproducible, I will do it myself.
Click to expand...


Please let me know if I can provide any further info!

--Matt
 
seems to work here, at least with a "dummy" interface - will try later with a real physical NIC interface.

if it is possible, a debug log create with lxc-start would be nice to have. here it shows the following when I power off the container, and the network device "test" is returned to the host correctly:
Code: 
...
      lxc-start 1467615185.311 DEBUG    lxc_start - start.c:signal_handler:316 - container init process exited
      lxc-start 1467615185.311 DEBUG    lxc_start - start.c:__lxc_start:1239 - Container halting
      lxc-start 1467615185.311 DEBUG    lxc_start - start.c:__lxc_start:1254 - Pushing physical nics back to host namespace
      lxc-start 1467615185.311 INFO     lxc_conf - conf.c:lxc_rename_phys_nics_on_shutdown:2662 - running to reset 1 nic names
      lxc-start 1467615185.333 INFO     lxc_conf - conf.c:lxc_rename_phys_nics_on_shutdown:2666 - resetting nic 7 to test
...

a reboot within the container also works as expected.
 
some more digging showed the root of the problem:

lxc does the following steps for phys network devices:
  1. move devices back to host network namespace
  2. rename devices to original names
the first step will fail if the device that is moved has a name (in the container) which already exists (on the host). in that case, the second step will also fail, because now the device that should be renamed (in the host name space) does not exist because it was not moved in step 1 (it is accessed by index, not by name).

the easy workaround for now is to use a device name in the container which does not exist on the host (the name does not have to start with "eth" btw ;)). I will check whether this issue still exists in LXC 2.0

in any case, as soon as the LXC process exits, the device should be moved back to the host namespace anyway (because the container name space ceases to exist) - but it will have the wrong name (in my case, it was consistently renamed to "dev5" because it was the 5th device). you can simply rename it with "ip link set OLDNAME name NEWNAME", e.g. "ip link set dev5 name eth3". then the container should boot again.
 
I would just like to +1 this feature request.

We are using the workaround to pass a SR-IOV VF to a LXC container so we can run Suricata at wirespeed. For the most part it works, but if you change the "lxc.network.name" in the vmid.conf it will not be picked up until the host is rebooted.

It would be nice to have a supported/official way to do this. We are in the age of NFV and this requirement will become more and more common.
 
+1 for this, it would be great that there would be a GUI option to define/assign a network interface as physical for an LXC container. As the "veth" type interfaces are already neatly listed under the "network" tab already.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back