Installing OpenWRT as LXC. How convert LXC template to Proxmox Template?

[rkk2025]

Hi,

I'm trying to install OpenWRT as LXC. I have a LXC template file that can be inported with the command "lxc image import template.tar.gz". Source to build the image is here: github com mikma lxd-openwrt
and a binary version can be found here: Google Drive Share: 1qQL4-cqmN3Q6e5xUOV1I0MLH-kYHDkxn

Anybody knows how to create a template that works with Proxmox? From what I could see in the official templates from Proxmox the template files simply have the root file system in the root of the archive, but where and how to add the "Guest Operating System Options" in it? Or is the only way for this to work if Proxmox officialy supports OpenWRT?

PD: Was not able to add the URLs directly as it's blocked by antispam. just paste the github into google and it should show up. For the Google share, create a file on your Google drive, get the share link and replace the ID number with the one I posted here, that should work.

 

[Alwin]

but where and how to add the "Guest Operating System Options" in it?

What do you mean by that? If you made a rootfs (x86) and use the naming convention, then a simple creation of the CT on the GUI/CLI should do. There you can select the template then.

<OS>-<OSVERSION>-<NAME>_<VERSION>_<ARCH>.tar.gz

 

[rkk2025]

What do you mean by that? If you made a rootfs (x86) and use the naming convention, then a simple creation of the CT on the GUI/CLI should do. There you can select the template then.

<OS>-<OSVERSION>-<NAME>_<VERSION>_<ARCH>.tar.gz

So I renamed the rootfs file to "openwrt-18.06.1-openwrt_18.06.1_amd64.tar.gz" following the template filename syntax, and when I'm creating the CT I get an error saying "ERROR: unable to detect OS distribution". That's what I actually meant with my previous post, how can I set the os-distribution when it is not officially supported by proxmox. Is there any way to add support for openwrt? Or is this some kind of core change to proxmox to do so?

 

[Alwin]

You possibly need to set the ostype to unmanaged. https://pve.proxmox.com/pve-docs/chapter-pct.html#_guest_operating_system_configuration

 

[rkk2025]

You possibly need to set the ostype to unmanaged. https://pve.proxmox.com/pve-docs/chapter-pct.html#_guest_operating_system_configuration

Nice! It works!! Thank you very much Alwin. Even defining the Network interfaces from Proxmox works! IPs gets set but when you reboot they are lost, but can be persisted with the proper configuration in the system (I guess that's the unmanaged part). The whole system is working flawlessly so far. Thank you very much for the support.

Any chance that OpenWRT will be officially added to Proxmox as a template? Its really great to setup quickly a virtual router for VPN or other network stuff.

 

[Alwin]

Any chance that OpenWRT will be officially added to Proxmox as a template? Its really great to setup quickly a virtual router for VPN or other network stuff.

You may add a feature request and/or send patches for the ostype identification and CT configuration.
https://bugzilla.proxmox.com/

 

[gechu]

Hi @rkk2025, how have things been working out running OpenWRT in a container? Guys over at https://github.com/mikma/lxd-openwrt seems quite active, but i do much prefer to mange things a coherent way through the Proxmox UI.

EDIT: The reason I´m looking into running OpenWRT closer to the host is that I get somewhat crappy bufferbloat scores in dslreport´s test. even though i´m using virtio net drivers.

 

[rkk2025]

Hi @rkk2025, how have things been working out running OpenWRT in a container? Guys over at https://github.com/mikma/lxd-openwrt seems quite active, but i do much prefer to mange things a coherent way through the Proxmox UI.

EDIT: The reason I´m looking into running OpenWRT closer to the host is that I get somewhat crappy bufferbloat scores in dslreport´s test. even though i´m using virtio net drivers.

Hi gechu,

I've been running several OpenWRT containers on Proxmox for almost a year now, and they have been running very snappy and stable so far. I have one virtual router setup as VPN client, another that acts as WireGuard VPN server that I use to interconnect my home networks, I have another virtual router acting as NAT for other Proxmox containers (I use it as easy to manage port forwarding UI) and now recently I discovered that it is possible to pass-trough the host WiFi device so I actually use another virtual router as my wireless AP router at home.

Here is the bug report I opened a while ago regarding this and also has details on how to currently spin up an OpenWRT container in Proxmox:
https://bugzilla.proxmox.com/show_bug.cgi?id=2044

I also tried running OpenWRT in KVM in Proxmox, but I also get extremely bad performance with all network drivers (CPU usage seems to go over the roof whenever there is any kind of network transfer or processing happening inside the VM), for me the VM from the Host side is idling at around 10-20% while the OpenWRT VM is doing nothing, and it only gets worse when I create a WiFi AP where the CPU usage can go much higher.

In the LXC containers I don't have those issues, where CPU usage usually stays between 0 to 0.5% while idling. WiFi CPU consumption happens in the Kernel and therefore is not visible in the LXC CPU usage (But is defenitely just a fraction of what the VM consumed), but performance seems rock solid for me.

I do recommend you to try OpenWRT inside LXC, I'm pretty sure you'll get much better performance.

P.D: I actually also contributed to the mikma/lxd-openwrt repo with fixes (You can now reboot the container properly, while before it would just shutdown on reboot).

 

[gechu]

Excellent! Thanks for quick and helpful reply! Gotto try this out right away!

 

[Di.]

Hello!
Can anyone share a lxd container openwrt for proxmox?
I did not understand how to build it yourself =(

 

[gechu]

Here´s a writeup https://forum.openwrt.org/t/guide-run-openwrt-as-a-container-in-proxmox/44683

I´d be updating it soon with 19.x instructions, ways to upgrade, LXC nic passthrough and more.

Let me know if you still can´t build it

 

[Di.]

Here´s a writeup https://forum.openwrt.org/t/guide-run-openwrt-as-a-container-in-proxmox/44683

I´d be updating it soon with 19.x instructions, ways to upgrade, LXC nic passthrough and more.

Let me know if you still can´t build it

Thank you for the link to the article.
I do not use ubuntu and would not like to install it yet.
Can you drop the file with the container to the file hosting?

 

[gechu]

Hi! Here´s a link to OpenWrt 19.07 RC2 that I compiled a week ago. Note: I did run into some "tainted kernel" issues when I enabled SQM in OpenWrt. Not sure why, but I think it was related to the kernel modules mapping between the host and the container was somehow not fully aligned. Will file a bug report at https://github.com/mikma/lxd-openwrt

Also, read that OpenWrt 19.x should be easier to setup than 18.x (no need to run script to create some "dev" folders) using a precompiled image over at http://www.makikiweb.com/Pi/lxc_openwrt.html. I haven´t tried it yet, but I know Mikael over at https://github.com/mikma/lxd-openwrt also applies a reboot patch that I assume you want

Finally, someone has written a page at the official OpenWrt site about running OpenWrt in a LXC container. I don´t know why the author claims s/he needs to map the user ID`s from the host though..

Hope this gets you going forward! For myself, I´m trying to just stick with OpenWrt on my Ubiquiti EdgeRouter X (ER-X) but I just can´t let the thought go of running OpenWrt in a container in my server to be able to make use of the massive CPU power available to push SQM at my 250 mbit internet connection

Good luck!

 

[Di.]

error when I try to install (

Spoiler: error

()
  Using default stripesize 64.00 KiB.
  For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.
  Logical volume "vm-117-disk-0" created.
  WARNING: Sum of all thin volume sizes (60.00 GiB) exceeds the size of thin pool pve/data and the amount of free space in volume group (13.87 GiB)!
mke2fs 1.43.4 (31-Jan-2017)
Discarding device blocks:    4096/2097152               done                           
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: 647bc538-fcdf-4c4a-91b5-277a3e8fa2d5
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables:  0/64     done                           
Writing inode tables:  0/64     done                           
Creating journal (16384 blocks): done
Multiple mount protection is enabled with update interval 5 seconds.
Writing superblocks and filesystem accounting information:  0/64     done

extracting archive '/mnt/wdblue1tb/template/cache/openwrt-19.07.0-rc2-x86-64-plain.tar.gz'
Total bytes read: 11130880 (11MiB, 83MiB/s)
Detected container architecture: amd64
  Logical volume "vm-117-disk-0" successfully removed
TASK ERROR: unable to create CT 117 - unable to detect OS distribution

 

[pocok]

Hello,

I just created and installed the latest build (19.07.1) using mikma's build script. All is nice, however, before starting the container, I modified the size of the filesystem to 512M (the default is 120M as I have seen). Not exactly sure this is OpenWrt specific or LXC or Proxmox, but when I log in into OpenWrt-s console I can only see 93M of available / space on the "filesystem". Is there a way to expand this to the maximum specified available for this container?

Thank You!

 

[pocok]

Hello,

I just created and installed the latest build (19.07.1) using mikma's build script. All is nice, however, before starting the container, I modified the size of the filesystem to 512M (the default is 120M as I have seen). Not exactly sure this is OpenWrt specific or LXC or Proxmox, but when I log in into OpenWrt-s console I can only see 93M of available / space on the "filesystem". Is there a way to expand this to the maximum specified available for this container?

Thank You!

Never mind me . I expanded first from the Proxmox web GUI, added 0.5G as an increment, and it worked now. df -h in the container shows the proper storage now.

Thanks!

 

[vapour]

.... I just can´t let the thought go of running OpenWrt in a container in my server to be able to make use of the massive CPU power available to push SQM at my 250 mbit internet connection

I'm about to hit the button and install OpenWRT in a container in a new system. Its almost been 1 year since you made the above comment. Do you still feel the same and is there new advice you can add?

Thanks and Cheers
Vapour

 

[gechu]

Hi! I've been running openwrt 19.x as a VPN gateway for quite some time and it works well.

I'm actually about to try openwrt in a container as my primary router, again. I learnt that you probably want to match kernel version between proxmox host and container, especially if you want to use things like sqm which relies on kernel modules.

For proxmox 6.2 I believe it's kernel 5.4. To match that kernel version in openwrt, you have to use a snapshot release.

DazzyWalkman (see link below) provides two github projects: one for building the container template, and another for installation or upgrading.

https://forum.openwrt.org/t/guide-run-openwrt-as-a-container-in-proxmox/44683/15

 

[pocok]

I'm about to hit the button and install OpenWRT in a container in a new system. Its almost been 1 year since you made the above comment. Do you still feel the same and is there new advice you can add?

Thanks and Cheers
Vapour

Hi! I've been running openwrt 19.x as a VPN gateway for quite some time and it works well.

I'm actually about to try openwrt in a container as my primary router, again. I learnt that you probably want to match kernel version between proxmox host and container, especially if you want to use things like sqm which relies on kernel modules.

For proxmox 6.2 I believe it's kernel 5.4. To match that kernel version in openwrt, you have to use a snapshot release.

DazzyWalkman (see link below) provides two github projects: one for building the container template, and another for installation or upgrading.

https://forum.openwrt.org/t/guide-run-openwrt-as-a-container-in-proxmox/44683/15

For you guys above, especially the last one. There is no need to "match" kernel version, as LXC images do not include any sort of a kernel. LXC containers use the hosts kernel. To explain it properly, if you would like to use Wireguard in an OpenWrt LXC container, you need to install wireguard-dkms on the Proxmox host itself, NOT in the OpenWrt container namespace.

All that being said, I was too hestiant at first, but I'm currently running an OpenWrt 19.07.x stable LXC image, as my main router/gateway since a couple of weeks. I have a FTTH "1Gbps" internet connection, usually that maxes out at around 840Mbps, according to fast.com. I have dedicated one of the network adapters as "phys" for the WAN to the container, that is configured with a PPPoE protocol in OpenWrt. The internal LAN part is handeled by one of these dual Intel PCI-E 1Gbit adapters, configured as a trunk LACP bond on the Proxmox host, presented as a Linux Bridge for OpenWrt's LAN adapter side.

I had the luxury of using a Mikrotik RB450G with a special OpenWrt build replacing the stock firmware and shortcut-fe patches enabled. This device could do 950Mbps NAT, measued with iperf, between its separate WAN - LAN (eth0, eth1 switch chips). Right now, I have decomissioned this Miktorik, and exclusively using the OpenWrt LXC container as my primary router, on my i5 4th gen Proxmox host. All I can say, is I never had any issues before with the previous setup, and... I still don't have. It's such a seamless experience, really it's wire speed, what your ISP is providing you. Not to mention, I have set the container limits to 2 cores and 512 MB memory... it is overkill. The avereage usage is like... negligeble amount of ram/cpu for NATing at these speeds. Of course we are talking about an x86 CPU, it doesn't break a single sweat for such workloads.

Basically, it's a workable approach, it can be done, whoever is familiar with OpenWrt on a physical device, I can really recommend containerizing and scailing it. It's such a sleek and stable system with incredibly modest resource usage.

 

[gechu]

Thanks @pocok for sharing your experiences!

In the past I've had issues using SQM in an openwrt 18.x container.

I don't know the exact details, but DazzyWalkman mentions some "out-of-tree" kernel stuff which openwrt used in openwrt 18.x to enable sqm in this post is https://forum.openwrt.org/t/guide-run-openwrt-as-a-container-in-proxmox/44683/12?u=gechu

I never managed to sorted out the kernel issues I faced using openwrt 18x. They may have been related to something else.

@pocok have you tried SQM in container yet?

I need to do some reading regarding "trunk LACP bond", what benefits do you see?