Ubuntu 26.04 lxc can't resolve DNS

P

piersonjarvis

Renowned Member
Aug 27, 2018
12
1
66
29
Hello, I'm running into an interesting issue.

I've setup a number of lxc containers, recently I went through a wave of upgrade from ubuntu 24.04 to 26.04 using the built in do-release-upgrade command within the LXC, after the upgrade programs like nslookup or dig no longer work, and internal application running on these containers are having issues resolving dns records. Not all records, but anything that uses my internal DNS servers.

In way of troubleshooting I've tried the following with success replicating my issue. Tested nslookup on an ubuntu 24.04 lxc, worked as expected, upgraded that same lxc via do-release-upgrade then ran nslookup and got no result. No error, just drops me back to a normal prompt. I can't even type `nslookup` and drop into the applications shell to do lookup. Same with dig.

I then tried a fresh 26.04 lxc and got the very same results, nslookup and gid don't do anything whatsoever. I've tried moving hosts, I haven't tried updating hosts yet, running on 9.1.9 currently in my cluster. Next step could very well be updates to the host.

Just curious if anyone else has seen the same or can replicate.
 
leesteken said:
Tried it with PVE 9.2.21 and the Ubuntu 26.04 template. nslookup download.proxmox.com and dig don't give any output. apt update and apt dist-upgrade work fine (and also use DNS).
Click to expand...
Amazing so I'm not crazy! Nslookup and dig just fail to do anything for you as well?
 
Check the exit code right after the command returns:

dig download.proxmox.com; echo "exit: $?"

On the Proxmox host, watch the kernel log in a second shell while you repeat the command in the container:

journalctl -kf | grep -i -E 'apparmor|denied|segfault'

If an `apparmor="DENIED"` line appears for `dig` or `nslookup`, the container's AppArmor profile is likely blocking it. A segfault line, or an exit code above 128, means the binary itself crashed.

If neither shows anything, install `strace` in the container and run `strace -f dig download.proxmox.com 2>&1 | tail -20` to see the last system calls before it exits.

Drafted with AI.
 
  • Like
Reactions: Kingneutron, piersonjarvis and leesteken
I can confirm the issue.
Althou only one of my lxc is impacted the second one is working fine after th upgrade.
The lxc with the issue looses all ip configuration after startup.
A ping to an IP gets a Network not reachable
I am running PVE 9.2.21
 
Last edited:
et-ducky said:
Check the exit code right after the command returns:

dig download.proxmox.com; echo "exit: $?"

On the Proxmox host, watch the kernel log in a second shell while you repeat the command in the container:

journalctl -kf | grep -i -E 'apparmor|denied|segfault'

If an `apparmor="DENIED"` line appears for `dig` or `nslookup`, the container's AppArmor profile is likely blocking it. A segfault line, or an exit code above 128, means the binary itself crashed.

If neither shows anything, install `strace` in the container and run `strace -f dig download.proxmox.com 2>&1 | tail -20` to see the last system calls before it exits.

Drafted with AI.
Click to expand...
Okay finally got around to working through these commands, and found it is indeed AppArmor that is the problem. Below is the output of the journalctl command you listed.

kernel: audit: type=1400 audit(1792546962.857:19902): apparmor="DENIED" operation="file_inherit" class="file" namespace="root//lxc-111_<-var-lib-lxc>" profile="dig" name="/dev/tty1" pid=3268808 comm="dig" requested_mask="wr" denied_mask="wr" fsuid=100000 ouid=100000
 
The denial is on `/dev/tty1`, the console you're typing in, so check whether the lookup itself works when dig writes to a file:

dig download.proxmox.com > /tmp/dig.out 2>&1; cat /tmp/dig.out

Does that show an ANSWER SECTION with an address?

Drafted with AI.
 
Last edited:
et-ducky said:
The denial is on `/dev/tty1`, the console you're typing in, so check whether the lookup itself works when dig writes to a file:

dig download.proxmox.com > /tmp/dig.out 2>&1; cat /tmp/dig.out

Does that show an ANSWER SECTION with an address?

Drafted with AI.
Click to expand...
Nope, completely empty.
 
Leave this running on the host:

journalctl -kf | grep -i apparmor

Then run the redirected `dig` in the container again. Which `operation=` and `name=` values show up on the new DENIED lines?

Drafted with AI.
 
I can confirm the same as above, for a Ubuntu LXC with template ubuntu-26.04-standard.

You can also try other cli commands in the container (non-nw) such as a simple lsblk, which will also fail through apparmor.

Workaround (BUT NOT ADVISABLE!); is to remove apparmor container-side. You will still be covered host-side, but this is probably still insufficient in a production setting.

So IN THE CONTAINER:
Code: 
(sudo) apt purge apparmor -y
Tested & working.
 
Last edited:
et-ducky said:
Leave this running on the host:

journalctl -kf | grep -i apparmor

Then run the redirected `dig` in the container again. Which `operation=` and `name=` values show up on the new DENIED lines?

Drafted with AI.
Click to expand...
Here is the output of those denied lines

apparmor="DENIED" operation="file_inherit" class="file" namespace="root//lxc-111_<-var-lib-lxc>" profile="dig" name="/tmp/dig.out"

So `operation="file_inherit"` and `name="/tmp/dig.out"`

And if I run just a normal `dig` command i.e. `dig download.proxmox.com` I get the same operation failure with the name "/dev/tty1" instead.
 
  • Like
Reactions: leesteken
piersonjarvis said:
Here is the output of those denied lines

apparmor="DENIED" operation="file_inherit" class="file" namespace="root//lxc-111_<-var-lib-lxc>" profile="dig" name="/tmp/dig.out"

So `operation="file_inherit"` and `name="/tmp/dig.out"`

And if I run just a normal `dig` command i.e. `dig download.proxmox.com` I get the same operation failure with the name "/dev/tty1" instead.
Click to expand...
I also tried the above with nslookup and got a similar denied but with slightly different operation and name. With nslookup the operation is "open" while the name is "/proc/sys/net/ipv4/ip_local_port_range"

operation="open" class="file" namespace="root//lxc-111_<-var-lib-lxc>" profile="nslookup" name="/proc/sys/net/ipv4/ip_local_port_range"

I also want to note I tried to switch the dns of the container to a specific list of IP's instead of inherit from the host. That provided the same apparmor deny as above.
 
The `/dev/tty1` denial matches an RFC patch a Proxmox developer posted in June, which adds `/dev/tty[0-9]` to the console rules Proxmox writes for containers and names the `dig` profile in the Ubuntu 26.04 template https://lore.proxmox.com/all/20260622164522.836988-1-s.ivanov@proxmox.com/T/

That patch covers only the console path, so the `/tmp/dig.out` and `ip_local_port_range` denials likely come from the `dig` and `nslookup` profiles themselves.

Unload just those two profiles in the container instead of purging AppArmor. Confirm the file names first, then link them into `disable/` so they stay off after a restart:

ls /etc/apparmor.d/ | grep -E '^(dig|nslookup)$'
sudo ln -s /etc/apparmor.d/dig /etc/apparmor.d/nslookup /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/dig /etc/apparmor.d/nslookup

The rest of the container's profiles stay enforced.

Drafted with AI.
 
You must log in or register to reply here.
Top Bottom
Back