What can't be run inside a container and needs a VM?

[H25E]

Searching info about containers vs VM, I'm reading that not everything can be run inside a container but they don't specify why (?) or that some kernel modules can be difficult or impossible to load; but I'm struggling to find a list or some examples of what can't be run inside a container and why.

The only one it's a comment saying that modifying the firewall inside a container it's the same as modifying the host firewall.

I'm ignoring running binaries that need a different kernel than linux, obviously. So, what can't (or shouldn't) be run inside a container and needs a full VM?

 

[Dunuin]

Everything that needs SMB/NFS shares and is accessible from the internet is a bit annoying. Only privileged LXCs will allow you to mount SMB/NFS shares and privileged LXCs shouldn't be used online because of the weak isolation and therefore security risks. You can do a workaround by mounting them on the host and then bind-mounting them into the unprivileged LXC. But that's really annoying when you run a cluster and need to migrate them from time to time between nodes.

And then there are some cases, like docker, where it will run in a LXC but a VM would be recommended.

 

[LnxBil]

So, what can't (or shouldn't) be run inside a container and needs a full VM?

Anything that needs to be secure (so one for the "shouldn't list"). LX(C) containers do not offer good security and that is not going to change due to their design. You only need to break one layer to get access to all resources on the host, in a VM, you need multiple layers.

Software-wise (for the "can't list") : as you already said, anything that needs "real" hardware support (as in kernel support) like GPU, USB, VPNs, VoIP with a "real" hardware time, hardware assisted cryptography, ...

 

[leesteken]

Software-wise (for the "can't list") : as you already said, anything that needs "real" hardware support (as in kernel support) like GPU, USB, VPNs, VoIP with a "real" hardware time, hardware assisted cryptography, ...

Some stuff like USB TV tuners, GPU transcoding, VPN client can work fine by bind-mounting the device nodes and directories (and making sure all permissions and groups are set correctly). Anything that depends on udev running when a USB device is inserted definitely won't work in containers.

 

[H25E]

Everything that needs SMB/NFS shares and is accessible from the internet is a bit annoying. Only privileged LXCs will allow you to mount SMB/NFS shares and privileged LXCs shouldn't be used online because of the weak isolation and therefore security risks. You can do a workaround by mounting them on the host and then bind-mounting them into the unprivileged LXC. But that's really annoying when you run a cluster and need to migrate them from time to time between nodes.

And then there are some cases, like docker, where it will run in a LXC but a VM would be recommended.

Does that means that you can run an SMB/NFS server inside a container but not a client?

It's the SMB/NFS on unprivileged containers something that it's never going to be possible because of containers architecture or it's something "fixable"?

 

[Dunuin]

Does that means that you can run an SMB/NFS server inside a container but not a client?

Jup.

It's the SMB/NFS on unprivileged containers something that it's never going to be possible because of containers architecture or it's something "fixable"?

Not sure. I thought it would be because of security reasons. For NFS/SMB shares to work you need to enable the "CIFS" or "NFS" feature of the LXC and these feature checkboxes are greyed out in the webUI when using an unprivileged LXC.