Can't modprobe overlay

[Faris Raouf]

This may have nothing to do with PVE, but I thought I'd better check anyway, because I'm getting really confused.

I have some Centos 7 VMs where I can't load the overlay module using "modprobe overlay":

modprobe: ERROR: could not insert 'overlay': Operation not permitted

I can't see anything in dmesg or /var/log/messages that helps.

Obviously I'm doing this as root. selinux is not enabled.
For Ubuntu rather than Centos, I see various posts elsewhere talking about Secure Boot being a possible cause when the module isn't signed, but that doesn't help here.

One pair of systems is particularly confusing. One is a (full) clone of another VM, made some time ago. On the original VM, I can't load the module. On the clone it loads fine. I've been keeping both VMs up to date at the same time and with the same software, so it is even more mysterious.

Are there any hidden VM config settings that might be the cause? I've compared the configuration of the two systems and they look identical to me, other than for the expected things.

This is for the VM that can't load the module:

agent: 1
boot: c
bootdisk: scsi0
cores: 8
memory: 32768
name: [redacted]
net0: virtio=[redacted],bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-[redacted]-disk-0,discard=on,size=500G
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=[redacted]
sockets: 1
startup: order=1,up=100
vga: serial0

This is the clone

agent: 1
boot: c
bootdisk: scsi0
cores: 8
memory: 20480
name: [redacted]
net0: virtio=[redacted],bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-[redacted]-disk-0,discard=on,size=500G
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=[redacted]
sockets: 1
startup: up=80
vga: serial0

proxmox-ve: 6.1-2 (running kernel: 5.3.18-3-pve)
pve-manager: 6.1-8 (running version: 6.1-8/806edfe1)

 

[wolfgang]

Hi,

in a KVM / Qemu environment, you use the RedHat kernel,
and the loading of modules in this kernel is beyond the control of Proxmox VE.

One pair of systems is particularly confusing. One is a (full) clone of another VM, made some time ago. On the original VM, I can't load the module. On the clone it loads fine. I've been keeping both VMs up to date at the same time and with the same software, so it is even more mysterious.

This is odd because they are fully independent copies.
Anyway, both load an independent Kernel which have no dependencies on each other.

Are there any hidden VM config settings that might be the cause?

No there is only the config file.

I've compared the configuration of the two systems and they look identical to me, other than for the expected things.

The memory is different.

 

[Faris Raouf]

Thanks Wolfgang.

With that knowledge, I did some more experimentation, but to no avail.

But after installing this week's new kernel in the VM and doing a reboot, the problem has mysteriously gone away.
Unfortunately I don't know if it was just the reboot or if it was the kernel that resolved the matter. It is still a mystery really.