[SOLVED] "Failed to start Disable Transparent Huge Pages" in lxc container

[clouddicer]

After creating a lxc container on a quiet potent 64GB RAM Server, i am running into this problem while setting up nextcloud through an ansible playbook:

TASK [prep_os : enable service disable-transparent-hugepages] *********************************************************************************************************************************************************************************************************************************************************************************************************
Tuesday 17 March 2020 12:08:26 +0000 (0:00:00.196) 0:00:16.436 *********
fatal: [localhost]: FAILED! => changed=false
msg: |-
Unable to start service disable-transparent-huge-pages.service: Job for disable-transparent-huge-pages.service failed because the control process exited with error code.
See "systemctl status disable-transparent-huge-pages.service" and "journalctl -xe" for details.



root@cloud:~/nextcloud# journalctl -xe

shows me:
-- The unit disable-transparent-huge-pages.service has entered the 'failed' state with result 'exit-code'.
Mar 17 12:08:26 cloud systemd[1]: Failed to start Disable Transparent Huge Pages.


and

root@cloud:~/nextcloud# systemctl status disable-transparent-huge-pages.service
● disable-transparent-huge-pages.service - Disable Transparent Huge Pages
Loaded: loaded (/lib/systemd/system/disable-transparent-huge-pages.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2020-03-17 12:08:26 UTC; 4min 26s ago
Process: 2525 ExecStart=/bin/sh -c /bin/echo never > /sys/kernel/mm/transparent_hugepage/enabled (code=exited, status=2)
Main PID: 2525 (code=exited, status=2)

Mar 17 12:08:26 cloud systemd[1]: Starting Disable Transparent Huge Pages...
Mar 17 12:08:26 cloud sh[2525]: /bin/sh: 1: cannot create /sys/kernel/mm/transparent_hugepage/enabled: Read-only file system
Mar 17 12:08:26 cloud systemd[1]: disable-transparent-huge-pages.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Mar 17 12:08:26 cloud systemd[1]: disable-transparent-huge-pages.service: Failed with result 'exit-code'.
Mar 17 12:08:26 cloud systemd[1]: Failed to start Disable Transparent Huge Pages.


Any suggestions how i can fix this?

I already tried to disable described here: https://forum.proxmox.com/threads/huge-pages-support.15849/#post-81815
on the host, rebooted, but no success.

Your help is very welcome!

Thanks in advance,
clouddicer

 

[Stefan_R]

You cannot change kernel settings from within a container like that. If you want to disable THP, do so from the PVE host, inside the container the 'disable-transparent-huge-page.service' cannot work (it might in privileged, but in unprivileged it doesn't for sure).

 

[clouddicer]

As i wrote i tried this on the host - but that did not work out.

 

[BobhWasatch]

Maybe in container:

systemctl disable disable-transparent-huge-pages

 

[clouddicer]

seems to work

root@cloud:~/nextcloud# systemctl disable disable-transparent-huge-pages
Removed /etc/systemd/system/basic.target.wants/disable-transparent-huge-pages.service.

*checking*

unfortunately not (yet)

this error message:
fatal: [localhost]: FAILED! => changed=false
msg: |-
Unable to start service disable-transparent-huge-pages.service: Warning: The unit file, source configuration file or drop-ins of disable-transparent-huge-pages.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Job for disable-transparent-huge-pages.service failed because the control process exited with error code.
See "systemctl status disable-transparent-huge-pages.service" and "journalctl -xe" for details.

became that:
fatal: [localhost]: FAILED! => changed=false
msg: 'Error loading unit file ''disable-transparent-huge-pages.service'': org.freedesktop.systemd1.BadUnitSetting "Unit disable-transparent-huge-pages.service has a bad unit file setting."'

shorter, but still present.

 

[Stefan_R]

No, I didn't mean disable THP on the host and it will work - I meant you can only do it on the host. That's why your Ansible Playbook keeps failing, since it tries to do it in the container.

Remove the part that installs the 'disable-transparent-huge-pages.service' from your playbook and try again. It does not need to be installed, since it can't work anyway.

 

[clouddicer]

thank you, this is what i am focusing on now.

 

[clouddicer]

Is it actually possible to disable huge pages when creating a new lxc container?
Did this suggestion make it? https://pve.proxmox.com/pipermail/pve-devel/2015-September/017118.html

Disabling it in grub on the host system with
GRUB_CMDLINE_LINUX="transparent_hugepage=never"
did not do the trick in the lxc container.

Though on the host the command
root@agrivolatics:~# systemctl disable disable-transparent-huge-pages
Failed to disable unit: Unit file disable-transparent-huge-pages.service does not exist.

does not work anymore - as expected.

 

[clouddicer]

root@cloud:~# cat /sys/kernel/mm/transparent_hugepage/enabled
actually says:
always [madvise] never

And i found this:
https://lxadm.com/Disabling_transparent_hugepages_on_Ubuntu/Debian
Tried it on the host but it did not work in the client. Still need to fix the ansible script.
Any suggestion where to flip the right switch?
https://github.com/ReinerNippes/nextcloud
There are the two lines i commeted out in -> roles -> prep_os -> disable-transparent-huge-pages.service :

#ExecStart=/bin/sh -c '/bin/echo never > /sys/kernel/mm/transparent_hugepage/enabled'
#ExecStart=/bin/sh -c '/bin/echo never > /sys/kernel/mm/transparent_hugepage/defrag'

but it still trows the error.

 

[Stefan_R]

To disable in your Ansible script, why not just keep the service disabled? That should do the trick, i.e. in 'roles/prep_os/tasks/main.yml' change:

- name: enable service disable-transparent-hugepages
  service:
    name: disable-transparent-huge-pages.service
    enabled: true
    state: started

'enabled' to 'false' and then call ansible to redeploy your container.

There are the two lines i commeted out in -> roles -> prep_os -> disable-transparent-huge-pages.service :

#ExecStart=/bin/sh -c '/bin/echo never > /sys/kernel/mm/transparent_hugepage/enabled'
#ExecStart=/bin/sh -c '/bin/echo never > /sys/kernel/mm/transparent_hugepage/defrag'

Having no ExecStart is not a valid service file. Keep them in, or leave "ExecStart" but change the command to do nothing (e.g. just call 'echo' or whatever instead of the whole transparent_hugepage deal).

Disabling it in grub on the host system with
GRUB_CMDLINE_LINUX="transparent_hugepage=never"
did not do the trick in the lxc container.

Can you elaborate that point further? This should have worked. Did you call update-grub afterwards? Are you sure you're not using systemd-boot instead of Grub (this happens when you use a ZFS root on UEFI)? Have you rebooted afterwards? '/proc/cmdline' says?

 

[clouddicer]

Starting with your last suggestion first:
- yes, updated grub, no, suprisingly still did not work after reboot

Editing the main.yml file did the trick. Thank you a thousand times!

Though now the redis-server installation failed - fixed it with the nesting option of the container.

Made my day. Consider me a new subscriber of Proxmox.

Greetings,
Clouddicer

To disable in your Ansible script, why not just keep the service disabled? That should do the trick, i.e. in 'roles/prep_os/tasks/main.yml' change:

- name: enable service disable-transparent-hugepages
  service:
    name: disable-transparent-huge-pages.service
    enabled: true
    state: started

'enabled' to 'false' and then call ansible to redeploy your container.


Having no ExecStart is not a valid service file. Keep them in, or leave "ExecStart" but change the command to do nothing (e.g. just call 'echo' or whatever instead of the whole transparent_hugepage deal).


Can you elaborate that point further? This should have worked. Did you call update-grub afterwards? Are you sure you're not using systemd-boot instead of Grub (this happens when you use a ZFS root on UEFI)? Have you rebooted afterwards? '/proc/cmdline' says?