Memory hot plug initial memory allocation

[FingerlessGloves]

I've been trying to workout why with Memory Hotplug enabled, I can not PXE boot a distro installer, even if I have 16GB of memory set with no ballooning. Soon as I turn off hotplug it works no problem.

I've tried adding `memhp_default_state=online` to the kernel line of installer which uses kernel 5.14. I then compared what Proxmox is doing to another kvm based hypervisor and noticed on the raw KVM process arguments the `-m` is starting with 1024mb of memory but on my other kvm based hypervisor it's set to 3072mb, When you disable hotplug that value sets set to the full 16GB.

Now is that just set to start at 1GB of memory or is that related to how the "slots" of memory are allocated when you want to add more memory to the VM?

For the moment I shall have to build the VM with memory hotplug off and then turn it on once the install is done, but wondered if there's a way to fix the problem from the start?

 

[fba]

AFAIK the base memory is 1 GB and any other memory object must be set online by the kernel booted to be available. If the kernel doesn't do that, the other memory dimms are there but not active/usable.
Increasing the base memory, it is set in /usr/share/perl5/PVE/QemuServer/Memory.pm, is possible, but would be changed back on each update of this file, so your workaround is the way to go, imho.

You didn't share the os + cmdline, but it might be Alma/Rocky/RHEL 9 with mentioned kernel 5.14. You could remove rd.live.ram from cmdline and also make the inst.stage2 available via some nfs-target.

 

[FingerlessGloves]

It's EL9 I'm trying to network boot and then use kickstart to build the VM. I think the fix it to temporarily disable hotplug and add it back, which is fine because I'm using ansible to full automate the VM deployment. But it would be nice to be able to specify the base memory. It's the initial initrd that's the issue as it's trying to hold the tftp downloaded initrd and then extract it before the kernel gets a chance to request the hotplugged memory I believe the issue is.

 

[ghandalf]

Hi,

I had a similar issue with a Debian 13 booted over network.
As it was an appliance, where I haven't been able to edit boot or kernel settings, the only option was to disable hotplug memory.
It would be great, if you could define the bare minimum memory in the config, too.

Same issue I had with an very old legacy VM, which has been migrated to Proxmox recently and the services failed after some time with OOM kills.
I thought, why does this VM now needs more memory than before, but the VM was only using 1 GB of the 16 it had.
After manually enabling the memory, it started to work again.

I have created a systemd job to online the memory as I haven't thought about how the kernel could online it automatically...