problem with hotplug and 64GB ram

Z

z00te

Renowned Member
Feb 3, 2011
7
1
68
hi, i am having problems with some test vm with 64GB Ram and hotplug (no probs with 32GB and until 45056 MiB).
simply the vm refuse to start with the error :

TASK ERROR: memory size (65536) must be aligned to 2048 for hotplugging

there are no other vm on the proxmox
free -m total used free shared buff/cache available Mem: 515644 4616 510466 80 561 509192 Swap: 16383 0 16383



proxmox updated to:
proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
....

there are some restriction to use memory hotplug and vm with a lot of ram?
 
It's not a limitation, it appears to be a bug. I'll look into it, for now a workaround would be to add 1024 MiB of memory to your VM (i.e. make it 65GiB/66560MiB).

Thanks for reporting!
 
tnx for your answer.
if can be useful i have tried the workaround but i get a different error:

kvm: -device pc-dimm,id=dimm59,memdev=mem-dimm59,node=3: a used vhost backend has no free memory slots left TASK ERROR: start failed: QEMU exited with code 1


while it seems there are some free mem:
free -m total used free shared buff/cache available Mem: 515644 112572 359600 88 43471 400179 Swap: 16383 0 16383

tnx
 
This time it actually is a limitation, the kernel sets how many memory dimms per vhost it supports. You can increase the limit by creating a file /etc/modprobe.d/vhost.conf containing options vhost max_mem_regions=128 (default is 64, you can try increasing it further if necessary) and then rebooting the machine.
 
Hi,
ran into the same issue. We are running Proxmox 6.2-10 with kernel 5.4.44-2. /etc/modprobe.d/vhost.conf is set to "options vhost max_mem_regions=509". When trying to start VMs with memory hotplug enabled, and size above approx 40-45GB, they would not start, giving the "TASK ERROR: memory size (65536) must be aligned to 2048 for hotplugging". Tried adding 2048 to memory size given, and didn't help. Made sure the memory size was dividable by 2048, still not working.

Then started it with 32GB and dynamically added memory to wanted 1TB. This worked fine, and the number then calculated by proxmox and set as new value made it possible to boot VM directly into 1TB memory without issues with mem hotplug enabled.

Taking the proxmox calculated new value (1033216) and dividing it by 2048 gave me a "reminder" of 0,5. (504,5)
So trying to boot VM with any memory value giving 0,5 as "reminder" seems to work fine. Doing as the error message suggest and make sure its dividable by 2048 will not start it. Tested OK with several sizes......

Maybe when entering memory sizes manually, proxmox should adjust them to nearest compatible size automatically ?
 
  • Like
Reactions: bjsko
We still have both of these these bugs a year later in pve-manager/6.3-6/2184247e.
I guess we should open a bug report, if noone else has?
 
Hi,
this bug is back in pve7!
Code: 
qm start 100
memory size (51200) must be aligned to 2048 for hotplugging

echo "51200/2048." | bc
25

pveversion 
pve-manager/7.0-11/63d82f4e (running kernel: 5.11.22-4-pve)
If I add 1024, I can start the VM…
I've tested this to know, if we need "options vhost max_mem_regions=509" in /etc/modprobe.d/vhost.conf still in pve7.

Udo
 
  • Like
Reactions: tuxick, mishki, Tacioandrade and 1 other person
Same issue here, was trying to setup a VM with 384GB memory, told to align to 8192

8192*48 = 393216

"TASK ERROR: memory size (393216) must be aligned to 8192 for hotplugging"

1631378750498.png


Code: 
dmidecode -t 16

# dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

Handle 0x1000, DMI type 16, 23 bytes
Physical Memory Array
    Location: System Board Or Motherboard
    Use: System Memory
    Error Correction Type: Multi-bit ECC
    Maximum Capacity: 3 TB
    Error Information Handle: Not Provided
    Number Of Devices: 48

dmidecode -t 17
# dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

Handle 0x1100, DMI type 17, 34 bytes
Memory Device
    Array Handle: 0x1000
    Error Information Handle: Not Provided
    Total Width: 72 bits
    Data Width: 64 bits
    Size: 16 GB
    Form Factor: DIMM
    Set: 1
    Locator: DIMM_A1 
    Bank Locator: Not Specified
    Type: DDR3
    Type Detail: Synchronous Registered (Buffered)
    Speed: 1866 MT/s
    Manufacturer: 00AD00B300AD
    Serial Number: 218AFA3A
    Asset Tag: 01142263
    Part Number: HMT42GR7AFR4C-RD  
    Rank: 2
    Configured Memory Speed: 1333 MT/s

Code: 
# pveversion 
pve-manager/7.0-11/63d82f4e (running kernel: 5.11.22-3-pve)
 
  • Like
Reactions: Jackobli
Hello everyone

I am new to proxmox (just 3 weeks). I am using proxmox 7.1-1 and one of the VMs I have created is an Ubuntu 18.04.5. The pc I am using has a 64 GB RAM.
For CPU and Memory Hotplug and Hot-unplug I followed the instructions from https://pve.proxmox.com/wiki/Hotplug_(qemu_disk,nic,cpu,memory) and from the book Mastering Proxmox, Build virtualized environments using the Proxmox VE hypervisor (2017 third edition), except from this part which I cannot understand (where and what do edit):
"
Memory Hot Unplug
- Memory unplug currently does not work on Windows (<=win10 currently)

- Memory unplug can be unstable on Linux (<= kernel 4.10 currently)

For Linux memory unplug, you need to have movable zone enabled, in the kernel config (not enabled by default on Debian/Ubuntu):

CONFIG_MOVABLE_NODE=YES
and the "movable_node" boot kernel parameter enabled.
"

Nevertheless, CPU Hotplug and Hot-unplug works fine.
For the case of the Memory Hotplug I had issues when the Memory Hotplug Option was enabled. With the VM shut (offline) I gave the RAM value 30000 MiB (the value when the Memory Hotplug Option was disabled, which worked fine) and when I tried to start the VM it gave an error message: memory size (30000) must be aligned to 1024 for hotplugging

I overcame this issue by giving the value 29696 MiB for the RAM (which was "translated" by the GUI of proxmox as 29 GiB), which is 29x1024 MiB. I had no issue hotpluging the memory using 58368=57x1024 MiB or any other integer of multiplication of 1024.

My question involves the hot-unpluging of the memory (decrease of the RAM). Where can I find the kernel config? Where can I find CONFIG_MOVABLE_NODE=YES? Where can I enable the "movable_node" boot kernel parameter (although I edited the grub file for the CPU, I do not understand to what the "movable node" is referring to)?

Best regards (sorry for the large message)

Sotiris
 
Last edited:
  • Like
Reactions: Jackobli
the KConfig was removed in 2017 with kernel 4.13, so likely all modern linux distros you use in a VM have it unconditionally enabled.

the kernel docs have the following to say about the kernel command line switch:
movable_node [KNL] Boot-time switch to make hotplugable memory
NUMA nodes to be movable. This means that the memory
of such nodes will be usable only for movable
allocations which rules out almost all kernel
allocations. Use with caution!
Click to expand...

more details available in the kernel admin guide: https://www.kernel.org/doc/html/latest/admin-guide/mm/memory-hotplug.html
 
running into the same problem here.

TASK ERROR: memory size (65536) must be aligned to 2048 for hotplugging
i'm on proxmox 7.1-11.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back