VM Windows cannot start with RAM more 48GB

[sand-max]

Hi guys !
I have problem with VM (windows server 2012) .
VM cannot start with RAM more 49152 MB,
I got error:

TASK ERROR: memory size (65536) must be aligned to 2048 for hotplugging

But 65536/2048 = 32.

P.S. I start VM with RAM 48GB and increase RAM to 64GB and that works without problem

 

[sand-max]

I found workaround in post https://forum.proxmox.com/threads/problem-with-hotplug-and-64gb-ram.73621/#post-335003

It's bug and is fix exists?

 

[fabian]

could you post the full config and pveversion -v output? thanks!

 

[sand-max]

could you post the full config and pveversion -v output? thanks!

Config:

agent: 1
balloon: 0
bootdisk: scsi0
cores: 12
cpulimit: 20
hotplug: disk,network,usb,memory,cpu
ide2: none,media=cdrom
memory: 68608
name: term2012-1
net0: virtio=30:65:ec:b0:a9:14,bridge=vmbr0,tag=401
numa: 1
onboot: 1
scsi0: ssd-node6:vm-810-disk-0,cache=writethrough,size=160G
scsihw: virtio-scsi-pci
smbios1: uuid=62063d0b-6b1f-49ab-902c-10aa1e118226
sockets: 2
vcpus: 20
vmgenid: 02a97a49-e25d-4a1b-b9da-74f2f0c1598a

Version:

proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.3: 6.1-6
pve-kernel-5.0: 6.0-11
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve2
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-2
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-12
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-11
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1

 

[fabian]

thanks. yeah, that message is kind of misleading. it's actually the memory minus the amount of static, non-hotpluggable memory which needs to be aligned to 2048.

the logic works as follows (for non-hugepages, with hugepages it's similar but with different numbers)

1. 1024M static memory
2. up to 32 512M dimms
3. up to 32 1024M dimms
4. up to 32 2048M dimms
5. ...

so in your case with 64G, we'd be at step 4 and after plugging in 7 2048M dimms, we'd need to add 1G more memory, but we only have 2048M dimms, so we can't and die. with 65G it works again. with 66G, we are at the same issue after having added 8 2048M dimms, with 67G it works again.

once we've added all 32 2G dimms, the same would repeat but now in 4G increments, and then after 32 of those we'd jump to 8G dimms, etc.

 

[Jackobli]

thanks. yeah, that message is kind of misleading. it's actually the memory minus the amount of static, non-hotpluggable memory which needs to be aligned to 2048.

the logic works as follows (for non-hugepages, with hugepages it's similar but with different numbers)

1. 1024M static memory
2. up to 32 512M dimms
3. up to 32 1024M dimms
4. up to 32 2048M dimms
5. ...

so in your case with 64G, we'd be at step 4 and after plugging in 7 2048M dimms, we'd need to add 1G more memory, but we only have 2048M dimms, so we can't and die. with 65G it works again. with 66G, we are at the same issue after having added 8 2048M dimms, with 67G it works again.

once we've added all 32 2G dimms, the same would repeat but now in 4G increments, and then after 32 of those we'd jump to 8G dimms, etc.

I know, it's an old topic, but still valid.
Could we add the knowledge to this article found in the Wiki?
And I still don't get the maths behind it.
Is there a max of virtuall dimms?
Could we add 32 * 16 GB DIMM? And it would still be necessairy to add another 1024M?

 

[fabian]

there's work going on to improve memory hotplug that should make the whole thing a lot better: https://lists.proxmox.com/pipermail/pve-devel/2023-June/057690.html