[SOLVED] Memory ballooning, PCIE passthrough booting from nvme

O

ozioh

Active Member
Feb 12, 2017
12
0
41
46
Hi,
I passthrough my nvme disk and boot directoy from it for Windows VM.
When I use memory ballooning, my VM gets assinged max memory and it never goes back down.
I have unraid VM also, for it, memory ballooning seems working. It starts with the min memory.

Anyone can tell me how I can get it work for the windows VM?
I have installed virtio drivers for the ballooning.

Thank you.
 
Last edited:
When using PCI(e) passthrough, all VM memory must be pinned into actual host memory because of possible device-initiated DMA. Ballooning and KSM won't work for such a VM in Proxmox. Manual memory hot-(un)plug can work fine, if you configure the operating system inside the VM correctly.
 
  • Like
Reactions: kaja
Thank you. Exactly what happened. My memory usage was increasing 100mb every second in Windows VM. I disabled ballooning.
 
Last edited:
leesteken said:
When using PCI(e) passthrough, all VM memory must be pinned into actual host memory because of possible device-initiated DMA. Ballooning and KSM won't work for such a VM in Proxmox. Manual memory hot-(un)plug can work fine, if you configure the operating system inside the VM correctly.
Click to expand...
Sorry for asking so late into the party but could you clarify if this apply to all guest VM's running on host with PCI(e) passthrough enabled or just the VM that getting PCI(e) device?

I'm asking since it looks like all my other VM's are getting issues with ballooning enabled on them :(
 
Chris_ said:
Sorry for asking so late into the party but could you clarify if this apply to all guest VM's running on host with PCI(e) passthrough enabled or just the VM that getting PCI(e) device?

I'm asking since it looks like all my other VM's are getting issues with ballooning enabled on them :(
Click to expand...
Passthough is per VM setting, not a general setting of a Proxmox host. VMs that are running with one or more PCI(e) passthrough need all of their memory pinned and dynamic memory management does not work for those VMs (memory hot-(un)plug does appear to work though). VMs that don't use PCI(e) passthrough are not affected.
 
  • Like
Reactions: i_am_jam
melroy89 said:
What about nvme pass-through is that also considered PCIe pass-through? And therefor you should disable memory ballooning?
Click to expand...
Ayn NVMe is a PCIe device (and can do DMA), so yes.Other flash memory drives like USB or SATA M.2 drives are not PCI(e) devices and don't require VM memory pinning into host RAM.

EDIT: Correction, if you use disk passthrough of a NVMe drive or partition (and don't use any PCI(e) passthrough), then it's the same as other drives/disks/partitions and memory pinning is not necessary and ballooning can work.
 
Last edited:
  • Like
Reactions: melroy89
leesteken said:
Ayn NVMe is a PCIe device (and can do DMA), so yes.Other flash memory drives like USB or SATA M.2 drives are not PCI(e) devices and don't require VM memory pinning into host RAM.

EDIT: Correction, if you use disk passthrough of a NVMe drive or partition (and don't use any PCI(e) passthrough), then it's the same as other drives/disks/partitions and memory pinning is not necessary and ballooning can work.
Click to expand...
OK.. uhm. Let's be clear for once and for all, I have 4 disks in total, see below. Two nvme SSDs and 1 SATA SSD via SCSI and finally 1 virtual disk (the VM disk)..
Can this support ballooning?

1721035259571.png
 
melroy89 said:
OK.. uhm. Let's be clear for once and for all, I have 4 disks in total, see below. Two nvme SSDs and 1 SATA SSD via SCSI and finally 1 virtual disk (the VM disk)..
Can this support ballooning?

View attachment 71355
Click to expand...
You are using disk passthrough, so it can use ballooning and it does not matter what kind of disk drive (NVMe, SATA, IDE, USB, (i)SCSI, etc.). I don't see any PCI(e) passthrough in your screenshot.
PS: Often it is more readable (and complete) to show the VM configuration file (or use qm showconfig VM_ID_NUMBER).
 
Last edited:
  • Like
Reactions: melroy89
leesteken said:
You are using disk passthrough, so it can use ballooning and it does not matter what kind of disk drive (NVMe, SATA, IDE, USB, (i)SCSI, etc.). I don't see any PCI(e) passthrough in your screenshot.
PS: Often it is more readable (and complete) to show the VM configuration file (or use qm showconfig VM_ID_NUMBER).
Click to expand...
I think you mean: qm config <VM_ID>.

Anyway, there is the output indeed:

Code: 
qm config 100
agent: 1
boot: order=virtio0;ide2
cores: 24
cpu: host,flags=+pdpe1gb;+aes
description: My Ubuntu Server
ide2: none,media=cdrom
memory: 98304
meta: creation-qemu=7.2.0,ctime=1679853594
name: Ubuntu-Server
net0: virtio=82:50:C2:6F:09:CD,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
parent: first-snapshot
scsi1: /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_S7DPNJ0W914407X,backup=0,discard=on,size=3907018584K,ssd=1
scsi2: /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_S7DPNJ0W914464F,backup=0,discard=on,size=3907018584K,ssd=1
scsi3: /dev/disk/by-id/ata-CT4000MX500SSD1_2320E6D56C26,backup=0,discard=on,size=3907018584K
scsihw: virtio-scsi-single
smbios1: uuid=0c297d03-057c-4fd1-9e2d-2d22c7d861d2
sockets: 1
tablet: 0
tags: linux
virtio0: vmdata:vm-100-disk-0,discard=on,iothread=1,size=300G
vmgenid: e99c1565-ec29-448c-b40a-e0f0eb13c05a
 
melroy89 said:
Code: 
qm config 100
agent: 1
boot: order=virtio0;ide2
cores: 24
cpu: host,flags=+pdpe1gb;+aes
description: My Ubuntu Server
ide2: none,media=cdrom
memory: 98304
meta: creation-qemu=7.2.0,ctime=1679853594
name: Ubuntu-Server
net0: virtio=82:50:C2:6F:09:CD,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
parent: first-snapshot
scsi1: /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_S7DPNJ0W914407X,backup=0,discard=on,size=3907018584K,ssd=1
scsi2: /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_S7DPNJ0W914464F,backup=0,discard=on,size=3907018584K,ssd=1
scsi3: /dev/disk/by-id/ata-CT4000MX500SSD1_2320E6D56C26,backup=0,discard=on,size=3907018584K
scsihw: virtio-scsi-single
smbios1: uuid=0c297d03-057c-4fd1-9e2d-2d22c7d861d2
sockets: 1
tablet: 0
tags: linux
virtio0: vmdata:vm-100-disk-0,discard=on,iothread=1,size=300G
vmgenid: e99c1565-ec29-448c-b40a-e0f0eb13c05a
Click to expand...
Ballooning could work but you have not activated it (by setting a lower amount for Minimum memory, under Advanced), and it will only start when the host memory usage is 80%: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_memory
 
  • Like
Reactions: melroy89
leesteken said:
Ballooning could work but you have not activated it (by setting a lower amount for Minimum memory, under Advanced), and it will only start when the host memory usage is 80%: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_memory
Click to expand...
AH nice. Got it!


1721049032660.png
Code: 
agent: 1
balloon: 65536
boot: order=virtio0;ide2
cores: 24
cpu: host,flags=+pdpe1gb;+aes
description: My Ubuntu Server
ide2: none,media=cdrom
memory: 98304
meta: creation-qemu=7.2.0,ctime=1679853594
name: Ubuntu-Server
net0: virtio=82:50:C2:6F:09:CD,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
parent: first-snapshot
scsi1: /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_S7DPNJ0W914407X,backup=0,discard=on,size=3907018584K,ssd=1
scsi2: /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_4TB_S7DPNJ0W914464F,backup=0,discard=on,size=3907018584K,ssd=1
scsi3: /dev/disk/by-id/ata-CT4000MX500SSD1_2320E6D56C26,backup=0,discard=on,size=3907018584K
scsihw: virtio-scsi-single
shares: 3000
smbios1: uuid=0c297d03-057c-4fd1-9e2d-2d22c7d861d2
sockets: 1
tablet: 0
tags: linux
virtio0: vmdata:vm-100-disk-0,discard=on,iothread=1,size=300G
vmgenid: e99c1565-ec29-448c-b40a-e0f0eb13c05a
 

Attachments

  • 1721048821221.png
    1721048821221.png
    8.2 KB · Views: 2
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back