[SOLVED] VM won't start with PCI passthrough after upgrade to 9.0

I thought it was included in the update.
Or is there an error in this setting?

root@pve:~# qm set 629 -args '-cpu host,level=35,hv_passthrough,+vmx,guest-phys-bits=39'
update VM 629: -args -cpu host,level=32,hv_passthrough,+vmx,guest-phys-bits=39
root@pve:~# qm start 629
kvm: -device vfio-pci,host=0000:00:02.1,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0,romfile=/usr/share/kvm/alderlake_igpu_N100.bin: vfio 0000:00:02.1: Failed to set vIOMMU: aw-bits 48 > host aw-bits 39
start failed: QEMU exited with code 1
root@pve:~# apt policy qemu-serverqemu-server:
Installed: 9.0.18
Candidate: 9.0.18
root@pve:~# cat /proc/cpuinfo
address sizes : 39 bits physical, 48 bits virtual
Click to expand...

https://github.com/proxmox/qemu-server/commit/bf3fc3ce6d4480075dfd1e33d0b95f29c7a511ab

qemu-server (9.0.18) trixie; urgency=medium
* close #6378: expose guest-phys-bits CPU option.
 
Last edited:
I think it is to be added to machine but I can't add it, I think it is not augs but I can add it but it won't start.

I would appreciate it if you could tell me how to set this up.

root@pve:/etc/pve/qemu-server# qm set 629 -machine q35,viommu=intel,aw-bits=39
400 Parameter verification failed.
machine: invalid format - format error
machine.aw-bits: property is not defined in schema and the schema does not allow additional properties

qm set <vmid> [OPTIONS]
root@pve:/etc/pve/qemu-server# qm set 629 -machine q35,viommu=intel
update VM 629: -machine q35,viommu=intel
root@pve:/etc/pve/qemu-server# qm set 629 -machine q35,viommu=intel,aw-bits=39
400 Parameter verification failed.
machine: invalid format - format error
machine.aw-bits: property is not defined in schema and the schema does not allow additional properties

qm set <vmid> [OPTIONS]
root@pve:/etc/pve/qemu-server# qm set 629 -args '-cpu host,level=32,hv_passthrough,+vmx,guest-phys-bits=39,aw-bits=39'
update VM 629: -args -cpu host,level=32,hv_passthrough,+vmx,guest-phys-bits=39,aw-bits=39
root@pve:/etc/pve/qemu-server# qm start 629kvm: can't apply global host-x86_64-cpu.aw-bits=39: Property 'host-x86_64-cpu.aw-bits' not found
start failed: QEMU exited with code 1
Click to expand...
 
think said:
You won't even get to the live session / setup: Bootscreen runs endless.
Click to expand...
I can't speak to your setup, but I just installed Manjaro setting it up similar to my debian setup in which I passthrough my video card minus the video card as it was in use, but everything else was set the same. I was able to both boot and run manjaro without issue using virtio viommu, but I will say that the qemu-guest-agent is not installed in the live disk, nor does it install by default. That being said both the live installer and the fully installed (plus agent) didn't crash. Now I wasn't paying attention when I first installed it and was using the i440fx machine instead of the q35 machine and when enabling viommu it did indeed crash the installer and the installed system, but starting over with the q35 (v9.2) I was able to do exactly as my previous system.
 
Just to verify for me because I'm running TrueNAS in a VM with PCI passthrough: Is/Was this only a problem with Intel CPUs or did anyone experience problems with AMD (Ryzen 5 5500GT in my case) as well?
 
Last edited:
You can check if you're affected by the following command.

AMD is 48-bit so it's unlikely, but please check and make a decision.

cat /proc/cpuinfo

address sizes : 48 bits physical, 48 bits virtual
 
Good morning, I’m getting the exact same error.


kvm: -device vfio-pci,host=0000:00:17.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio 0000:00:17.0: Failed to set vIOMMU: aw-bits 48 > host aw-bits 39
TASK ERROR: start failed: QEMU exited with code 1.


With the settings below, my system does boot; however, a different error message appears. It seems to work anyway.

#boot: order=scsi0;ide2;net0
#cores: 4
#cpu: host
#memory: 16384
#sockets: 1
#numa: 0
#onboot: 1
#ostype: l26
#machine: q35
#bios: ovmf
#hostpci0: 0000:00:17.0,pcie=1
# vIOMMU auf 39 Bit begrenzen
#args: -device virtio-iommu-pci,aw-bits=39
#ide2: local:iso/TrueNAS-SCALE-24.04.0.iso,media=cdrom,size=1588230K
#scsi0: ZFS-Pool:vm-101-disk-0,discard=on,iothread=1,size=32G,ssd=1
#scsihw: virtio-scsi-single
#net0: virtio=BC:24:11:82:EF:63,bridge=vmbr2,mtu=9000,tag=10
#net1: virtio=BC:24:11:DD:5A:79,bridge=vmbr2,mtu=9000,tag=20
#name: VMTrueNas
#startup: order=4
#vmgenid: 305ca741-389f-4f46-b327-a72be7a0a332
#smbios1: uuid=e47cc6b8-62e5-4674-b7f5-3d7d8f41994d


WARN: no efidisk configured! Using temporary efivars disk.
kvm: vfio: Cannot reset device 0000:00:17.0, no available reset mechanism.
kvm: vfio: Cannot reset device 0000:00:17.0, no available reset mechanism.
TASK WARNINGS: 1
 
There is no impact unless you use the following settings.

machine: pc-q35-9.2+pve1,viommu=intel
 
guest-phys-bits is available, but aw-bits is not.

Since these refer to the same bit, wouldn't it be impossible to use either if neither can be set?
 
uzumo said:
There is no impact unless you use the following settings.

machine: pc-q35-9.2+pve1,viommu=intel
Click to expand...
Just to update this thread too [0], the Intel vIOMMU's default address bit width value was increased from 39 bits to 48 bits in QEMU 9.2, so the error

Code: 
Failed to set vIOMMU: aw-bits 48 > host aw-bits 39

can be mitigated by pinning the machine version to 9.1 or lower, which still has the old default value of 39 bits for aw-bits. I'll send a patch, which allows setting the aw-bits property for the vIOMMU.

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6608#c5
 
dakralex said:
Just to update this thread too [0], the Intel vIOMMU's default address bit width value was increased from 39 bits to 48 bits in QEMU 9.2, so the error

Code: 
Failed to set vIOMMU: aw-bits 48 > host aw-bits 39

can be mitigated by pinning the machine version to 9.1 or lower, which still has the old default value of 39 bits for aw-bits. I'll send a patch, which allows setting the aw-bits property for the vIOMMU.

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=6608#c5
Click to expand...
Thank you. Do I need to apply this patch myself to use it? Will it be patched in a future release?
 
uzumo said:
Thank you. Do I need to apply this patch myself to use it? Will it be patched in a future release?
Click to expand...
I just sent the patch to the mailing list for review, if we go with exposing the aw-bits option then this patch will be available in a future release.

For now, setting the machine version to 9.1 or lower or setting qm set <vmid> -args '-global intel-iommu.aw-bits=39' should mitigate the change until the change made it into a future release.
 
  • Like
Reactions: cryptecks
dakralex said:
I just sent the patch to the mailing list for review, if we go with exposing the aw-bits option then this patch will be available in a future release.

For now, setting the machine version to 9.1 or lower or setting qm set <vmid> -args '-global intel-iommu.aw-bits=39' should mitigate the change until the change made it into a future release.
Click to expand...
Thank you. I'll give it a try.

Also, I would appreciate it if you could confirm whether this method for setting the guest physical bits is correct.

qm set vmid -args '-cpu host,hv_passthrough,level=35,+vmx,guest-phys-bits=46 -global intel-iommu.aw-bits=39'
 
This workaround has worked well for several virtual machines.

Can this patch be any value from 39-48?

I saw 41. Other possibilities are also possible.

Code: 
kvm: -device vfio-pci,host=0000:00:02.7,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:00:02.7: Failed to set vIOMMU: aw-bits 48 > host aw-bits 41
TASK ERROR: start failed: QEMU exited with code 1
 
You must log in or register to reply here.
Top Bottom