why am I getting the immou errors ?

R

rifaterdemsahin

Member
Apr 27, 2022
116
5
18
Cambridge UK
www.devops.engineering
Code: 
dmesg -w
Code: 
[ 4270.661320] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661322] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661323] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661325] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661327] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661328] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661330] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661332] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661333] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661335] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661337] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661338] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661340] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661342] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661344] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661345] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661347] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661349] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661350] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661352] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661354] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661355] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661357] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661359] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661361] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661362] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661364] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661366] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661367] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.661369] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0xffff@0xffc
[ 4270.670998] vfio-pci 0000:03:00.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 4271.715854] vfio-pci 0000:03:00.1: can't change power state from D0 to D3hot (config space inaccessible)
[ 4271.715861] vfio-pci 0000:03:00.0: invalid power transition (from D3cold to D3hot)
[ 4271.716337] fwbr111i0: port 2(tap111i0) entered disabled state
[ 4271.716506] fwbr111i0: port 2(tap111i0) entered disabled state
[ 4271.917092] AMD-Vi: Completion-Wait loop timed out
[ 4272.044700] AMD-Vi: Completion-Wait loop timed out
[ 4272.791409] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x1002335d0]
[ 4273.793277] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x100233610]
[ 4273.793285] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x100233630]
[ 5156.061770] vfio-pci 0000:03:00.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 5156.813463] vfio-pci 0000:03:00.0: timed out waiting for pending transaction; performing function level reset anyway
[ 5158.061449] vfio-pci 0000:03:00.0: not ready 1023ms after FLR; waiting
[ 5159.117416] vfio-pci 0000:03:00.0: not ready 2047ms after FLR; waiting
[ 5161.261372] vfio-pci 0000:03:00.0: not ready 4095ms after FLR; waiting
[ 5165.613314] vfio-pci 0000:03:00.0: not ready 8191ms after FLR; waiting
[ 5174.061130] vfio-pci 0000:03:00.0: not ready 16383ms after FLR; waiting
[ 5192.236751] vfio-pci 0000:03:00.0: not ready 32767ms after FLR; waiting
[ 5227.052069] vfio-pci 0000:03:00.0: not ready 65535ms after FLR; giving up
[ 5227.664685] device tap111i0 entered promiscuous mode
[ 5227.671553] fwbr111i0: port 1(fwln111i0) entered disabled state
[ 5227.671759] vmbr0: port 4(fwpr111p0) entered disabled state
[ 5227.671984] device fwln111i0 left promiscuous mode
[ 5227.671986] fwbr111i0: port 1(fwln111i0) entered disabled state
[ 5227.692336] device fwpr111p0 left promiscuous mode
[ 5227.692340] vmbr0: port 4(fwpr111p0) entered disabled state
[ 5227.871875] fwbr111i0: port 1(fwln111i0) entered blocking state
[ 5227.871880] fwbr111i0: port 1(fwln111i0) entered disabled state
[ 5227.871961] device fwln111i0 entered promiscuous mode
[ 5227.872024] fwbr111i0: port 1(fwln111i0) entered blocking state
[ 5227.872026] fwbr111i0: port 1(fwln111i0) entered forwarding state
[ 5227.875668] vmbr0: port 4(fwpr111p0) entered blocking state
[ 5227.875674] vmbr0: port 4(fwpr111p0) entered disabled state
[ 5227.875758] device fwpr111p0 entered promiscuous mode
[ 5227.875788] vmbr0: port 4(fwpr111p0) entered blocking state
[ 5227.875790] vmbr0: port 4(fwpr111p0) entered forwarding state
[ 5227.879019] fwbr111i0: port 2(tap111i0) entered blocking state
[ 5227.879023] fwbr111i0: port 2(tap111i0) entered disabled state
[ 5227.879111] fwbr111i0: port 2(tap111i0) entered blocking state
[ 5227.879112] fwbr111i0: port 2(tap111i0) entered forwarding state
[ 5228.068321] AMD-Vi: Completion-Wait loop timed out
[ 5228.196082] AMD-Vi: Completion-Wait loop timed out
[ 5228.323820] AMD-Vi: Completion-Wait loop timed out
[ 5228.452055] AMD-Vi: Completion-Wait loop timed out
[ 5228.942679] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x100233660]
[ 5229.944534] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x1002336a0]
[ 5229.944539] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x1002336c0]
[ 5229.974620] vfio-pci 0000:03:00.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 5229.974660] vfio-pci 0000:03:00.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 5229.974750] vfio-pci 0000:03:00.0: can't change power state from D3cold to D0 (config space inaccessible)
[ 5230.699967] vfio-pci 0000:03:00.0: timed out waiting for pending transaction; performing function level reset anyway
[ 5230.946416] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x1002336f0]
[ 5231.947943] vfio-pci 0000:03:00.0: not ready 1023ms after FLR; waiting
[ 5231.948291] iommu ivhd3: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=03:00.0 address=0x100233720]
[ 5233.003919] vfio-pci 0000:03:00.0: not ready 2047ms after FLR; waiting
[ 5235.243874] vfio-pci 0000:03:00.0: not ready 4095ms after FLR; waiting
[ 5239.595793] vfio-pci 0000:03:00.0: not ready 8191ms after FLR; waiting
[ 5248.043637] vfio-pci 0000:03:00.0: not ready 16383ms after FLR; waiting
 
leesteken said:
Is 03:00.0 your Radeon VII? Please try vendor-reset to work around AMD GPU reset issues. I used this guide. When using kernel 5.15 you need to set the reset_method to device_specific.
Can you share the VM configuration file (from the /etc/pve/qemu-server/ directory)?
Click to expand...
file for 6900 XT
Code: 
root@workstation:/etc/pve/qemu-server# cat 100.conf
#realvnc based action > login to the box
#use the usb monitor as backup and better resolution when needed
#serial works in here
#no guest agent error!
bios: ovmf
boot: order=virtio0;net0;ide0;ide2
cores: 24
cpu: host
efidisk0: zfsonfirecuda:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:43:00,pcie=1,x-vga=1
hostpci1: 0000:44:00,pcie=1
hostpci2: 0000:65:00,pcie=1
ide2: local:iso/virtio-win-0.1.215.iso,media=cdrom,size=528322K
machine: pc-q35-6.1
memory: 22048
meta: creation-qemu=6.1.0,ctime=1651135806
name: BlackLionWP11Screens
net0: virtio=22:CE:C5:98:69:EA,bridge=vmbr0,firewall=1
net1: rtl8139=CE:A0:66:CF:54:D5,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: win11
parent: acronisbackup
runningcpu: kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep
runningmachine: pc-q35-6.1+pve0
scsihw: virtio-scsi-pci
smbios1: uuid=2e349ac4-75bc-494f-aaea-a40cb31de45d
sockets: 2
startup: order=1
tpmstate0: zfsonfirecuda:vm-100-disk-1,size=4M,version=v2.0
usb0: host=04d8:eed2,usb3=1
usb1: host=17e9:4300,usb3=1
usb2: host=0bda:2550,usb3=1
usb3: host=145f:01ad,usb3=1
usb4: host=1235:8210,usb3=1
virtio0: zfsonfirecuda:vm-100-disk-2,size=128G
vmgenid: a04f4117-2219-4865-bed3-757ecc6bfb05

[acronisbackup]
bios: ovmf
boot: order=virtio0;net0;ide0;ide2
cores: 24
cpu: host
efidisk0: zfsonfirecuda:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:43:00,pcie=1,x-vga=1
hostpci1: 0000:44:00,pcie=1
hostpci2: 0000:65:00,pcie=1
ide2: local:iso/virtio-win-0.1.215.iso,media=cdrom,size=528322K
machine: pc-q35-6.1
memory: 22048
meta: creation-qemu=6.1.0,ctime=1651135806
name: BlackLionWP11Screens
net0: virtio=22:CE:C5:98:69:EA,bridge=vmbr0,firewall=1
net1: rtl8139=CE:A0:66:CF:54:D5,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
parent: audiosolvewithbluetooth
runningcpu: kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep
runningmachine: pc-q35-6.1+pve0
scsihw: virtio-scsi-pci
smbios1: uuid=2e349ac4-75bc-494f-aaea-a40cb31de45d
snaptime: 1651363383
sockets: 2
tpmstate0: zfsonfirecuda:vm-100-disk-1,size=4M,version=v2.0
usb0: host=04d8:eed2,usb3=1
usb1: host=17e9:4300,usb3=1
usb2: host=0bda:2550,usb3=1
usb3: host=145f:01ad,usb3=1
usb4: host=1235:8210,usb3=1
virtio0: zfsonfirecuda:vm-100-disk-2,size=128G
vmgenid: a04f4117-2219-4865-bed3-757ecc6bfb05
vmstate: zfsonfirecuda:vm-100-state-acronisbackup

[audiosolvewithbluetooth]
#beforeprogramsinstall
bios: ovmf
boot: order=virtio0;net0;ide0;ide2
cores: 24
cpu: host
efidisk0: zfsonfirecuda:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:43:00,pcie=1,x-vga=1
ide2: local:iso/virtio-win-0.1.215.iso,media=cdrom,size=528322K
machine: pc-q35-6.1
memory: 22048
meta: creation-qemu=6.1.0,ctime=1651135806
name: erdemwindowspro11
net0: virtio=22:CE:C5:98:69:EA,bridge=vmbr0,firewall=1
net1: rtl8139=CE:A0:66:CF:54:D5,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: win11
parent: workswith5screens
runningcpu: kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep
runningmachine: pc-q35-6.1+pve0
scsihw: virtio-scsi-pci
smbios1: uuid=2e349ac4-75bc-494f-aaea-a40cb31de45d
snaptime: 1651347816
sockets: 2
tpmstate0: zfsonfirecuda:vm-100-disk-1,size=4M,version=v2.0
usb0: host=04d8:eed2,usb3=1
usb1: host=17e9:4300,usb3=1
usb2: host=0bda:2550,usb3=1
usb3: host=145f:01ad,usb3=1
virtio0: zfsonfirecuda:vm-100-disk-2,size=128G
vmgenid: a04f4117-2219-4865-bed3-757ecc6bfb05
vmstate: zfsonfirecuda:vm-100-state-audiosolvewithbluetooth

[realvncfearlicencewindows]
bios: ovmf
boot: order=virtio0;ide2;net0;ide0
cores: 24
efidisk0: zfsonfirecuda:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
ide0: local:iso/virtio-win-0.1.217.iso,media=cdrom,size=519096K
ide2: local:iso/Win11_EnglishInternational_x64v1.iso,media=cdrom
machine: pc-q35-6.1
memory: 22048
meta: creation-qemu=6.1.0,ctime=1651135806
name: erdemwindowspro11
net0: virtio=22:CE:C5:98:69:EA,bridge=vmbr0,firewall=1
net1: rtl8139=CE:A0:66:CF:54:D5,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: win11
runningcpu: kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep
runningmachine: pc-q35-6.1+pve0
scsihw: virtio-scsi-pci
smbios1: uuid=2e349ac4-75bc-494f-aaea-a40cb31de45d
snaptime: 1651304210
sockets: 2
tpmstate0: zfsonfirecuda:vm-100-disk-1,size=4M,version=v2.0
usb1: host=17e9:4300,usb3=1
usb2: host=0bda:2550,usb3=1
usb3: host=04d8:eed2,usb3=1
virtio0: zfsonfirecuda:vm-100-disk-2,size=128G
vmgenid: a04f4117-2219-4865-bed3-757ecc6bfb05
vmstate: zfsonfirecuda:vm-100-state-realvncfearlicencewindows

[workswith5screens]
bios: ovmf
boot: order=virtio0;net0;ide0;ide2
cores: 24
cpu: host
efidisk0: zfsonfirecuda:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:43:00,pcie=1,x-vga=1
ide2: local:iso/virtio-win-0.1.215.iso,media=cdrom,size=528322K
machine: pc-q35-6.1
memory: 22048
meta: creation-qemu=6.1.0,ctime=1651135806
name: erdemwindowspro11
net0: virtio=22:CE:C5:98:69:EA,bridge=vmbr0,firewall=1
net1: rtl8139=CE:A0:66:CF:54:D5,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: win11
parent: realvncfearlicencewindows
runningcpu: kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep
runningmachine: pc-q35-6.1+pve0
scsihw: virtio-scsi-pci
smbios1: uuid=2e349ac4-75bc-494f-aaea-a40cb31de45d
snaptime: 1651347364
sockets: 2
tpmstate0: zfsonfirecuda:vm-100-disk-1,size=4M,version=v2.0
usb0: host=04d8:eed2,usb3=1
usb1: host=17e9:4300,usb3=1
usb2: host=0bda:2550,usb3=1
usb3: host=145f:01ad,usb3=1
virtio0: zfsonfirecuda:vm-100-disk-2,size=128G
vmgenid: a04f4117-2219-4865-bed3-757ecc6bfb05
vmstate: zfsonfirecuda:vm-100-state-workswith5screens
root@workstation:/etc/pve/qemu-server#
 
leesteken said:
Is 03:00.0 your Radeon VII? Please try vendor-reset to work around AMD GPU reset issues. I used this guide. When using kernel 5.15 you need to set the reset_method to device_specific.
Can you share the VM configuration file (from the /etc/pve/qemu-server/ directory)?
Click to expand...

file for radeon 7
Code: 
root@workstation:/etc/pve/qemu-server# cat 111.conf
agent: 1
bios: ovmf
boot: order=sata0;net0;ide2
cores: 4
cpu: host
efidisk0: local-lvm:vm-111-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:03:00,pcie=1,x-vga=1
ide2: local:iso/virtio-win-0.1.217.iso,media=cdrom,size=519096K
machine: pc-q35-6.1
memory: 12048
meta: creation-qemu=6.1.0,ctime=1651350125
name: BlueLionWindows11Games
net0: rtl8139=26:5E:27:F5:96:0C,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: win11
sata0: local-lvm:vm-111-disk-0,cache=writeback,size=64G
scsihw: virtio-scsi-pci
smbios1: uuid=0e40f7ee-7dea-478c-a11b-542ec7491959
sockets: 4
startup: order=4
tpmstate0: local-lvm:vm-111-disk-2,size=4M,version=v2.0
vga: none
vmgenid: e37c155c-d84b-4d8d-af7c-dbd29ea50982
root@workstation:/etc/pve/qemu-server#
 
This happened to me as well (granted - on nVidia, but could still be a contributing case for AMD), the causes were 2:
1. resize bar was enabled in the BIOS
2. the PCIe slot was set to "raid" mode (4x4x4x4) instead of 16x.
 
You must log in or register to reply here.
Top Bottom