Can't install Ubtunu 23.04 using machine type Q35 and intel arc a380

[HomebrewD]

Hi all

After having some issues with my ububtu vm freezing, I'm trying to reinstall the vm using machine type Q35.

For some reason the installer gets stuck with this as the last message:

i915 0000:01:00.0: [drm] Local memeory available: 0x000000017c800000

I've tried installing ubuntu 22.04 and then upgrading the kernel to 6.2 but it has the same issue after reboot.

I've already tried looking on the internet but I can't find any similar issue.
It works with i440fx.

I'm hoping someone here can point me in the right direction.

Here is the output of pveversion -v:

root@hive:~# pveversion -v
proxmox-ve: 7.4-1 (running kernel: 6.2.11-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-6.2: 7.4-2
pve-kernel-5.15: 7.4-2
pve-kernel-5.11: 7.0-10
pve-kernel-6.2.11-1-pve: 6.2.11-1
pve-kernel-5.15.107-1-pve: 5.15.107-1
pve-kernel-5.15.102-1-pve: 5.15.102-1
pve-kernel-5.15.85-1-pve: 5.15.85-1
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.11.17-1-pve: 5.11.17-1
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve2
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-2
pve-firewall: 4.3-1
pve-firmware: 3.6-5
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.11-pve1

And the settings of the vm:

root@hive:~# qm config 102
agent: 1,fstrim_cloned_disks=1
balloon: 16384
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 16
cpu: host
efidisk0: SwiftData:vm-102-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:03:00,pcie=1
ide2: local:iso/ubuntu-23.04-live-server-amd64.iso,media=cdrom,size=2579360K
machine: q35
memory: 32768
name: DockerHubHive
net0: virtio=1E:AD:4C:F9:E3:E4,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: FastData:vm-102-disk-0,aio=threads,discard=on,iothread=1,size=200G,ssd=1
scsi1: FastCache:vm-102-disk-0,aio=threads,backup=0,cache=writethrough,discard=on,iothread=1,size=100G,ssd=1
scsihw: virtio-scsi-single
shares: 5000
smbios1: uuid=3560f2d6-cedc-42fb-917a-0cfdd2d665f4
sockets: 1
vmgenid: 8bf8b083-1882-4644-98b2-935fbf60c066

Thanks in advance!

 

[leesteken]

I know Intel Arc needs Resizable BAR for good performance, but since the problem appears with Q35 which adds PCI-express: please make sure to disable Resizable BAR in the motherboard BIOS as it is not (yet) supported by QEMU/Proxmox.
I would expect no significant (performance) differences between i440fx and q35 machines types for a LInux VM. Maybe you can just use i440fx as a work-around? Or try not enabling PCI-Express in your VM configuration?

 

[HomebrewD]

Interesting. I did have ReBAR enabled in the bios. Didn't know it wasn't supported yet.
In the meantime I went with i440fx to get my services up running again.
I did try with PCI-Express disabled but didn't work.

I guess a vm that freezes every few hours is better than no vm.

Maybe I can try disabling it again in the bios to stop the vm from freezing until proxmox supports it.

 

[leesteken]

Interesting. I did have ReBAR enabled in the bios. Didn't know it wasn't supported yet.
In the meantime I went with i440fx to get my services up running again.
I did try with PCI-Express disabled but didn't work.

I guess a vm that freezes every few hours is better than no vm.

Sorry, I missed that it doesn't fully work with i440fx either. Does it work stable without PCI(e) passthrough? If not, they maybe you have a weak memory chip somewhere.

Maybe I can try disabling it again in the bios to stop the vm from freezing until proxmox supports it.

That would be my suggestion also. Maybe disable Above 4G Decoding as well.

Note that ballooning cannot work with PCI(e) passthrough. Make sure to lease some memory and cores for Proxmox itself.

 

[HomebrewD]

Sorry, I missed that it doesn't fully work with i440fx either. Does it work stable without PCI(e) passthrough? If not, they maybe you have a weak memory chip somewhere.

That would be my suggestion also. Maybe disable Above 4G Decoding as well.

Note that ballooning cannot work with PCI(e) passthrough. Make sure to lease some memory and cores for Proxmox itself.

Disabled the Above 4G Decoding as well. Before the vm was running pretty stable. It only went downhill after trying to get the gpu to work.

I disabled the ballooning since I just upgraded to 128GB ram. (Crashed before as well so probably not ram related)

Proxmox itself should have enough memeory and cores as the other vm's run without issue.

Just finished setting up the vm again so now the waiting game begins. Fingers crossed.

 

[HomebrewD]

Ok so small update.
I could not get the gpu to work at all with ReBAR and Above 4G Decoding disabled (Guc/Huc firmware would fail to load).

However I did do some changes and the vm has been running stable for more than a day now.

I enabled the following settings in the bios:
- Resizable BAR
- Above 4G Decoding
- ASPM (Jellyfin guide)

Than on the Ubuntu vm:
- Use machine type i1440fx
- Fresh install 23.04
- Manually updated to latest kernel (I think 6.3.2?)
- Manually updated the i915 firmware from the git repository so the Guc/Huc firmware loads

Finally proxmox itself:
- Update to the opt-in 6.2 kernel (I saw somewhere ReBAR support was added in 6.1?)

Now transcoding using QSV works in Jellyfin.

I'll try and keep this thread updated in case I run into issues again. Otherwise I'll post my findings in a week or so in case other people stumble on this post.