Fix pci passthrough documentation

[MinerAle00]

Hi guys, I found a misstake on the pci passthrough documentation.
This page right here: https://pve.proxmox.com/wiki/Pci_passthrough
When you need to blacklist driver u must add echo "blacklist amdgpu".

echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf

Yesterday I tried to passthrough one rx 470 and the problem was amdgpu.
I hope this will help you!

 

[noel.]

thank you, I updated the wiki to include amdgpu

 

[leesteken]

thank you, I updated the wiki to include amdgpu

While you're at it, why not include all the Intel graphics drivers?
EDIT: Maybe combine the sometimes out-of-date information from that page with the newer(?) PCI(e)_Passthrough Wiki page and the more up-to-date Proxmox manual?

 

[noel.]

Good idea, will do!

 

[Neobin]

While you're at it, why not include all the Intel graphics drivers?
EDIT: Maybe combine the sometimes out-of-date information from that page with the newer(?) PCI(e)_Passthrough Wiki page and the more up-to-date Proxmox manual?

A consolidation of the old: [1] and new: [2] wiki entries and marking the old one somehow "deprecated" and prominently link to the new one, as well as bringing the new one up-to-date and expanding it, would be really desirable!

This topic (PCIe-passthrough) comes up on a really frequent basis here in the forum and some users seem to only refer to the older wiki entry, that still mentions e.g.: amd_iommu=on, which is (in the meantime?) not necessary, nor even a valid/existing parameter (on) at all (from the, in Proxmox, currently default 5.15 kernel docs: [3]):

amd_iommu=      [HW,X86-64]
                Pass parameters to the AMD IOMMU driver in the system.
                Possible values are:
                fullflush - Deprecated, equivalent to iommu.strict=1
                off       - do not initialize any AMD IOMMU found in
                            the system
                force_isolation - Force device isolation for all
                                  devices. The IOMMU driver is not
                                  allowed anymore to lift isolation
                                  requirements as needed. This option
                                  does not override iommu=pt
                force_enable - Force enable the IOMMU on platforms known
                               to be buggy with IOMMU enabled. Use this
                               option with care.

Things worth mentioning in the old: [1] wiki entry:

• amd_iommu=on not valid/existing (anymore?).
• dmesg | grep -e DMAR -e IOMMU and: dmesg | grep 'remapping' are split up, instead of simply: dmesg | grep -e DMAR -e IOMMU -e AMD-Vi from the new one.
• update-initramfs -u -k all missing at all after adding the required modules to: /etc/modules.

Things worth mentioning in the new: [2] wiki entry:

• The GPU-driver-names for blacklisting are missing completely.
  • Additionally, it should be mentioned in the wiki, that one might not need and/or want at all to blacklist all GPU-drivers, especially if it is intended to also add the Intel-driver-names too! This might lead to an unintended situation, if one would "blindly" copy-paste all entries for blacklisting from the wiki.
  • On the other hand, it might be considered to not list the driver-names at all and instead guide the user how to look it up with: lspci -k or the already later mentioned: lspci -nnk on their own; to prevent blindly copy-pasting at all.
• Mention of: pcie_acs_override= completely missing.

Things worth mentioning regarding both entries:

• Again mention and describe the: pcie_acs_override=-option more detailed and what its possible parameters are, as well as its cons, as in: [4].
• Mention and explain the: softdep-option: [5] at all. (Especially for the commonly used/needed situation to have: vfio-pci loaded before a specific driver/module.)
• Replace: find /sys/kernel/iommu_groups/ -type l with (or at least add extra):

  for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

  since it is way more meaningful, not only for the user, but also for people who want to help them!

Maybe @leesteken can add and/or change things to/on this list.

I want to emphasize again, despite the fact that I also listed things regarding the old wiki entry, in my humble opinion, it would really make sense to have only one up-to-date wiki entry for this topic!

[1] https://pve.proxmox.com/wiki/Pci_passthrough
[2] https://pve.proxmox.com/wiki/PCI(e)_Passthrough
[3] https://www.kernel.org/doc/html/v5.15/admin-guide/kernel-parameters.html
[4] https://wiki.archlinux.org/title/PC...passing_the_IOMMU_groups_(ACS_override_patch)
[5] https://manpages.debian.org/bullseye/kmod/modprobe.d.5.en.html

 

[noel.]

A consolidation of the old: [1] and new: [2] wiki entries and marking the old one somehow "deprecated" and prominently link to the new one, as well as bringing the new one up-to-date and expanding it, would be really desirable!

I was thinking of just redirecting the old PCI article to the newer PCI(e) article, but I'll have to discuss that with the team and also always appreciate the forum users thoughts. I totally agree that having two articles is counterproductive.

Thank you also very much for all the specific things you would like to see changed/updated

Additionally, it should be mentioned in the wiki, that one might not need and/or want at all to blacklist all GPU-drivers,

I did quickly update that while adding the amdgpu drivers, however a more in-depth explanation is definitely in order.

On the other hand, it might be considered to not list the driver-names at all and instead guide the user how to look it up with: lspci -k or the already later mentioned: lspci -nnk on their own; to prevent blindly copy-pasting at all.

This is a very good idea, I like this.

I will have to read up on kernel params and pci(e) stuff, think a little about the layout and talk to the rest of the team, so don't expect the updated wiki to appear today or tomorrow.

 

[tgm]

From my point of view, the situation surrounding GPU passthrough is out of control. Some have suggested that it is easy. I’ve had the opposite experience with two different installations and I have yet to get either to work.

There are several distinct configurations that need to be addressed (and documented) in detail:

• Are you using a motherboard/CPU based GPU?
• Do you have one or more separate GPU cards?
• Is the system AMD or Intel based and what CPU series is used?
• Is the GPU AMD, NVidea, or Intel based (possibly others)?
• What type of VM are you trying do use (Linux, Windows, etc)?
• Then there is the situation with ProxMox versions (pre 7.2) and kernel versions.

At the moment, most everything seems to be guesswork. What documentation is out there is often conflicting. The level of documentation detail is terribly inadequate. The how and why of the various options are just not explained. This is also a problem with many of the options associated with just creating any VM or CT (with passthrough or not).

 

[noel.]

Some have suggested that it is easy.

I have never heard anyone say that. I mainly hear people complain that it is finicky. Not only in Proxmox but in general.

There are several distinct configurations that need to be addressed (and documented) in detail:

Thank you for the list, I will try to include and answer your points in the updated version.

This is also a problem with many of the options associated with just creating any VM or CT (with passthrough or not).

Have you had a look at the documentation [1]? I don't think you can be much more precise and detailed in a technical documentation.

---

https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_virtual_machines_settings

 

[leesteken]

Thank you for the list, I will try to include and answer your points in the updated version.

The otherwise fine manual and Wiki still state "For Intel CPUs, you may also need to enable the IOMMU on the kernel command line for older (pre-5.15)kernels by adding", but this has not been true since the on-by-default was reverted in the 5.15 kernels. I think "For Intel CPUs, you also need to enable the IOMMU on the kernel command line by adding" would be better.

 

[noel.]

The otherwise fine manual and Wiki still state "For Intel CPUs, you may also need to enable the IOMMU on the kernel command line for older (pre-5.15)kernels by adding", but this has not been true since the on-by-default was reverted in the 5.15 kernels. I think "For Intel CPUs, you also need to enable the IOMMU on the kernel command line by adding" would be better.

good catch, thank you