Patch Proposal PVE Kernel Resizable Bar Support for Intel Arc GPUs without Uefi hacks

[propstat]

There are a couple of discussions ongoing inside the Linux community to enable resizable bar from the software side to enable machines that are not receiving UEFI updates in that direction. Running PoCs are also available, such as the project Intel-ARC-Rebar (unfortunately source available and not open source).

An approach is to release BAR0 of the PCIe switch built into the card. The kernel can then grow the root port’s prefetchable window, and the GPU’s BAR2 can be resized from 256 MiB to full VRAM, I want to be honest, this approach is heavily Claude informed and outside of my usual comfort zone (I imagine some of you might scold me for that), but born out of the economic neccessity as we all suffer the current situation.

Two fixes for resizing PCI BARs, mainly to give GPUs that are passed through to VMs access to their full VRAM on hosts whose firmware does not enable Resizable BAR.

On hosts whose firmware does not enable Resizable BAR, Intel Arc GPUs stayat a 256 MiB BAR2 instead of exposing their full VRAM. This matters mostfor GPUs passed through to VMs. This change adds two kernel patches tothe Proxmox kernel (7.0) that let the kernel enlarge the BAR itself,without a Resizable BAR option in the BIOS.

What Problem is addressed:

An Intel Arc card contains its own PCIe switch. The upstream port of thatswitch has an 8 MiB BAR0, which firmware places inside the prefetchablewindow of the root port above the card. When the GPU's BAR2 is resized,the kernel releases the bridge windows up the chain to make room, butnot that BAR0. The root port window stays pinned, the resize fails with-ENOSPC, and the GPU keeps its 256 MiB BAR.A second, unrelated bug affects devices that sit directly on a root bus,with no bridge above them: the resize releases their BARs but neverassigns them again, while reporting success. The device is left withoutusable BARs. This hits some ARM boards and GPUs that a VM guest sees onits root bus.

## Changes

1. PCI: Fix BAR resize for devices on a root bus:
   A cherry-pick of mainline commit d58384c22 (merged for 7.3, marked for stable). It reassigns the released BARs, rolls back cleanly if they don't fit, and fixes an unbalanced lock on that path. Ubuntu tracks the bug for its 7.0 kernel as LP: #2159596.
2. PCI: Release BAR0 of an integrated bridge to allow GPU BAR resize:
   A quirk by Ilpo Järvinen (Intel) for the Arc switch device IDs 0x4fa0, 0x4fa1 and 0xe2ff. It releases that BAR0 when the switch is enabled, so the root port window can grow. Backported to 7.0 with one adaptation: the BAR is only marked unused if releasing it succeeded.

After this, xe and i915 resize BAR2 when they probe. For passthrough,BAR2 can be resized through the GPU's `resource2_resize` sysfs attributewhile no driver is bound, before handing the card to vfio-pci.

Requirements

• No Resizable BAR option in the BIOS is needed.
• "Above 4G decoding" (64-bit MMIO) must be enabled, so the firmware gives the host bridge a 64-bit window above 4 GiB.
• That window must hold all enlarged BARs with their natural alignment, e.g. about 64 GiB for an Arc Pro B60 Dual (2 × 32 GiB).

Requirement Verification

    readlink -f /sys/bus/pci/devices/<gpu>             # host bridge, e.g. pci0000:80
    grep '^[0-9a-f].* : PCI Bus 0000:80' /proc/iomem   # its windows above 0x100000000
    lspci -vv -s <gpu> | grep 'BAR 2'                  # size the GPU supports

If a requirement is not met, nothing "SHOULD" break:
the resize fails as beforeand the GPU keeps its 256 MiB BAR.## Multiple GPUsEach GPU is resized on its own. Hosts with several Arc GPUs work,including bifurcated dual-GPU cards such as the Arc Pro B60 Dual, as longas every GPU has its own root port and the window has room for all ofthem. GPUs behind a shared external PCIe switch are not covered: theother GPU's BAR keeps the shared window pinned.

Upstream status and Risk
Patch 1 is merged upstream. Patch 2 is posted upstream but not merged.Review questioned whether the released BAR0 keeps decoding its oldaddress in hardware; that question is still open. It is offered as aninterim fix until upstream either merges it or lands the planned reworkof resource fitting, which would make it unnecessary.## TestingBoth patches apply on top of 7.0.14, and the changed files compile.

References

• Arc quirk
  https://lore.kernel.org/all/20250918-xe-pci-rebar-2-v1-1-6c094702a074@intel.com/
• Root-bus fix (v6)
  https://patchew.org/linux/20260918035633.566823-1-lizf@honeycomb.io/
• Ubuntu bug
  https://bugs.launchpad.net/ubuntu/+source/linux-hwe-7.0/+bug/2159596

Proposed Changes

drivers/pci/quirks.c (Arc quirk):

@@ -6380,3 +6380,28 @@ static void pci_mask_replay_timer_timeout(struct pci_dev *pdev)
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_GLI, 0x9750, pci_mask_replay_timer_timeout);
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_GLI, 0x9755, pci_mask_replay_timer_timeout);
 #endif
+
+/*
+ * PCI switches integrated into Intel Arc GPUs have BAR0 that prevents
+ * resizing the BARs of the GPU device due to that bridge BAR0 pinning the
+ * bridge window it's under in place. Nothing in pcieport requires that
+ * BAR0.
+ *
+ * Release and disable BAR0 permanently by clearing its flags to prevent
+ * anything from assigning it again.
+ */
+static void pci_release_bar0(struct pci_dev *pdev)
+{
+    struct resource *res = pci_resource_n(pdev, 0);
+
+    if (!res->parent)
+        return;
+
+    if (pci_release_resource(pdev, 0))
+        return;
+
+    res->flags = 0;
+}
+DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_INTEL, 0x4fa0, pci_release_bar0);
+DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_INTEL, 0x4fa1, pci_release_bar0);
+DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_INTEL, 0xe2ff, pci_release_bar0);

drivers/pci/setup-bus.c (root-bus fix):

@@ -2374,6 +2374,7 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
     struct resource *res = pci_resource_n(pdev, resno);
     struct pci_dev_resource *dev_res;
     struct pci_bus *bus = pdev->bus;
+    struct pci_dev *bridge = pci_upstream_bridge(pdev);
     struct resource *b_win, *r;
     LIST_HEAD(saved);
     unsigned int i;
@@ -2391,6 +2392,8 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
     if (ret)
         return ret;
 
+    down_read(&pci_bus_sem);
+
     pci_dev_for_each_resource(pdev, r, i) {
         if (i >= PCI_BRIDGE_RESOURCES)
             break;
@@ -2409,13 +2412,21 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
 
     pci_resize_resource_set_size(pdev, resno, size);
 
-    if (!bus->self)
-        goto out;
+    if (bridge) {
+        ret = pbus_reassign_bridge_resources(bus, res, &saved);
+        if (ret)
+            goto restore;
+    } else {
+        /* No bridge window to adjust; let the core reassign the bus. */
+        pci_bus_assign_resources(bus);
 
-    down_read(&pci_bus_sem);
-    ret = pbus_reassign_bridge_resources(bus, res, &saved);
-    if (ret)
-        goto restore;
+        list_for_each_entry(dev_res, &saved, list) {
+            if (!resource_assigned(dev_res->res)) {
+                ret = -ENOSPC;
+                goto restore;
+            }
+        }
+    }
 
 out:
     up_read(&pci_bus_sem);

 

[propstat]

@dcsapak This circles back to https://forum.proxmox.com/threads/intel-vgpu-and-proxmox.178024/#post-824996