How to do MxGPU on Proxmox 8.x for S7150x2?

Q

Q3tNHn

New Member
Dec 30, 2024
13
1
3
Hi all,
I have a S7150X2 GPU, How should I create virtual GPUs on Proxmox 8.x? The GIM module is not available for kernel 6.1.x version.
Please help me!
 
First, install git,gcc,make,linux-headers-$(uname -r) through apt for compilation, then refer to the documentation on the web to open IOMMU, and then run "ln -sf /usr/src/linux-headers-$(uname -r) /lib/modules/$(uname -r)/build", then get the source code from the https://github.com/kasperlewau/MxGPU-Virtualization and run the gim.sh, after success, set the blacklist and configure the parameters according to the documents in the repository. After the configuration is completed, load this module through "modprobe gim", now you should be able to see the virtual device through "lspci", add the virtual device to your virtual machine through the webGUI of pve and download the client driver from the AMD official website
（im chinese and my english not very good，so this is by microsoft translate，The original text is as follows）
首先，通过apt安装git,gcc,make,linux-headers-$(uname -r)用于编译,然后参照网上的文档开启IOMMU，接着执行“ln -sf /usr/src/linux-headers-$(uname -r) /lib/modules/$(uname -r)/build”然后从https://github.com/kasperlewau/MxGP...im.sh，成功之后按照仓库中的文档设置黑名单并配置参数。配置完成后通过“modprobe gim”加载这个模块，现在你应该可以通过“lspci”看到虚拟出的设备了，通过pve的WebGUI中给你的虚拟机添加虚拟出的设备并从AMD官网下载客户机驱动即可
 
  • Like
Reactions: david_tao
Dear @kevinlels , I test you tutorial. Here is the error massages I get.

root@diz-ummd-s01:/home/TH_2025/V2/MxGPU-Virtualization# ./gim.sh
Current User is root
check: /usr/src/linux-headers-6.8.12-8-pve

make -C /lib/modules/6.8.12-8-pve/build M=/home/TH_2025/V2/MxGPU-Virtualization/drv modules
make[1]: Entering directory '/usr/src/linux-headers-6.8.12-8-pve'
CC [M] /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.o
In file included from /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:50:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
73 | #define PCI_SRIOV_CTRL_VFE 0x01/* VF Enable */
|
In file included from ./include/uapi/linux/pci.h:21,
from ./include/linux/pci.h:42,
from /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:25:
./include/uapi/linux/pci_regs.h:949: note: this is the location of the previous definition
949 | #define PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */
|
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
74 | #define PCI_SRIOV_CTRL_MSE 0x08/* VF Memory Space Enable */
|
./include/uapi/linux/pci_regs.h:952: note: this is the location of the previous definition
952 | #define PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */
|
In file included from /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:52:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_reset.h:69: warning: "PCI_ERR_UNCOR_STATUS" redefined
69 | #define PCI_ERR_UNCOR_STATUS 4
|
./include/uapi/linux/pci_regs.h:751: note: this is the location of the previous definition
751 | #define PCI_ERR_UNCOR_STATUS 0x04 /* Uncorrectable Error Status */
|
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_reset.h:71: warning: "PCI_ERR_UNCOR_MASK" redefined
71 | #define PCI_ERR_UNCOR_MASK 8/* Uncorrectable Error Mask */
|
./include/uapi/linux/pci_regs.h:769: note: this is the location of the previous definition
769 | #define PCI_ERR_UNCOR_MASK 0x08 /* Uncorrectable Error Mask */
|
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:169:6: warning: no previous prototype for ‘gim_read_rom_from_reg’ [-Wmissing-prototypes]
169 | void gim_read_rom_from_reg(struct adapter *adapt,
| ^~~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:204:6: warning: no previous prototype for ‘gim_vbios_posted’ [-Wmissing-prototypes]
204 | bool gim_vbios_posted(struct adapter *adapt)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:222:5: warning: no previous prototype for ‘gim_read_vbios’ [-Wmissing-prototypes]
222 | int gim_read_vbios(struct adapter *adapt)
| ^~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:332:7: warning: no previous prototype for ‘map_doorbell’ [-Wmissing-prototypes]
332 | void *map_doorbell(struct pci_dev *pdev)
| ^~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:360:7: warning: no previous prototype for ‘map_fb’ [-Wmissing-prototypes]
360 | void *map_fb(struct pci_dev *pdev)
| ^~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:509:5: warning: no previous prototype for ‘mask_unrecoverable_errors’ [-Wmissing-prototypes]
509 | int mask_unrecoverable_errors(struct pci_dev *dev)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:526:6: warning: no previous prototype for ‘clear_unrecoverable_errors’ [-Wmissing-prototypes]
526 | void clear_unrecoverable_errors(struct pci_dev *dev)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:546:6: warning: no previous prototype for ‘sched_work_handler’ [-Wmissing-prototypes]
546 | void sched_work_handler(struct work_struct *work)
| ^~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c: In function ‘set_new_adapter’:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:697:17: error: implicit declaration of function ‘pci_set_dma_mask’ [-Werror=implicit-function-declaration]
697 | pci_set_dma_mask(curr->pf.pci_dev, 0xffffffffffull);
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c: At top level:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1016:6: warning: no previous prototype for ‘unmap_mmr_base’ [-Wmissing-prototypes]
1016 | void unmap_mmr_base(struct adapter *p_adapt)
| ^~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1027:6: warning: no previous prototype for ‘pause_all_schedulers’ [-Wmissing-prototypes]
1027 | void pause_all_schedulers(void)
| ^~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1040:6: warning: no previous prototype for ‘idle_one_adapter’ [-Wmissing-prototypes]
1040 | void idle_one_adapter(struct adapter *adapt)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1380:6: warning: no previous prototype for ‘check_me_cntl’ [-Wmissing-prototypes]
1380 | void check_me_cntl(struct adapter *adapt, char *comment)
| ^~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1399:6: warning: no previous prototype for ‘check_base_addrs’ [-Wmissing-prototypes]
1399 | void check_base_addrs(struct function *func, char *comment)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:2027:5: warning: no previous prototype for ‘world_switch_divided’ [-Wmissing-prototypes]
2027 | int world_switch_divided(struct function *func, struct function *next_func)
| ^~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3279:6: warning: no previous prototype for ‘check_smu_int’ [-Wmissing-prototypes]
3279 | void check_smu_int(uint32_t bdf, char *comment)
| ^~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3344:6: warning: no previous prototype for ‘dump_ucode’ [-Wmissing-prototypes]
3344 | void dump_ucode(struct adapter *adapt, int addr_register,
| ^~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3364:6: warning: no previous prototype for ‘vbios_check’ [-Wmissing-prototypes]
3364 | void vbios_check(struct adapter *adapt, uint32_t bdf, char *comment)
| ^~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3444:6: warning: no previous prototype for ‘read_ih_regs’ [-Wmissing-prototypes]
3444 | void read_ih_regs(uint32_t bdf, char *comment)
| ^~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3500:6: warning: no previous prototype for ‘save_mmio_regs’ [-Wmissing-prototypes]
3500 | void save_mmio_regs(uint32_t bdf, uint32_t *buf, struct named_reg_list *list)
| ^~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3511:6: warning: no previous prototype for ‘restore_mmio_regs’ [-Wmissing-prototypes]
3511 | void restore_mmio_regs(uint32_t bdf, uint32_t *buf, struct named_reg_list *list)
| ^~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3523:6: warning: no previous prototype for ‘compare_mmio_regs’ [-Wmissing-prototypes]
3523 | void compare_mmio_regs(uint32_t *buf1, uint32_t *buf2,
| ^~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3536:6: warning: no previous prototype for ‘check_pf_regs’ [-Wmissing-prototypes]
3536 | void check_pf_regs(uint32_t bdf, char *comment)
| ^~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3876:5: warning: no previous prototype for ‘check_cp_status’ [-Wmissing-prototypes]
3876 | int check_cp_status(struct adapter *adapt, char *comment, struct function *func)
| ^~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3896:7: warning: no previous prototype for ‘decode_ram_usage’ [-Wmissing-prototypes]
3896 | char *decode_ram_usage(uint32_t index)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3940:6: warning: no previous prototype for ‘dump_scratch_ram’ [-Wmissing-prototypes]
3940 | void dump_scratch_ram(struct adapter *adapt,
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:4003:5: warning: no previous prototype for ‘amd_try_spinlock’ [-Wmissing-prototypes]
4003 | int amd_try_spinlock(spinlock_t *lock, uint32_t usec_timeout)
| ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make[3]: *** [scripts/Makefile.build:243: /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.o] Error 1
make[2]: *** [/usr/src/linux-headers-6.8.12-8-pve/Makefile:1925: /home/TH_2025/V2/MxGPU-Virtualization/drv] Error 2
make[1]: *** [Makefile:240: __sub-make] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-6.8.12-8-pve'
make: *** [Makefile:39: all] Error 2

make -C /lib/modules/6.8.12-8-pve/build M=/home/TH_2025/V2/MxGPU-Virtualization/drv modules_install
make[1]: Entering directory '/usr/src/linux-headers-6.8.12-8-pve'
DEPMOD /lib/modules/6.8.12-8-pve
Warning: modules_install: missing 'System.map' file. Skipping depmod.
make[1]: Leaving directory '/usr/src/linux-headers-6.8.12-8-pve'
 
T.Herrmann said:
Dear @kevinlels , I test you tutorial. Here is the error massages I get.

root@diz-ummd-s01:/home/TH_2025/V2/MxGPU-Virtualization# ./gim.sh
Current User is root
check: /usr/src/linux-headers-6.8.12-8-pve

make -C /lib/modules/6.8.12-8-pve/build M=/home/TH_2025/V2/MxGPU-Virtualization/drv modules
make[1]: Entering directory '/usr/src/linux-headers-6.8.12-8-pve'
CC [M] /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.o
In file included from /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:50:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
73 | #define PCI_SRIOV_CTRL_VFE 0x01/* VF Enable */
|
In file included from ./include/uapi/linux/pci.h:21,
from ./include/linux/pci.h:42,
from /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:25:
./include/uapi/linux/pci_regs.h:949: note: this is the location of the previous definition
949 | #define PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */
|
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
74 | #define PCI_SRIOV_CTRL_MSE 0x08/* VF Memory Space Enable */
|
./include/uapi/linux/pci_regs.h:952: note: this is the location of the previous definition
952 | #define PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */
|
In file included from /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:52:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_reset.h:69: warning: "PCI_ERR_UNCOR_STATUS" redefined
69 | #define PCI_ERR_UNCOR_STATUS 4
|
./include/uapi/linux/pci_regs.h:751: note: this is the location of the previous definition
751 | #define PCI_ERR_UNCOR_STATUS 0x04 /* Uncorrectable Error Status */
|
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_reset.h:71: warning: "PCI_ERR_UNCOR_MASK" redefined
71 | #define PCI_ERR_UNCOR_MASK 8/* Uncorrectable Error Mask */
|
./include/uapi/linux/pci_regs.h:769: note: this is the location of the previous definition
769 | #define PCI_ERR_UNCOR_MASK 0x08 /* Uncorrectable Error Mask */
|
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:169:6: warning: no previous prototype for ‘gim_read_rom_from_reg’ [-Wmissing-prototypes]
169 | void gim_read_rom_from_reg(struct adapter *adapt,
| ^~~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:204:6: warning: no previous prototype for ‘gim_vbios_posted’ [-Wmissing-prototypes]
204 | bool gim_vbios_posted(struct adapter *adapt)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:222:5: warning: no previous prototype for ‘gim_read_vbios’ [-Wmissing-prototypes]
222 | int gim_read_vbios(struct adapter *adapt)
| ^~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:332:7: warning: no previous prototype for ‘map_doorbell’ [-Wmissing-prototypes]
332 | void *map_doorbell(struct pci_dev *pdev)
| ^~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:360:7: warning: no previous prototype for ‘map_fb’ [-Wmissing-prototypes]
360 | void *map_fb(struct pci_dev *pdev)
| ^~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:509:5: warning: no previous prototype for ‘mask_unrecoverable_errors’ [-Wmissing-prototypes]
509 | int mask_unrecoverable_errors(struct pci_dev *dev)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:526:6: warning: no previous prototype for ‘clear_unrecoverable_errors’ [-Wmissing-prototypes]
526 | void clear_unrecoverable_errors(struct pci_dev *dev)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:546:6: warning: no previous prototype for ‘sched_work_handler’ [-Wmissing-prototypes]
546 | void sched_work_handler(struct work_struct *work)
| ^~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c: In function ‘set_new_adapter’:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:697:17: error: implicit declaration of function ‘pci_set_dma_mask’ [-Werror=implicit-function-declaration]
697 | pci_set_dma_mask(curr->pf.pci_dev, 0xffffffffffull);
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c: At top level:
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1016:6: warning: no previous prototype for ‘unmap_mmr_base’ [-Wmissing-prototypes]
1016 | void unmap_mmr_base(struct adapter *p_adapt)
| ^~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1027:6: warning: no previous prototype for ‘pause_all_schedulers’ [-Wmissing-prototypes]
1027 | void pause_all_schedulers(void)
| ^~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1040:6: warning: no previous prototype for ‘idle_one_adapter’ [-Wmissing-prototypes]
1040 | void idle_one_adapter(struct adapter *adapt)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1380:6: warning: no previous prototype for ‘check_me_cntl’ [-Wmissing-prototypes]
1380 | void check_me_cntl(struct adapter *adapt, char *comment)
| ^~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:1399:6: warning: no previous prototype for ‘check_base_addrs’ [-Wmissing-prototypes]
1399 | void check_base_addrs(struct function *func, char *comment)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:2027:5: warning: no previous prototype for ‘world_switch_divided’ [-Wmissing-prototypes]
2027 | int world_switch_divided(struct function *func, struct function *next_func)
| ^~~~~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3279:6: warning: no previous prototype for ‘check_smu_int’ [-Wmissing-prototypes]
3279 | void check_smu_int(uint32_t bdf, char *comment)
| ^~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3344:6: warning: no previous prototype for ‘dump_ucode’ [-Wmissing-prototypes]
3344 | void dump_ucode(struct adapter *adapt, int addr_register,
| ^~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3364:6: warning: no previous prototype for ‘vbios_check’ [-Wmissing-prototypes]
3364 | void vbios_check(struct adapter *adapt, uint32_t bdf, char *comment)
| ^~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3444:6: warning: no previous prototype for ‘read_ih_regs’ [-Wmissing-prototypes]
3444 | void read_ih_regs(uint32_t bdf, char *comment)
| ^~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3500:6: warning: no previous prototype for ‘save_mmio_regs’ [-Wmissing-prototypes]
3500 | void save_mmio_regs(uint32_t bdf, uint32_t *buf, struct named_reg_list *list)
| ^~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3511:6: warning: no previous prototype for ‘restore_mmio_regs’ [-Wmissing-prototypes]
3511 | void restore_mmio_regs(uint32_t bdf, uint32_t *buf, struct named_reg_list *list)
| ^~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3523:6: warning: no previous prototype for ‘compare_mmio_regs’ [-Wmissing-prototypes]
3523 | void compare_mmio_regs(uint32_t *buf1, uint32_t *buf2,
| ^~~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3536:6: warning: no previous prototype for ‘check_pf_regs’ [-Wmissing-prototypes]
3536 | void check_pf_regs(uint32_t bdf, char *comment)
| ^~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3876:5: warning: no previous prototype for ‘check_cp_status’ [-Wmissing-prototypes]
3876 | int check_cp_status(struct adapter *adapt, char *comment, struct function *func)
| ^~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3896:7: warning: no previous prototype for ‘decode_ram_usage’ [-Wmissing-prototypes]
3896 | char *decode_ram_usage(uint32_t index)
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:3940:6: warning: no previous prototype for ‘dump_scratch_ram’ [-Wmissing-prototypes]
3940 | void dump_scratch_ram(struct adapter *adapt,
| ^~~~~~~~~~~~~~~~
/home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.c:4003:5: warning: no previous prototype for ‘amd_try_spinlock’ [-Wmissing-prototypes]
4003 | int amd_try_spinlock(spinlock_t *lock, uint32_t usec_timeout)
| ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make[3]: *** [scripts/Makefile.build:243: /home/TH_2025/V2/MxGPU-Virtualization/drv/gim_adapter.o] Error 1
make[2]: *** [/usr/src/linux-headers-6.8.12-8-pve/Makefile:1925: /home/TH_2025/V2/MxGPU-Virtualization/drv] Error 2
make[1]: *** [Makefile:240: __sub-make] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-6.8.12-8-pve'
make: *** [Makefile:39: all] Error 2

make -C /lib/modules/6.8.12-8-pve/build M=/home/TH_2025/V2/MxGPU-Virtualization/drv modules_install
make[1]: Entering directory '/usr/src/linux-headers-6.8.12-8-pve'
DEPMOD /lib/modules/6.8.12-8-pve
Warning: modules_install: missing 'System.map' file. Skipping depmod.
make[1]: Leaving directory '/usr/src/linux-headers-6.8.12-8-pve'
Click to expand...
im build this module on pve 7.4.1(download from http://download.proxmox.com/iso/proxmox-ve_7.4-1.iso) and success,may be this repo not support linux kenrnel 6.x? if you wish up to use latest pve,you can try to use this repo:https://github.com/nicman23/MxGPU-Virtualization
 
kevinlels said:
im build this module on pve 7.4.1(download from http://download.proxmox.com/iso/proxmox-ve_7.4-1.iso) and success,may be this repo not support linux kenrnel 6.x? if you wish up to use latest pve,you can try to use this repo:https://github.com/nicman23/MxGPU-Virtualization
Click to expand...
Dear @kevinlels, this repo works with PVE 8.3 and Kernel 6.8.

Now the problem is to load the gim.ko by modprobe gim.

INSTALL /lib/modules/6.8.12-8-pve/updates/gim.ko

even if I copy the gim.ko modul direct to /lib/modules/6.8.12-8-pve I get error massage.

>>> modprobe: FATAL: Module gim not found in directory /lib/modules/6.8.12-8-pve
 
T.Herrmann said:
Dear @kevinlels, this repo works with PVE 8.3 and Kernel 6.8.

Now the problem is to load the gim.ko by modprobe gim.

INSTALL /lib/modules/6.8.12-8-pve/updates/gim.ko

even if I copy the gim.ko modul direct to /lib/modules/6.8.12-8-pve I get error massage.

>>> modprobe: FATAL: Module gim not found in directory /lib/modules/6.8.12-8-pve
Click to expand...
you can use
Bash: 
insmod /path/to/gim.ko
or try to
Bash: 
depmod -a
modprobe gim
By the way, if you don't get the windows error code43 or BSOD when installing the client driver, please send me the client driver you tried, I really need it
 
Last edited:
Dear @kevinlels , thank you now it works.

We will test now a Linux Ubuntu 24 VM and a Windows 10 VM.

I will inform you if we are have a stable running for both operation systems.

Bildschirmfoto 2025-03-24 um 08.42.54.png
 
kevinlels said:
First, install git,gcc,make,linux-headers-$(uname -r) through apt for compilation, then refer to the documentation on the web to open IOMMU, and then run "ln -sf /usr/src/linux-headers-$(uname -r) /lib/modules/$(uname -r)/build", then get the source code from the https://github.com/kasperlewau/MxGPU-Virtualization and run the gim.sh, after success, set the blacklist and configure the parameters according to the documents in the repository. After the configuration is completed, load this module through "modprobe gim", now you should be able to see the virtual device through "lspci", add the virtual device to your virtual machine through the webGUI of pve and download the client driver from the AMD official website
（im chinese and my english not very good，so this is by microsoft translate，The original text is as follows）
首先，通过apt安装git,gcc,make,linux-headers-$(uname -r)用于编译,然后参照网上的文档开启IOMMU，接着执行“ln -sf /usr/src/linux-headers-$(uname -r) /lib/modules/$(uname -r)/build”然后从https://github.com/kasperlewau/MxGP...im.sh，成功之后按照仓库中的文档设置黑名单并配置参数。配置完成后通过“modprobe gim”加载这个模块，现在你应该可以通过“lspci”看到虚拟出的设备了，通过pve的WebGUI中给你的虚拟机添加虚拟出的设备并从AMD官网下载客户机驱动即可
Click to expand...
That is not for latest kernel version of PVE.
 
T.Herrmann said:
Dear @Q3tNHn this works for the latest PVE Kernel https://github.com/nicman23/MxGPU-Virtualization
Click to expand...
The latest PVE kernel is 6.8 but this is still 6.5. I compiled and installed the mod anyway and I don't see any vGPU was created by GIM.
```
# dmesg -T | grep gim
[Mon Mar 31 11:24:13 2025] gim: module verification failed: signature and/or required key missing - tainting kernel
[Mon Mar 31 11:24:13 2025] gim info:(gim_init:149) Start AMD open source GIM initialization
[Mon Mar 31 11:24:13 2025] gim info:(gim_init:151) GPU IOV MODULE - version 0.0
[Mon Mar 31 11:24:13 2025] gim info:(gim_init:154) Copyright (c) 2014-2017 Advanced Micro Devices, Inc. All rights reserved.
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM fb_option = 0
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM sched_option = 0
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM vf_num = 10
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM pf_fb = 0
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM vf_fb = 0
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM sched_interval = 0
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM sched_interval_us = 0
[Mon Mar 31 11:24:13 2025] gim info:(parse_config_file:217) AMD GIM fb_clear = 0
[Mon Mar 31 11:24:13 2025] gim info:(init_config:341) INIT CONFIG
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:573) curr allocated at 00000000d9c6c923
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:580) SRIOV is supported
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:588) found PCI bridge device
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:589) found: 06:8.0
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:609) mmio_base = 00000000ac742ba6
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:611) doorbell = 00000000a34b7afd
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:613) pf.fb_va = 00000000cb56d33c
[Mon Mar 31 11:24:13 2025] gim info:(sriov_is_ari_enabled:164) PCI_SRIOV_CAP = 0x00000002
[Mon Mar 31 11:24:13 2025] gim info:(sriov_is_ari_enabled:174) PCI_SRIOV_CTRL = 0x00000010
[Mon Mar 31 11:24:13 2025] gim info:(sriov_is_ari_enabled:177) PCI_SRIOV_CTRL_ARI is set --> ARI is supported
[Mon Mar 31 11:24:13 2025] gim info:(program_ari_mode:442) Read bif_strap8 = 0x00200004
[Mon Mar 31 11:24:13 2025] gim info:(program_ari_mode:447) program_ari_mode - Set ARI_Mode = PF_BUS
[Mon Mar 31 11:24:13 2025] gim info:(program_ari_mode:457) Write bif_strap8 = 0x00000004
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_rom_from_reg:182) Reading VBios from ROM
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:244) VBIOS starts: 0x55, 0xaa
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:247) VBios size is 0x10000
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:250) vbios allocated at 00000000658c5a87
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_rom_from_reg:182) Reading VBios from ROM
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:256) BIOS Version Major 0xF Minor 0x31
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:271) Valid video BIOS image,
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:272) size = 0x10000, check sum is 0x548300
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:662) Scheduler Time interval is per-vf from XL
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:663) config file
[Mon Mar 31 11:24:13 2025] gim info:(enable_sriov:298) Enable SRIOV
[Mon Mar 31 11:24:13 2025] gim info:(enable_sriov:299) Enable SRIOV vfs count = 10
[Mon Mar 31 11:24:13 2025] gim error:(enable_sriov:310) Fail to enable sriov, status = fffffff4
[Mon Mar 31 11:24:13 2025] gim error:(set_new_adapter:669) Failed to properly enable SRIOV
[Mon Mar 31 11:24:13 2025] gim info:(gim_probe:91) AMD GIM probe: pf_count = 1
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:573) curr allocated at 0000000030b9bedd
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:580) SRIOV is supported
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:588) found PCI bridge device
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:589) found: 06:10.0
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:609) mmio_base = 000000001f039bff
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:611) doorbell = 000000007aa5de4a
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:613) pf.fb_va = 000000005d325d24
[Mon Mar 31 11:24:13 2025] gim info:(sriov_is_ari_enabled:164) PCI_SRIOV_CAP = 0x00000002
[Mon Mar 31 11:24:13 2025] gim info:(sriov_is_ari_enabled:174) PCI_SRIOV_CTRL = 0x00000010
[Mon Mar 31 11:24:13 2025] gim info:(sriov_is_ari_enabled:177) PCI_SRIOV_CTRL_ARI is set --> ARI is supported
[Mon Mar 31 11:24:13 2025] gim info:(program_ari_mode:442) Read bif_strap8 = 0x00200004
[Mon Mar 31 11:24:13 2025] gim info:(program_ari_mode:447) program_ari_mode - Set ARI_Mode = PF_BUS
[Mon Mar 31 11:24:13 2025] gim info:(program_ari_mode:457) Write bif_strap8 = 0x00000004
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_rom_from_reg:182) Reading VBios from ROM
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:244) VBIOS starts: 0x55, 0xaa
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:247) VBios size is 0x10000
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:250) vbios allocated at 00000000421c6df2
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_rom_from_reg:182) Reading VBios from ROM
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:256) BIOS Version Major 0xF Minor 0x31
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:271) Valid video BIOS image,
[Mon Mar 31 11:24:13 2025] gim info:(gim_read_vbios:272) size = 0x10000, check sum is 0x548300
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:662) Scheduler Time interval is per-vf from XL
[Mon Mar 31 11:24:13 2025] gim info:(set_new_adapter:663) config file
[Mon Mar 31 11:24:13 2025] gim info:(enable_sriov:298) Enable SRIOV
[Mon Mar 31 11:24:13 2025] gim info:(enable_sriov:299) Enable SRIOV vfs count = 10
[Mon Mar 31 11:24:13 2025] gim error:(enable_sriov:310) Fail to enable sriov, status = fffffff4
[Mon Mar 31 11:24:13 2025] gim error:(set_new_adapter:669) Failed to properly enable SRIOV
[Mon Mar 31 11:24:13 2025] gim info:(gim_probe:91) AMD GIM probe: pf_count = 2

```
Here is the content of /etc/gim_config
```
cat /etc/gim_config
fb_option=0
sched_option=0
vf_num=10
pf_fb=0
vf_fb=0
sched_interval=0
sched_interval_us=0
fb_clear=0
 
Last edited:
You must log in or register to reply here.
Top Bottom