Grafik-Beschleunigung ohne Passthrough

gladiolah · Oct 25, 2019

Das hatte ich gemacht... aber mit einer Meldung in Gelb...

Code:

root@XXX:~/MxGPU-Virtualization/drv# git checkout kernel5.0
error: pathspec 'kernel5.0' did not match any file(s) known to git
hint: 'kernel5.0' matched more than one remote tracking branch.
hint: We found 2 remotes with a reference that matched. So we fell back
hint: on trying to resolve the argument as a path, but failed there too!
hint:
hint: If you meant to check out a remote tracking branch on, e.g. 'origin',
hint: you can do so by fully qualifying the name with the --track option:
hint:
hint:     git checkout --track origin/<name>
hint:
hint: If you'd like to always have checkouts of an ambiguous <name> prefer
hint: one remote, e.g. the 'origin' remote, consider setting
hint: checkout.defaultRemote=origin in your config.
root@XXX:~/MxGPU-Virtualization/drv#

Ist das normal?

dcsapak · Oct 28, 2019

mhmm was sagt denn 'git branch -av'
wenn ein remote 'flumm' heißt sollte das auschecken mit
'git checkout flumm/kernel5.0'
funktionieren (und dann bauen)

gladiolah · Oct 28, 2019

Ich vermute, dass das Problem daher kommt, da jetzt zweimal der Eintrag Kernel5.0 vorhanden ist...

Code:

root@XXX:~/MxGPU-Virtualization/drv# git branch -av
* master                   3f297dd The open source GIM-3.0 release
  remotes/flumm/kernel5.0  2d4abed make gim compatible with kernel >= 5.0
  remotes/flumm/master     3f297dd The open source GIM-3.0 release
  remotes/origin/HEAD      -> origin/master
  remotes/origin/kernel5.0 2d4abed make gim compatible with kernel >= 5.0
  remotes/origin/master    3f297dd The open source GIM-3.0 release
root@XXX:~/MxGPU-Virtualization/drv#

Ich weiss aber nicht, wie ich das ändern kann. apt-get purge git? und dann git neu installieren?

Viele Grüsse

dcsapak · Oct 28, 2019

bitte mal mit 'git checkout flumm/kernel5.0' oder 'git checkout origin/kernel5.0' versuchen

gladiolah · Jan 18, 2020

Ich nochmals...

Mein Boss wird ungeduldig

, die für teuer Geld erworbene Karte muss laufen.
Ich habe heute nochmals alles von der Platte geschmissen und alles neu geladen.

Habe mir eine kleine To Do List gebastelt, die soweit auch gut klappt:


apt install pve-headers make git gcc
git clone https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization
git init
git remote add flumm https://github.com/flumm/MxGPU-Virtualization
git fetch --all
git checkout kernel5.0
cd MxGPU-Virtualization/drv
make
insmod gim.ko

sowie dkms config unter /usr/src/gim-3.0 ablegen


nano /usr/src/gim-3.0

dort einfügen:


PACKAGE_NAME=gim
PACKAGE_VERSION=3.0
MAKE[0]="make -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build"
CLEAN="make -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build clean"
BUILT_MODULE_NAME=gim
DEST_MODULE_LOCATION=/extra
REMAKE_INITRD=yes
AUTOINSTALL=yes

und im dkms verankern:


dkms add gim-3.0

Es klappt alles bis auf den make-Befehl, ich vermute ich habe irgendetwaws übersehen oder irgend etwas stimmt mit dem Server nicht.

Ich würde mich über Hilfe freuen, sonst muss ich ein Supportticket lösen. Bei der Ausgabe des Kompilers muss ich passen, ich weiss nicht wo das Problem liegt....

Hier die Ausgabe des make-Befehls, nachdem alles Fehlerfrei durchlief:


root@ILB-SVR00:/MxGPU-Virtualization/drv# make

make -C /lib/modules/5.3.13-1-pve/build M=/MxGPU-Virtualization/drv modules
make[1]: Entering directory '/usr/src/linux-headers-5.3.13-1-pve'
  CC [M]  /MxGPU-Virtualization/drv/gim_adapter.o
In file included from /MxGPU-Virtualization/drv/gim_adapter.c:50:
/MxGPU-Virtualization/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
 #define  PCI_SRIOV_CTRL_VFE                 0x01/* VF Enable */
 
In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /MxGPU-Virtualization/drv/gim_adapter.c:25:
./include/uapi/linux/pci_regs.h:912: note: this is the location of the previous definition
 #define  PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */
 
In file included from /MxGPU-Virtualization/drv/gim_adapter.c:50:
/MxGPU-Virtualization/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
 #define  PCI_SRIOV_CTRL_MSE                 0x08/* VF Memory Space Enable */
 
In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /MxGPU-Virtualization/drv/gim_adapter.c:25:
./include/uapi/linux/pci_regs.h:915: note: this is the location of the previous definition
 #define  PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */
 
/MxGPU-Virtualization/drv/gim_adapter.c: In function ‘idle_vf’:
/MxGPU-Virtualization/drv/gim_adapter.c:1799:9: error: implicit declaration of function ‘timespec_sub’; did you mean ‘timespec64_sub’? [-Werror=implicit-function-declaration]
   tmp = timespec_sub(tmp, func->time_log.active_last_tick);
         ^~~~~~~~~~~~
         timespec64_sub
/MxGPU-Virtualization/drv/gim_adapter.c:1799:7: error: incompatible types when assigning to type ‘struct timespec’ from type ‘int’
   tmp = timespec_sub(tmp, func->time_log.active_last_tick);
       ^
/MxGPU-Virtualization/drv/gim_adapter.c:1801:4: error: implicit declaration of function ‘timespec_add’; did you mean ‘timespec64_add’? [-Werror=implicit-function-declaration]
    timespec_add(func->time_log.active_time, tmp);
    ^~~~~~~~~~~~
    timespec64_add
/MxGPU-Virtualization/drv/gim_adapter.c:1800:30: error: incompatible types when assigning to type ‘struct timespec’ from type ‘int’
   func->time_log.active_time =
                              ^
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:290: /MxGPU-Virtualization/drv/gim_adapter.o] Error 1
make[1]: *** [Makefile:1655: _module_/MxGPU-Virtualization/drv] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.3.13-1-pve'
make: *** [Makefile:39: all] Error 2

Hat irgendwer eine Idee oder Lust mit mir via Teamviewer und Telefon sein Glück zu versuchen? Any Ideas?

Greez und ein Gutes WE alles Lesern.
Gladiolah

mac.linux.free · Jan 18, 2020

bei mir läufts mit bis jetzt auf 5 virtuellen win 10...kann dir gern helfen auch per teamviewer

gladiolah · Jan 20, 2020

mac.linux.free said:
bei mir läufts mit bis jetzt auf 5 virtuellen win 10...kann dir gern helfen auch per teamviewer

Vielen Dank, aber ich glaube das kompilieren hat geklappt. Scheinbar lagen noch alte Dateien im Homeverzeichnis...

root@ILB-SVR00:~/MxGPU-Virtualization/drv# make
-D DRV_VERSION=1.1.4
make -C /lib/modules/5.3.13-1-pve/build M=/root/MxGPU-Virtualization/drv modules
make[1]: Entering directory '/usr/src/linux-headers-5.3.13-1-pve'
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_drv.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_interface.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_adapter.o
In file included from /root/MxGPU-Virtualization/drv/drv/gim_adapter.c:50:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
#define  PCI_SRIOV_CTRL_VFE                 0x01/* VF Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_adapter.c:25:
./include/uapi/linux/pci_regs.h:912: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */

In file included from /root/MxGPU-Virtualization/drv/drv/gim_adapter.c:50:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
#define  PCI_SRIOV_CTRL_MSE                 0x08/* VF Memory Space Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_adapter.c:25:
./include/uapi/linux/pci_regs.h:915: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */

  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_pci.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_unwrapper.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_gpuiov.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_config.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_timer.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_fb.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_debug.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_flr.o
In file included from /root/MxGPU-Virtualization/drv/drv/gim_flr.c:32:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
#define  PCI_SRIOV_CTRL_VFE                 0x01/* VF Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_flr.c:23:
./include/uapi/linux/pci_regs.h:912: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */

In file included from /root/MxGPU-Virtualization/drv/drv/gim_flr.c:32:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
#define  PCI_SRIOV_CTRL_MSE                 0x08/* VF Memory Space Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_flr.c:23:
./include/uapi/linux/pci_regs.h:915: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */

  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_atom.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_atombios.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_os_service.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_irqmgr.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_kcl_os.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_pci_config.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.o
In file included from /root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.c:25:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
#define  PCI_SRIOV_CTRL_VFE                 0x01/* VF Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.c:23:
./include/uapi/linux/pci_regs.h:912: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */

In file included from /root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.c:25:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
#define  PCI_SRIOV_CTRL_MSE                 0x08/* VF Memory Space Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.c:23:
./include/uapi/linux/pci_regs.h:915: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */

  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_reset.o
In file included from /root/MxGPU-Virtualization/drv/drv/gim_reset.c:27:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
#define  PCI_SRIOV_CTRL_VFE                 0x01/* VF Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_reset.c:23:
./include/uapi/linux/pci_regs.h:912: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */

In file included from /root/MxGPU-Virtualization/drv/drv/gim_reset.c:27:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
#define  PCI_SRIOV_CTRL_MSE                 0x08/* VF Memory Space Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_reset.c:23:
./include/uapi/linux/pci_regs.h:915: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */

  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_dma.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_monitor.o
In file included from /root/MxGPU-Virtualization/drv/drv/gim_monitor.c:32:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:73: warning: "PCI_SRIOV_CTRL_VFE" redefined
#define  PCI_SRIOV_CTRL_VFE                 0x01/* VF Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_pci_config.h:27,
                 from /root/MxGPU-Virtualization/drv/drv/gim_adapter.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_monitor.c:26:
./include/uapi/linux/pci_regs.h:912: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_VFE 0x0001 /* VF Enable */

In file included from /root/MxGPU-Virtualization/drv/drv/gim_monitor.c:32:
/root/MxGPU-Virtualization/drv/drv/gim_kcl_pci.h:74: warning: "PCI_SRIOV_CTRL_MSE" redefined
#define  PCI_SRIOV_CTRL_MSE                 0x08/* VF Memory Space Enable */

In file included from ./include/uapi/linux/pci.h:21,
                 from ./include/linux/pci.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_pci_config.h:27,
                 from /root/MxGPU-Virtualization/drv/drv/gim_adapter.h:35,
                 from /root/MxGPU-Virtualization/drv/drv/gim_monitor.c:26:
./include/uapi/linux/pci_regs.h:915: note: this is the location of the previous definition
#define  PCI_SRIOV_CTRL_MSE 0x0008 /* VF Memory Space Enable */

  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_monitor_ioctl.o
  CC [M]  /root/MxGPU-Virtualization/drv/drv/gim_monitor_tonga.o
  LD [M]  /root/MxGPU-Virtualization/drv/gim.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /root/MxGPU-Virtualization/drv/gim.mod.o
  LD [M]  /root/MxGPU-Virtualization/drv/gim.ko
make[1]: Leaving directory '/usr/src/linux-headers-5.3.13-1-pve'
root@ILB-SVR00:~/MxGPU-Virtualization/drv#

Jetzt geht es mal an das installieren...

EDIT:

insmod gim.ko hat problemlos geklappt, allerdings bekomme ich dann auf dem Screen des Servers folgende Meldung:

Any Ideas?! Benötigt die GraKa ein anderes Bios?

Es handelt sich ja bei der Karte um die Dual-GPU-Version.

Ich freue mich schon, wenn das Thema erledigt ist und die Karte läuft.

Grüezi

mac.linux.free · Jan 21, 2020

hatte ich auch...ihr müsst unter windows 10 die karte als zusatzkarte definieren und nicht als haupt vga...so hat es zumindest bei mir geklappt

Jsingh · May 7, 2020

Were you able to run the amd s7150 on the proxmox as I am planning to buy one for a epyc 7351p on a asrock rack epycd8-2 motherboard

mac.linux.free · May 7, 2020

Jsingh said:
Were you able to run the amd s7150 on the proxmox as I am planning to buy one for a epyc 7351p on a asrock rack epycd8-2 motherboard

Yes it is working for 10 CAD VMs. But like it is stated in the Proxmox Wiki. Only use real Server Mainboards like Supermicro or Dell.

gladiolah · May 8, 2020

Jsingh said:
Were you able to run the amd s7150 on the proxmox as I am planning to buy one for a epyc 7351p on a asrock rack epycd8-2 motherboard

mac.linux.free said:
es it is working for 10 CAD VMs. But like it is stated in the Proxmox Wiki. Only use real Server Mainboards like Supermicro or Dell.

This is true. I also were able to compile the drivers correctly and get this graphics card to work on this CPU and on a Gigabyte Mainboard. We now have 36 v-machines on this node. It works fine.
Thanks at all

Jsingh · Dec 8, 2020

mac.linux.free said:
Yes it is working for 10 CAD VMs. But like it is stated in the Proxmox Wiki. Only use real server mainboards like Supermicro or Dell.

So I am able to get the GPU up and running but I only see two VGPU's that can be passed through. Any way to increase this number?

Also the performance in the RDP session is slow , any way to increase it?

Jsingh · Jan 21, 2021

gladiolah said:
This is true. I also were able to compile the drivers correctly and get this graphics card to work on this CPU and on a Gigabyte mainboard. We now have 36 v-machines on this node. It works fine.
Thanks at all

I have bought the card but I am having deployment issues.

When I try the pass a virtual function to a VM, the guest crashes.

dcsapak · Jan 21, 2021

Jsingh said:
When I try the pass a virtual function to a VM, the guest crashes.

any logs?

Jsingh · Jan 21, 2021

dcsapak said:
any logs?

I have attached the log file.

My system config is:

AMD Epyc 7351p
Asrock Rack EPYCD8-2T
8X16-2666 Mhz Rdimms
Adaptec 71605 Raid Controller in HBA mode
4 X 1TB HDD in ZFS mirror
Intel 750 for L2ARC & SLOG

dcsapak · Jan 22, 2021

ok it seems it errors out in the gim driver itself, so i would ask on the official github of that project (i do not know the gim source/internals good enough to efficiently debug those errors) https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization

Jsingh · Jan 29, 2021

dcsapak said:
ok it seems it errors out in the gim driver itself, so i would ask on the official github of that project (i do not know the gim source / internals good enough to efficiently debug those errors) https://github.com/GPUOpen -LibrariesAndSDKs / MxGPU-Virtualization

I have made a posting on the project page but I am not that confident i'll get any resolution.

From what I've read around you guys have tested the card on a similar setup. Could you share some settings or pointers on what I should ensure?

dcsapak · Jan 29, 2021

Jsingh said:
From what I've read around you guys have tested the card on a similar setup. Could you share some settings or pointers on what I should ensure?

on our epyc setup, the card did simply work, there were no real troubles except some combination of guest drivers caused problems ( see https://pve.proxmox.com/wiki/MxGPU_with_AMD_S7150_under_Proxmox_VE_5.x for details)
i do not see anything in your logs where i would know what to do honestly...

Jsingh · Feb 1, 2021

dcsapak said:
on our epyc setup, the card did simply work, there were no real troubles except some combination of guest drivers caused problems (see https://pve.proxmox.com/wiki/MxGPU_with_AMD_S7150_under_Proxmox_VE_5.x for details)
i do not see anything in your logs where i would know what to do honestly ...

Ok, So I was able to get the card to work by disabling the primary GPU option for the pci device. Thanks

Jsingh · Feb 1, 2021

gladiolah said:
This is true. I also were able to compile the drivers correctly and get this graphics card to work on this CPU and on a Gigabyte mainboard. We now have 36 v-machines on this node. It works fine.
Thanks at all

mac.linux.free said:
Yes it is working for 10 CAD VMs. But like it is stated in the Proxmox Wiki. Only use real server mainboards like Supermicro or Dell.

I have a similar use case and using this using it for running 4 CAD VMS. What are you using for VDI, RDP seems slow?

Grafik-Beschleunigung ohne Passthrough

Active Member

Proxmox Staff Member

Active Member

Proxmox Staff Member

Active Member

Renowned Member

Active Member

Renowned Member

Well-Known Member

Renowned Member

Active Member

Well-Known Member

Well-Known Member

Proxmox Staff Member

Well-Known Member

Attachments

Proxmox Staff Member

Well-Known Member

Proxmox Staff Member

Well-Known Member

Well-Known Member