Cannot bind Mellanox Card to VM

[milkytwix]

So, I just reinstalled my Mellanox ConnectX-3 40g Card into my good old proxmox server. I had this card in it for some time, but driver issues after updates were to cumbersome, so I went back to my more reliable (albeit way slower) 10g Intel Card. But now I need some "oomph" on a VM and reinstalled the card. Added it via PCI-pasthrough, started the VM and got the following error:
TASK ERROR: Cannot bind 0000:61:00.0 to vfio
When I look at the card itself it is recognized by the host:

lspci
...
60:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
61:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]
61:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
61:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
62:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10)
...

What is the issue here, how to proceed? When I do a search with this error all I find a GPU issues, so I am bit lost

mst status

mst status
MST modules:
------------
    MST PCI module loaded
    MST PCI configuration module loaded

MST devices:
------------
/dev/mst/mt4099_pciconf0         - PCI configuration cycles access.
                                   domain:bus:dev.fn=0000:61:00.0 addr.reg=88 data.reg=92 cr_bar.gw_offset=-1
                                   Chip revision is: 01
/dev/mst/mt4099_pci_cr0          - PCI direct access.
                                   domain:bus:dev.fn=0000:61:00.0 bar=0xe8200000 size=0x100000
                                   Chip revision is: 01

mst query does not work?

mlxconfig -d /dev/mst/mt4099_pciconf0 query
-E- Failed to open device: /dev/mst/mt4099_pciconf0. Cannot perform operation, Driver might be down.

but driver should be loaded:

mst start
Starting MST (Mellanox Software Tools) driver set
[warn] mst_pci is already loaded, skipping
[warn] mst_pciconf is already loaded, skipping
Create devices

Whats wrong here?