Would anybody here happen to know how to set the MAC address for an Infiniband Virtual Function?

[alpha754293]

So far, I have gotten SR-IOV running in Proxmox 7.4-17 where when I list the PCI devices via lspci | grep Mell, I can see the virtual functions of my Mellanox ConnectX-4 card (on different/multiple nodes).

The problem that I am running into is that because the HW MAC address isn't unique between them the virtual functions, therefore; when I try to use them inside a VM, I get a "NO CARRIER" message when I check it with ip link

When I try to set the HW MAC address using this command:
ip link set ibp10s0f0 vf 0 mac 00:00:10:87:fe:80:00:00:00:00:00:00:24:8a:07:03:00:2b:1e:c7

the error message that I am getting is:
RTNETLINK answers: Operation not supported

I tried googling how to set it, but a LOT of references dealing with Infiniband is on Red Hat and/or Red Hat derived distros, and Debian, because it uses /etc/network/interfaces, works a little differently, i.e. I don't know if I can set or define the MAC address there, and then still be able to unbind and bind the virtual function so that it can be used by a VM.

Any help is greatly appreciated.

Thank you.

 

[jamesthetechie]

So far, I have gotten SR-IOV running in Proxmox 7.4-17 where when I list the PCI devices via lspci | grep Mell, I can see the virtual functions of my Mellanox ConnectX-4 card (on different/multiple nodes).

The problem that I am running into is that because the HW MAC address isn't unique between them the virtual functions, therefore; when I try to use them inside a VM, I get a "NO CARRIER" message when I check it with ip link

When I try to set the HW MAC address using this command:
ip link set ibp10s0f0 vf 0 mac 00:00:10:87:fe:80:00:00:00:00:00:00:24:8a:07:03:00:2b:1e:c7

the error message that I am getting is:
RTNETLINK answers: Operation not supported

I tried googling how to set it, but a LOT of references dealing with Infiniband is on Red Hat and/or Red Hat derived distros, and Debian, because it uses /etc/network/interfaces, works a little differently, i.e. I don't know if I can set or define the MAC address there, and then still be able to unbind and bind the virtual function so that it can be used by a VM.

Any help is greatly appreciated.

Thank you.

for infiniband you dont set mac you need to set port and node GUID.

i have a tutorial here you can follow, just keep in mind if you have multiple VF's, then you need to change the script a little:
https://forum.proxmox.com/threads/tutorial-proxmox-8-mellanox-infiniband-and-sr-iov.146196/