i have several infiniband cluster running with OFED and work good.
If you switch don't have SM, install opensmd at least on 2 node (one master, one slave)
i prefer more use mellanox opensm on the switch but also using on proxmox node for some clusters
mellanox release latest driver recently so...