Proxmox on a Cisco UCS 240-M5 and issues with the VIC 1457

[jlo]

Hoping that someone can point me in a direction please?

I am having some issues with being able to use the VIC 1457 on my proxmox build (current at 9.2.3 but installed about a year ago and I think it was 8.7.X

For the original installation I used only the 2 x onboard interfaces and they work just fine, however I am now trying to bring up a 10G connection over fiber on the VIC 1457 MLOM card.

The interfaces do show up in the proxmox interface and I do see link on my Catalyst 2960X switch, however cannot ping the interface. I have gone through the CIMC and turned off all kinds of bits and currently have them in 'physical NIC mode'



I have also turned off the FEC and have changed the VLAN mode to Access...



I create a bridge interface and use one of the NIC interfaces, but cannot ping the IP on the bridge



Interestingly, what I see on switch versus on the Proxmox CLI shows a different MAC...



and



Thus the MAC/interface that proxmox and the CIMC show is not what is showing up on my switch..

Any ideas? what am I missing?

Thanks, Jeff

 

[longer]

I don't have experience with Cisco UCS and VIC, but I have came across some thread [0] mentioning that marking the bridge devices as VLAN aware [1] will make the VIC cards work as designed. So I think you can attempt adding:

        bridge-vlan-aware yes
        bridge-vids 2-4094

To your vmbr2 config.

[0] https://forum.proxmox.com/threads/c...-network-issues-in-proxmox.114952/post-791408
[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_vlan_on_the_host

 

[jlo]

Thanks, being a network guy this would infer that I need to change the switch config to be a trunk and to specify a single or range of vlans present in the trunk.

How do I then specify which VLAN to use on the servers side it the VIDS are 2-4094? or would I specify a single vlan ID in the bridge-vids field? And a follow up question would then be - if I wanted to use multiple vlans on the port, would I have multiple bridge definitions in proxmox that use the same physical port?

 

[longer]

Yes you can start with bridge-vids 2-4094 as on the example. Later if this help to bring the interface working, you can adjusted to allow only the needed VLAN, as this advised from both security and performance wise.

Regarding the second question, yes correct for example from pve-doc [0]:

Example: Use VLAN 5 for the Proxmox VE management IP with VLAN aware Linux bridge
auto lo
iface lo inet loopback

iface eno1 inet manual


auto vmbr0.5
iface vmbr0.5 inet static
address 10.10.10.2/24
gateway 10.10.10.1

auto vmbr0
iface vmbr0 inet manual
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_vlan_on_the_host