Proxmox and Fibre Channel

[Jamie]

I have installed a dual slot 8gb/s fibre channel and I can see the device in an iommu group and can also see that its driver is loaded and I can see information about the device like link speed and status. However, I cannot see it as a network interface.

How do I configure each slot as a network interface?

Some attached screenies:




Thanks

 

[Stoiko Ivanov]

fibre-channel is not a network interface where you configure an ip ?!
if you export a LUN to that interface it should show up as a blockdevice (`lsblk` output, `dmesg` output)

I hope that helps!

 

[Jamie]

How can I set this up for ipfc ip over fc?

 

[ermanishchawla]

You have to use iscsi

 

[Stoiko Ivanov]

How can I set this up for ipfc ip over fc?

just learned that ip over fc does exist
I'm not sure that this has much widespread/well tested use - from a quick search on the net it seems that it's not too well supported (and support might have been dropped - based on a quick grep through the kernel-sources).

see e.g.: https://www.reddit.com/r/linuxadmin/comments/6sjzia/ip_over_fc/
https://serverfault.com/questions/300699/how-do-i-set-up-ip-over-fc

 

[Jamie]

just learned that ip over fc does exist
I'm not sure that this has much widespread/well tested use - from a quick search on the net it seems that it's not too well supported (and support might have been dropped - based on a quick grep through the kernel-sources).

see e.g.: https://www.reddit.com/r/linuxadmin/comments/6sjzia/ip_over_fc/
https://serverfault.com/questions/300699/how-do-i-set-up-ip-over-fc

If thats the case, then how can I achieve 10gb/s from node to node? I am not sure how iscsi works.

 

[BobhWasatch]

If thats the case, then how can I achieve 10gb/s from node to node? I am not sure how iscsi works.

10 Gbit Ethernet? Infiniband?

 

[Jamie]

so would a ConnectX-2 VPI do what I want?

 

[Stoiko Ivanov]

ConnectX-2 VPI

from a quick search this has been EOL since 2016 - https://www.mellanox.com/support/eol
as far as i remember the drivers for the ConnectX-3 (a newer model) are due to be dropped from the linux-kernel at some point soon - so I'd probably not invest in this currently

You could try to get a Intel 10G nic (x520, x740) - if it's within your budget...

 

[ermanishchawla]

I have setup my server with 2 10GbE NIC with Multipath for iscsi and I am getting a very good performance
I followed this

https://icicimov.github.io/blog/vir...-volume-to-Proxmox-to-support-Live-Migration/