10GBE card failed to ping

[phoneme]

Hi we are working on proxmox Virtual Environment 6.2-4 and trying to setup the three node cluster ,
Problem is configuring the Fibre Network over the nodes . The proxmox GUI shows the Network cards as active and but NOT able to ping them

vi /etc/network/interfaces

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno52 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

iface eno49 inet manual

iface eno50 inet manual

iface eno51 inet manual

auto ens1f1
iface ens1f1 inet manual

auto ens1f0
#iface ens1f0 inet manual
iface ens1f0 inet static
address 192.168.1.12
netmask 255.255.255.0
gateway 192.168.1.1


#Fiber port 0

#iface bond0 inet static
# address 172.16.15.112/24
# bond-slaves ens1f0 ens1f1
# bond-miimon 100
# bond-mode balance-rr
#Cluster Bond

auto vmbr0
iface vmbr0 inet static
address 172.16.15.12/24
gateway 172.16.15.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
#Proxmox Mgt Network




# bond-slaves ens1f0 ens1f1
# bond-miimon 100
# bond-mode balance-rr
#Cluster Bond

auto vmbr0
iface vmbr0 inet static
address 172.16.15.12/24
gateway 172.16.15.1
bridge-ports eno1
bridge-stp off

and when i check the state of interfaces it shows as DOWN

root@pve3:~# ip link show up
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
link/ether 14:02:ec:04:b1:70 brd ff:ff:ff:ff:ff:ff
10: ens1f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 00:90:fa:79:11:1e brd ff:ff:ff:ff:ff:ff
11: ens1f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 00:90:fa:79:11:22 brd ff:ff:ff:ff:ff:ff
13: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 14:02:ec:04:b1:70 brd ff:ff:ff:ff:ff:ff
root@pve3:~# ip link set dev ens1f0 up
root@pve3:~# ip link show up
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
link/ether 14:02:ec:04:b1:70 brd ff:ff:ff:ff:ff:ff
10: ens1f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 00:90:fa:79:11:1e brd ff:ff:ff:ff:ff:ff
11: ens1f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 00:90:fa:79:11:22 brd ff:ff:ff:ff:ff:ff
13: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 14:02:ec:04:b1:70 brd ff:ff:ff:ff:ff:ff



Please suggest where I am going wrong .

Rg

Phoneme

 

[spirit]

can you try to add "auto eno1" ?

also what is the result of "ethtool eno1" ?

 

[spirit]

and also
ethtool -m eno1 , to see fiber signal stats

 

[phoneme]

Hi
I have added the following :-

can you try to add "auto eno1" ?

I have added

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

iface eno52 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

iface eno49 inet manual

iface eno50 inet manual

iface eno51 inet manual

auto ens1f1
iface ens1f1 inet manual

auto ens1f0
#iface ens1f0 inet manual
iface ens1f0 inet static
address 192.168.1.12
netmask 255.255.255.0
gateway 192.168.1.1


#Fiber port 0

#auto bond0
#iface bond0 inet static
# address 172.16.15.112/24
# bond-slaves ens1f0 ens1f1
# bond-miimon 100
# bond-mode balance-rr
#Cluster Bond

auto vmbr0
iface vmbr0 inet static
address 172.16.15.12/24
gateway 172.16.15.1
bridge-ports eno1


also what is the result of "ethtool eno1" ?

My eno1 is ethernet port and it is working fine ..
ens1f0/1 are fibre ports and they are not working .

 

[phoneme]

root@pve3:~# ethtool -m ens1f0
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x07 (LC)
Transceiver codes : 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Transceiver type : 10G Ethernet: 10G Base-SR
Encoding : 0x06 (64B/66B)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 0km
Length (SMF) : 0m
Length (50um) : 300m
Length (62.5um) : 20m
Length (Copper) : 0m
Length (OM3) : 300m
Laser wavelength : 850nm
Vendor name : Syrotech
Vendor OUI : 00:17:6a
Vendor PN : GOXP-8596-02
Vendor rev : C
Option values : 0x00 0x1a
Option : RX_LOS implemented
Option : TX_FAULT implemented
Option : TX_DISABLE implemented
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : SY1907-HDC0032
Date code : 190627
Optical diagnostics support : Yes
Laser bias current : 6.942 mA
Laser output power : 0.7079 mW / -1.50 dBm
Receiver signal average optical power : 0.4536 mW / -3.43 dBm
Module temperature : 29.50 degrees C / 85.10 degrees F
Module voltage : 3.2728 V
Alarm/warning flags implemented : Yes
Laser bias current high alarm : Off
Laser bias current low alarm : Off
Laser bias current high warning : Off
Laser bias current low warning : Off
Laser output power high alarm : Off
Laser output power low alarm : Off
Laser output power high warning : Off
Laser output power low warning : Off
Module temperature high alarm : Off
Module temperature low alarm : Off
Module temperature high warning : Off
Module temperature low warning : Off
Module voltage high alarm : Off
Module voltage low alarm : Off
Module voltage high warning : Off
Module voltage low warning : Off
Laser rx power high alarm : Off
Laser rx power low alarm : Off
Laser rx power high warning : Off
Laser rx power low warning : Off
Laser bias current high alarm threshold : 95.000 mA
Laser bias current low alarm threshold : 0.000 mA
Laser bias current high warning threshold : 95.026 mA
Laser bias current low warning threshold : 0.000 mA
Laser output power high alarm threshold : 1.2580 mW / 1.00 dBm
Laser output power low alarm threshold : 0.1990 mW / -7.01 dBm
Laser output power high warning threshold : 1.2580 mW / 1.00 dBm
Laser output power low warning threshold : 0.1990 mW / -7.01 dBm
Module temperature high alarm threshold : 90.00 degrees C / 194.00 degrees F
Module temperature low alarm threshold : -25.00 degrees C / -13.00 degrees F
Module temperature high warning threshold : 90.00 degrees C / 194.00 degrees F
Module temperature low warning threshold : -20.00 degrees C / -4.00 degrees F
Module voltage high alarm threshold : 3.8000 V
Module voltage low alarm threshold : 2.7000 V
Module voltage high warning threshold : 3.8000 V
Module voltage low warning threshold : 2.7000 V
Laser rx power high alarm threshold : 1.2580 mW / 1.00 dBm
Laser rx power low alarm threshold : 0.0500 mW / -13.01 dBm
Laser rx power high warning threshold : 1.2580 mW / 1.00 dBm
Laser rx power low warning threshold : 0.0500 mW / -13.01 dBm
root@pve3:~# ping 192.168.1.10

 

[phoneme]

root@pve2:~# ip link set dev ens1f0 up
RTNETLINK answers: Input/output error