Hello.
I have an Intel x520-DA2 10Gig card, SFPs from Intel, that does not "start up" after booting the current PVE version. As in, during boot the link leds on the card go off and stay that way.
However, when i physically unplug & replug the SFP, the link becomes available (ixgbe 0000:81:00.1 eth3: NIC Link is Up 10 Gbps, Flow Control: RX/TX).
I do not have that problem with a barebone Debian Jessie install, the interfaces come up just fine (Kernel 3.16.0-4-amd64, ixgbe 3.19.1-k).
Tried the "allow_unsupported_sfp" and "ip link set ethX up" or "ethtool -r eth2" without success. So far, only pulling the SFP out & back in works so far.
So, maybe ixgbe version related ? Anyone got any hints on where to look ?
Thanks.
I have an Intel x520-DA2 10Gig card, SFPs from Intel, that does not "start up" after booting the current PVE version. As in, during boot the link leds on the card go off and stay that way.
However, when i physically unplug & replug the SFP, the link becomes available (ixgbe 0000:81:00.1 eth3: NIC Link is Up 10 Gbps, Flow Control: RX/TX).
I do not have that problem with a barebone Debian Jessie install, the interfaces come up just fine (Kernel 3.16.0-4-amd64, ixgbe 3.19.1-k).
Tried the "allow_unsupported_sfp" and "ip link set ethX up" or "ethtool -r eth2" without success. So far, only pulling the SFP out & back in works so far.
Code:
proxmox-ve: 4.2-58 (running kernel: 4.4.13-2-pve)
pve-manager: 4.2-17 (running version: 4.2-17/e1400248)
pve-kernel-4.4.6-1-pve: 4.4.6-48
pve-kernel-4.4.13-1-pve: 4.4.13-56
pve-kernel-4.4.13-2-pve: 4.4.13-58
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-43
qemu-server: 4.0-85
pve-firmware: 1.1-8
libpve-common-perl: 4.0-71
libpve-access-control: 4.0-17
libpve-storage-perl: 4.0-56
pve-libspice-server1: 0.12.8-1
vncterm: 1.2-1
pve-qemu-kvm: 2.6-1
pve-container: 1.0-71
pve-firewall: 2.0-29
pve-ha-manager: 1.0-32
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5.7-pve10~bpo80
Code:
filename: /lib/modules/4.4.13-2-pve/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
version: 4.4.6
license: GPL
description: Intel(R) 10GbE PCI Express Linux Network Driver
author: Intel Corporation, <linux.nics@intel.com>
srcversion: A0F436FF26ECE3DC8455D1E
alias: pci:v00008086d000015ADsv*sd*bc*sc*i*
alias: pci:v00008086d000015ACsv*sd*bc*sc*i*
alias: pci:v00008086d000015ABsv*sd*bc*sc*i*
alias: pci:v00008086d000015AAsv*sd*bc*sc*i*
alias: pci:v00008086d000015D1sv*sd*bc*sc*i*
alias: pci:v00008086d00001563sv*sd*bc*sc*i*
alias: pci:v00008086d00001560sv*sd*bc*sc*i*
alias: pci:v00008086d00001558sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Asv*sd*bc*sc*i*
alias: pci:v00008086d00001557sv*sd*bc*sc*i*
alias: pci:v00008086d0000154Fsv*sd*bc*sc*i*
alias: pci:v00008086d0000154Dsv*sd*bc*sc*i*
alias: pci:v00008086d00001528sv*sd*bc*sc*i*
alias: pci:v00008086d000010F8sv*sd*bc*sc*i*
alias: pci:v00008086d0000151Csv*sd*bc*sc*i*
alias: pci:v00008086d00001529sv*sd*bc*sc*i*
alias: pci:v00008086d0000152Asv*sd*bc*sc*i*
alias: pci:v00008086d000010F9sv*sd*bc*sc*i*
alias: pci:v00008086d00001514sv*sd*bc*sc*i*
alias: pci:v00008086d00001507sv*sd*bc*sc*i*
alias: pci:v00008086d000010FBsv*sd*bc*sc*i*
alias: pci:v00008086d00001517sv*sd*bc*sc*i*
alias: pci:v00008086d000010FCsv*sd*bc*sc*i*
alias: pci:v00008086d000010F7sv*sd*bc*sc*i*
alias: pci:v00008086d00001508sv*sd*bc*sc*i*
alias: pci:v00008086d000010DBsv*sd*bc*sc*i*
alias: pci:v00008086d000010F4sv*sd*bc*sc*i*
alias: pci:v00008086d000010E1sv*sd*bc*sc*i*
alias: pci:v00008086d000010F1sv*sd*bc*sc*i*
alias: pci:v00008086d000010ECsv*sd*bc*sc*i*
alias: pci:v00008086d000010DDsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Bsv*sd*bc*sc*i*
alias: pci:v00008086d000010C8sv*sd*bc*sc*i*
alias: pci:v00008086d000010C7sv*sd*bc*sc*i*
alias: pci:v00008086d000010C6sv*sd*bc*sc*i*
alias: pci:v00008086d000010B6sv*sd*bc*sc*i*
depends: ptp,dca,vxlan
vermagic: 4.4.13-2-pve SMP mod_unload modversions
parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default IntMode (deprecated) (array of int)
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int)
parm: MQ:Disable or enable Multiple Queues, default 1 (array of int)
parm: DCA:Disable or enable Direct Cache Access, 0=disabled, 1=descriptor only, 2=descriptor and data (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues, default 0=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 = disable (1 queue) 2-16 enable (default=8) (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int)
parm: VEPA:VEPA Bridge Mode: 0 = VEB (default), 1 = VEPA (array of int)
parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (0,1,956-488281), default 1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1) (array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500) (array of int)
parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type (array of int)
parm: LLIVLANP:Low Latency Interrupt on VLAN priority threshold (array of int)
parm: FdirPballoc:Flow Director packet buffer allocation level:
1 = 8k hash filters or 2k perfect filters
2 = 16k hash filters or 4k perfect filters
3 = 32k hash filters or 8k perfect filters (array of int)
parm: AtrSampleRate:Software ATR Tx packet sample rate (array of int)
parm: FCoE:Disable or enable FCoE Offload, default 1 (array of int)
parm: MDD:Malicious Driver Detection: (0,1), default 1 = on (array of int)
parm: LRO:Large Receive Offload (0,1), default 0 = off (array of int)
parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599 based adapters, default 0 = Disable (array of int)
parm: dmac_watchdog:DMA coalescing watchdog in microseconds (0,41-10000), default 0 = off (array of int)
parm: vxlan_rx:VXLAN receive checksum offload (0,1), default 1 = Enable (array of int)
Code:
[ 2.517622] ixgbe: 0000:04:00.0: ixgbe_check_options: FCoE Offload feature enabled
[ 3.150936] ixgbe 0000:04:00.0: PCI Express bandwidth of 32GT/s available
[ 3.150939] ixgbe 0000:04:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[ 3.191182] ixgbe 0000:04:00.0 eth0: MAC: 3, PHY: 3, PBA No: 030C00-000
[ 3.191188] ixgbe 0000:04:00.0: 0c:c4:7a:6f:37:a0
[ 3.191192] ixgbe 0000:04:00.0 eth0: Enabled Features: RxQ: 40 TxQ: 40 FdirHash
[ 3.213378] ixgbe 0000:04:00.0 eth0: Intel(R) 10 Gigabit Network Connection
[ 3.213623] ixgbe: 0000:04:00.1: ixgbe_check_options: FCoE Offload feature enabled
[ 3.851388] ixgbe 0000:04:00.1: PCI Express bandwidth of 32GT/s available
[ 3.851391] ixgbe 0000:04:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[ 3.891640] ixgbe 0000:04:00.1 eth1: MAC: 3, PHY: 3, PBA No: 030C00-000
[ 3.891648] ixgbe 0000:04:00.1: 0c:c4:7a:6f:37:a1
[ 3.891652] ixgbe 0000:04:00.1 eth1: Enabled Features: RxQ: 40 TxQ: 40 FdirHash
[ 3.913851] ixgbe 0000:04:00.1 eth1: Intel(R) 10 Gigabit Network Connection
[ 3.934518] ixgbe: 0000:81:00.0: ixgbe_check_options: FCoE Offload feature enabled
[ 4.073508] ixgbe 0000:81:00.0: PCI Express bandwidth of 32GT/s available
[ 4.073511] ixgbe 0000:81:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[ 4.073594] ixgbe 0000:81:00.0 eth2: MAC: 2, PHY: 18, SFP+: 5, PBA No: E68793-006
[ 4.073596] ixgbe 0000:81:00.0: 90:e2:ba:61:15:f4
[ 4.073599] ixgbe 0000:81:00.0 eth2: Enabled Features: RxQ: 40 TxQ: 40 FdirHash
[ 4.073648] ixgbe 0000:81:00.0 eth2: Intel(R) 10 Gigabit Network Connection
[ 4.094227] ixgbe: 0000:81:00.1: ixgbe_check_options: FCoE Offload feature enabled
[ 4.241471] ixgbe 0000:81:00.1: PCI Express bandwidth of 32GT/s available
[ 4.241475] ixgbe 0000:81:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[ 4.241558] ixgbe 0000:81:00.1 eth3: MAC: 2, PHY: 18, SFP+: 6, PBA No: E68793-006
[ 4.241560] ixgbe 0000:81:00.1: 90:e2:ba:61:15:f5
[ 4.241562] ixgbe 0000:81:00.1 eth3: Enabled Features: RxQ: 40 TxQ: 40 FdirHash
[ 4.241609] ixgbe 0000:81:00.1 eth3: Intel(R) 10 Gigabit Network Connection
[ 10.611497] ixgbe 0000:81:00.0: registered PHC device on eth2
[ 10.779740] ixgbe 0000:81:00.0 eth2: detected SFP+: 5
[ 11.245392] ixgbe 0000:81:00.1: registered PHC device on eth3
[ 11.576488] ixgbe 0000:04:00.0: registered PHC device on eth0
[ 12.396296] ixgbe 0000:81:00.1 eth3: detected SFP+: 6
[ 14.703531] ixgbe 0000:04:00.0 eth0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
So, maybe ixgbe version related ? Anyone got any hints on where to look ?
Thanks.
Last edited: