4.15 based test kernel for PVE 5.x available

Discussion in 'Proxmox VE: Installation and configuration' started by fabian, Mar 12, 2018.

  1. 0xFelix

    0xFelix New Member

    Joined:
    Oct 25, 2017
    Messages:
    15
    Likes Received:
    2
    The 4.15.17-3 test kernel with the intel out of tree drivers fixes the problem I have with my HP DL160 G6. The network is coming up normally again after booting.
     
    micro and tjh like this.
  2. micro

    micro Member
    Proxmox Subscriber

    Joined:
    Nov 28, 2014
    Messages:
    58
    Likes Received:
    12
    I was unable to boot one of my nodes after upgrading it yesterday to PVE 5.2 and rebooting with the new 4.15 (kernel pve-kernel-4.15.17-2-pve / 4.15.17-10). The booting process was stuck with slowly spitting out some messages:

    and then Call trace.....

    which I can attach if you are interested in, but it is very ugly because of some remote IMM2 console artifacts.

    I guess the problem was that I'm having some Intel gigabit cards too?



    After two more unsuccessful attempts to boot with 4.15 I changed the default grub boot order and booted successfully with kernel 4.13

    Looks like PVE team already have decided that the problem is somewhere in the kernel tree (ixgbe driver) and decided to revert back to the separate (external driver) in a new build of 4.15 in the test repository.

    I can't afford to experiment with test repository kernels (except something critical happens), so for now I'm keeping the 4.13. I prefer to wait some time when this build arrives to the enterprise repository I give 4.15 a chance again.
     
  3. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,232
    Likes Received:
    169
    We now have a PVE 4.15 kernel with out-of-tree Intel Ethernet drivers available on our community repository. If you have the 'pve-kernel-4.15' meta package installed the update should get in automatically with an
    Code:
    apt update
    apt full-upgrade
    
    The respective package which should get installed is named: pve-kernel-4.15.17-3-pve in version 4.15.17-12

    This is a newer version as the manual downloadable kernel from my previous post, as you may have noticed.
    It is the same in regards to OOT intel drivers but also includes a backport fix for SUN NICs in combination with Open vSwitch.
    So if you installed the manual version and do not use SUN NICs (or OVS) then you may skip this specific version.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. djzort

    djzort New Member

    Joined:
    Aug 8, 2013
    Messages:
    27
    Likes Received:
    1
    is a debug package available to help analyzing crashes?
    i.e. pve-kernel-4.15.17-2-pve-dbg ?
     
  5. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    Reposting here https://forum.proxmox.com/threads/i...onnections-over-jumbo-frames-anomalies.44555/

    I noticed this thread just now, sorry for the noise ...

    ========
    I have recently routinely updated my 3 node pve 5.2 cluster setup, installing new pve-kernel-4.15.17-3-pve .

    I started to have problems in syncronizing drbd resources over a link with jumbo frames enabled:


    Jun 14 08:59:26 pve1 kernel: [40906.042440] drbd vm-102-disk-1/0 drbd103 pve3: Began resync as SyncSource (will sync 3476 KB [869 bits set]).
    Jun 14 08:59:39 pve1 kernel: [40918.313936] drbd vm-102-disk-1 pve3: [drbd_s_vm-102-d/3075] sending time expired, ko = 6
    Jun 14 08:59:45 pve1 kernel: [40924.458069] drbd vm-102-disk-1 pve3: [drbd_s_vm-102-d/3075] sending time expired, ko = 5
    Jun 14 08:59:51 pve1 kernel: [40930.602191] drbd vm-102-disk-1 pve3: [drbd_s_vm-102-d/3075] sending time expired, ko = 4
    Jun 14 08:59:57 pve1 kernel: [40936.746355] drbd vm-102-disk-1 pve3: [drbd_s_vm-102-d/3075] sending time expired, ko = 3


    My configuration uses drbd9 in a dedicated network mesh configuration described here:

    https://lists.gt.net/drbd/users/28251#28251

    In brief, I put two interfaces in a "drbdbr" bridge on each host, blocking forwarding with ebtables rules.

    Each interface has jumbo frames (mtu=9000) enabled.

    I suspect the problem is in the update (out of tree) igb driver,

    Intel(R) Gigabit Ethernet Linux Driver - version 5.3.5.18

    because in the forum I saw there was problems with jumbo frames in the past.

    I reverted back to the previous "pve-kernel-4.15.17-2-pve" kernel with in-tree igb driver:

    igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
    to restore correct functionality. Unfortunately removing latest kernel has as side effect the removal of proxmox-ve and pve-kernel-4.15 packages.

    Any hint?

    Thanks,
    rob
     
  6. Alwin

    Alwin Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2017
    Messages:
    2,317
    Likes Received:
    206
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    @Alwin Ehm, I would like to note respectfully that this is not double posting. I re-posted here because I thought was less confusing, not more. I have stated clearly (I guess) in the former thread (as well) that my issue is more appropriate here.
     
  8. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,232
    Likes Received:
    169
    Your message here is totally fine, you already excused for the extra noise, Alwin may just have overlooked it.

    I'll try to investigate, initial IGB tests looked just fine here, I'll retry high MTU tests.

    I've to admit that I really do not like the intel ethernet driver situation at the moment, even if it got already better...
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    Hello Thomas, nice to meet you here ... I think also that @Alwin overlooked it, after all that post seems duplicated a lot :) .

    Regarding Intel drivers, I had absolutely nothing to blame igb for till now; 4.15.17-2 (with in-tree igb driver) works flawlessly, and I have the neat feeling that this kernel (thanks to better bridging support?) gave a boost in drbd9 performance (remember that my situation is quite particular with two interfaces in a no-forward bridge for implementing mesh).

    I will keep 4.15 series for the moment, will be glad to help in tracking down this ...

    bye,
    rob
     
  10. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,232
    Likes Received:
    169
    Hi Roberto, likewise!

    Ah, not the driver code itself directly, it's just that the in vs out of tree model, which the continue to develop simultaneously, makes life not easier for us and PVE admins...
    The have often subtle bug differences and the out-of-tree ones are released in an unperiodic manner. Intel people say that the intree one has more features and will be the way to go, in the long run at least.
    But it seems we got bitten more by the intree than the outtree one from subtle bugs, so the switch back again...

    Hmm, which kernel did you run previously to the 4.15 ones? Because 4.13 and 4.4 (from PVE 4) had all the out of tree driver, like now 4.15.17-3 has again.

    You should be able to install an older version of pve-kernel-4.15 meta package to address this
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  11. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    I am under pve-no-subscription, and dist-upgrading quite often, so i passed form 4.13 and then through 4.15.17-1 4.15.17-2 and 4.15.17-3 . Only latest give me unresolvable syncronization problems under drbd9 with "mtu 9000" in place.

    Aaah right!

    Anyway at the moment I kept 4.15.17-3 installed and managed to default booting to 4.15.17-2 .

    bye,
    rob
     
  12. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    As a further note, I must say that the syncronization problem is very subtle. Most resources syncronize immediately, but two or three are steadily refusing: syncronization starts, goes till 100%, but never ends; in the syslog of the source node I see (quoting from my previous message):

    rob
     
  13. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    Another note: In the least powerful node (used only as quorum and storage redundancy) I had to add a fourth interface, which uses "jme" driver. This one is in bridge with an "igb" one to form the "drbdbr" interface. Will try to change this and put two igb in the bridge and see if something changes.

    rob
     
  14. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    A question: Under BSD the man page of igb suggests some tunables, and a couple are jumbo frame related:

    http://manpages.ubuntu.com/manpages/trusty/man4/if_igb.4freebsd.html
    kern.ipc.nmbclusters
    The maximum number of mbuf clusters allowed. If the system has
    more than one igb card or jumbo frames are enabled, this value
    will need to be increased.

    kern.ipc.nmbjumbo9k
    The maximum number of mbuf 9k jumbo clusters allowed. Increasing
    this to allow for at least 8192 extra clusters per interface can
    allow for an mtu of 8192.
    Are there similar tunables for linux os?

    rob
     
  15. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,232
    Likes Received:
    169
    Can confirm problems with the current 4.15 out of tree igb driver and MTUs which aren't 1500 (the default).
    I wasn't able to find the source today, I'll continue to investigate next week...

    No, not AFAIK.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. resoli

    resoli Member

    Joined:
    Mar 9, 2010
    Messages:
    109
    Likes Received:
    0
    Great.
    Ok, many thanks, have a nice we.

    rob
     
  17. Antonio Blanco

    Antonio Blanco New Member

    Joined:
    Nov 24, 2017
    Messages:
    5
    Likes Received:
    0
    Thank you for the effort but kernel 4.15.17-12 didn't work for me, still the same errors and instability with intel drivers as before
    the issue doesn't present after applying the ethtool command solved in the same way as I explained before
     
  18. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,232
    Likes Received:
    169
    I found an issue which fixes my reproducer here, I reported it upstream with a patch and build a new kernel with the fix included, it was just uploaded to pvetest, you could manually download it from:

    http://download.proxmox.com/debian/...pve-kernel-4.15.17-3-pve_4.15.17-13_amd64.deb
    Code:
    # md5sum pve-kernel-4.15.17-3-pve_4.15.17-13_amd64.deb
    93ee36afcf147ea4ab9ef0d04ca7e412  pve-kernel-4.15.17-3-pve_4.15.17-13_amd64.deb
    # sha512sum pve-kernel-4.15.17-3-pve_4.15.17-13_amd64.deb
    f8766a267b66614abe35e99ebb68bff6dd3fa01b494ac1b27782029fa6c2ca59c0a75ddd6e95eb649a05ae04000fe9293d491914bac57b3301b6910160fa3c0a  pve-kernel-4.15.17-3-pve_4.15.17-13_amd64.deb
    
    Hmm, no good...
    Could you please post the output of:
    Code:
    ethtool -e eth0 length 256
    ethtool -i eth0
    
    (where 'eth0' should be replaced with a problematic e1000e NIC link name)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  19. taf_321

    taf_321 New Member

    Joined:
    Jun 18, 2018
    Messages:
    3
    Likes Received:
    1
    t.lamprecht likes this.
  20. Antonio Blanco

    Antonio Blanco New Member

    Joined:
    Nov 24, 2017
    Messages:
    5
    Likes Received:
    0
    Sure, here:
    Code:
     ethtool -e eth0 length 256
    Offset        Values
    ------        ------
    0x0000:        90 1b 0e a3 c4 5f 01 08 ff ff 84 00 8e 00 00 80
    0x0010:        ff ff ff ff c3 10 1f 12 34 17 b7 15 00 00 00 00
    0x0020:        00 00 00 00 00 80 05 a7 2c 30 00 16 00 00 00 0c
    0x0030:        f4 18 42 0a 43 08 13 01 b7 15 ad ba b7 15 b8 15
    0x0040:        ad ba b7 15 ad ba b7 15 00 00 80 80 00 4e 86 08
    0x0050:        00 00 00 00 07 00 00 20 20 00 00 00 00 0e 00 00
    0x0060:        00 01 00 40 0a 01 07 40 ff ff ff ff ff ff ff ff
    0x0070:        ff ff ff ff ff ff ff ff ff ff 00 02 ff ff ef f7
    0x0080:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0090:        00 00 00 00 00 00 ff ff ff ff ff ff ff ff ff ff
    0x00a0:        94 b0 00 08 0a 00 04 90 b0 47 40 24 c2 c1 21 fb
    0x00b0:        80 60 1f 00 00 48 10 00 40 60 1f 00 04 d1 11 00
    0x00c0:        03 0a 12 00 00 00 1f 00 04 b4 30 00 1c 00 31 00
    0x00d0:        06 b4 30 00 09 00 31 00 07 b4 30 00 10 00 31 00
    0x00e0:        0a b4 30 00 18 00 31 00 0c b4 30 00 18 00 31 00
    0x00f0:        0d b4 30 00 18 00 31 00 01 fd 30 00 2c 9c 31 00
    
    Code:
    ethtool -e eth0
    Offset        Values
    ------        ------
    0x0000:        90 1b 0e a3 c4 5f 01 08 ff ff 84 00 8e 00 00 80
    0x0010:        ff ff ff ff c3 10 1f 12 34 17 b7 15 00 00 00 00
    0x0020:        00 00 00 00 00 80 05 a7 2c 30 00 16 00 00 00 0c
    0x0030:        f4 18 42 0a 43 08 13 01 b7 15 ad ba b7 15 b8 15
    0x0040:        ad ba b7 15 ad ba b7 15 00 00 80 80 00 4e 86 08
    0x0050:        00 00 00 00 07 00 00 20 20 00 00 00 00 0e 00 00
    0x0060:        00 01 00 40 0a 01 07 40 ff ff ff ff ff ff ff ff
    0x0070:        ff ff ff ff ff ff ff ff ff ff 00 02 ff ff ef f7
    0x0080:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0090:        00 00 00 00 00 00 ff ff ff ff ff ff ff ff ff ff
    0x00a0:        94 b0 00 08 0a 00 04 90 b0 47 40 24 c2 c1 21 fb
    0x00b0:        80 60 1f 00 00 48 10 00 40 60 1f 00 04 d1 11 00
    0x00c0:        03 0a 12 00 00 00 1f 00 04 b4 30 00 1c 00 31 00
    0x00d0:        06 b4 30 00 09 00 31 00 07 b4 30 00 10 00 31 00
    0x00e0:        0a b4 30 00 18 00 31 00 0c b4 30 00 18 00 31 00
    0x00f0:        0d b4 30 00 18 00 31 00 01 fd 30 00 2c 9c 31 00
    0x0100:        4c b6 30 00 05 00 31 00 ff ff ff ff ff ff ff ff
    0x0110:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0120:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0130:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0140:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0150:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0160:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0170:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0180:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0190:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x01a0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x01b0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x01c0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x01d0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x01e0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x01f0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0200:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0210:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0220:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0230:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0240:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0250:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0260:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0270:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0280:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0290:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x02a0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x02b0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x02c0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x02d0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x02e0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x02f0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0300:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0310:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0320:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0330:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0340:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0350:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0360:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0370:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0380:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0390:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x03a0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x03b0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x03c0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x03d0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x03e0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x03f0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0400:        69 53 84 03 01 00 00 00 00 00 00 00 00 00 00 00
    0x0410:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0420:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0430:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0440:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0450:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0460:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0470:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0480:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0490:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x04a0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x04b0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x04c0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x04d0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x04e0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x04f0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0500:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0510:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0520:        00 00 00 00 00 00 00 00 03 3d 00 00 00 00 00 00
    0x0530:        00 00 00 00 00 00 00 00 00 00 00 00 bc 0c 00 00
    0x0540:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0550:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0560:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0570:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0580:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0590:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x05a0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x05b0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x05c0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x05d0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x05e0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x05f0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0600:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0610:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0620:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0630:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0640:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0650:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0660:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0670:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0680:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0690:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x06a0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x06b0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x06c0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x06d0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x06e0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x06f0:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0700:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0710:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0720:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0730:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0740:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0750:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0760:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0770:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0780:        00 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff
    0x0790:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x07a0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x07b0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x07c0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x07d0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x07e0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x07f0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0800:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0810:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0820:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0830:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0840:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0850:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0860:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0870:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0880:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0890:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x08a0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x08b0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x08c0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x08d0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x08e0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x08f0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0900:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0910:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0920:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0930:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0940:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0950:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0960:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0970:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0980:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0990:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x09a0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x09b0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x09c0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x09d0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x09e0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x09f0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a00:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a10:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a20:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a30:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a40:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a50:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a60:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a70:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a80:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0a90:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0aa0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ab0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ac0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ad0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ae0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0af0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b00:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b10:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b20:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b30:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b40:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b50:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b60:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b70:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b80:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0b90:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ba0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0bb0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0bc0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0bd0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0be0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0bf0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c00:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c10:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c20:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c30:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c40:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c50:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c60:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c70:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c80:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0c90:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ca0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0cb0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0cc0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0cd0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ce0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0cf0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d00:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d10:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d20:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d30:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d40:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d50:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d60:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d70:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d80:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0d90:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0da0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0db0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0dc0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0dd0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0de0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0df0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e00:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e10:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e20:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e30:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e40:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e50:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e60:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e70:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e80:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0e90:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ea0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0eb0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ec0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ed0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ee0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ef0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f00:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f10:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f20:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f30:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f40:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f50:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f60:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f70:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f80:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0f90:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0fa0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0fb0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0fc0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0fd0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0fe0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    0x0ff0:        ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
    
    And another server:
    Code:
    ethtool -e enp0s31f6: length 256
    Offset        Values
    ------        ------
    0x0000:        90 1b 0e bb 34 93 01 08 ff ff 84 00 8e 00 00 80
    0x0010:        ff ff ff ff c3 10 1f 12 34 17 b7 15 00 00 00 00
    0x0020:        00 00 00 00 00 80 05 a7 2c 30 00 16 00 00 00 0c
    0x0030:        f4 18 42 0a 43 08 13 01 b7 15 ad ba b7 15 b8 15
    0x0040:        ad ba b7 15 ad ba b7 15 00 00 80 80 00 4e 86 08
    0x0050:        00 00 00 00 07 00 00 20 20 00 00 00 00 0e 00 00
    0x0060:        00 01 00 40 04 01 07 40 ff ff ff ff ff ff ff ff
    0x0070:        ff ff ff ff ff ff ff ff ff ff 00 02 ff ff 85 ac
    0x0080:        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0x0090:        00 00 00 00 00 00 ff ff ff ff ff ff ff ff ff ff
    0x00a0:        94 b0 00 08 0a 00 04 90 b0 47 40 24 c2 c1 21 fb
    0x00b0:        80 60 1f 00 00 48 10 00 40 60 1f 00 04 d1 11 00
    0x00c0:        03 0a 12 00 00 00 1f 00 04 b4 30 00 1c 00 31 00
    0x00d0:        06 b4 30 00 09 00 31 00 07 b4 30 00 10 00 31 00
    0x00e0:        0a b4 30 00 18 00 31 00 0c b4 30 00 18 00 31 00
    0x00f0:        0d b4 30 00 18 00 31 00 01 fd 30 00 2c 9c 31 00
    
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice