Opt-in Linux 6.8 Kernel for Proxmox VE 8 available on test & no-subscription

PS, i don't know if i should write at all, because as always i have no issues :-(
6.8.1-1-pve works on all my 5 Servers with different Hanrdware, without any issues.

PS: Its extremely weird, maybe im somewhat confused, but everything on my Hetzner Server with an E3-1275 v5, is like 10x faster with the 6.8 kernel.
Updating docker images and some scripts that i wrote, run like 100x faster. All VM's are like they are unleashed, its absolutely weird.
the E3-1275 v5 Server is Crap, it cannot be that fast as it is right now in theory.
Im extremely confused, need to check my other Servers speed.
Does anyone experience a similar thing, or is it some sort of imagination i have?

Cheers :)
 
Last edited:
Just ran a quick test with kernel 6.8 on my Win10 gaming VM. Finally, this again shows the same performance on this machine, as did the old 5.13 kernel. Anything since then had atrocious performance, with unbearable lags every few seconds.
So no idea what was done, but it is finally good again.
 
patch said:
Not sure what you had in mind there. Maybe
* name & MACAddress match -> proceed without user intervention
* name or MACAddress match -> user confirmation required to update non matching item noting total requiring user confirmation
* neither name or MACAddress match but outstanding interface yet to match -> user confirmation from all remaining to assign
* miss match between total number of interfaces to match -> report error

Or maybe a simplified subset of the above or something different entirely.
Click to expand...
I had just a simple checkbox in mind, that if ticked would make the installer create a systemd.link file for each interface with just enumerating some prefix, the rest then would be handled by systemd/udev – and yeah, if there's no match for an interface then it will fall back to the naming from the configured source (which is systemd's "predictable" naming scheme by default).

We might then allow overriding names in the installer in some advanced edit mode, but we'll keep it simple there as the rest of it – but having systermd.link configuration exposed via API/UI would be certainly something interesting for the long run.

But in any way a different topic than the 6.8 testing this thread here is talking about.
 
  • Like
Reactions: _gabriel
So far i've updated 5 Nodes, ranging from Consumer Hardware to Older Xeons, have to test still Genoa and Ryzen 7 5800x.
There is a huge difference in "snappiness" with 6.8 vs 6.5 Kernel. I need to do some benchmarks.
However, all 5 Hosts/Nodes are a lot snappier, like the latency gone down?

The biggest difference is in executing my own docker update shell scripts that i wrote.
The script compares the hash between the local and remote docker image and updates/downloads the newer image if neccessary.
While the download times are the same, the hash comparing, or running the scripts if no updates are available, took 2-3 days ago approximately 10 seconds, and now it takes 0.5 seconds....
Its so fast, that the serial console cant keep up.
Im not sure if thats because of lxc 6.0 or kernel 6.8 tbh, since i updated the pve host to both at same time.
And my docker instances run all inside unpriviliged lxc containers.
However on that particular host, with the LXC Docker Containers / Update scripts, i cannot boot back before tonight into 6.5 kernel to test, because its in production right now :-(

But i booted 2 nodes back to 6.5 and there is definitively a performance hit, so its definitively not an imagination that 6.8 is a lot faster.
I owe some benchmarks, so going to provide in the evening some.

PS: One node has 2 Numa nodes, whole the other 4 have only one Numa node, so it's not just Numa that is fixed since they are all faster.

PS 2: Even my internet connection at home is 5x snappier since the update to 6.8 kernel!
Im running at home an Proxmox Server, where the Opnsense-VM does the routing between vlans and internet/nat.
All webpages opens a lot snappier!
I mean its extremely unlikely that a kernel update speeds up an Freebsd vm, especially since opnsense doesn't consume here a lot CPU or has barely anything todo, but anyway, there is definitively a huge impact in browsing speed through the internet.
Tbh, it doesnt make logically any sense, but im not drunk....

EDIT:

1712569191314.png

Glad god i have a speedtest vm!!!
You see that the ID's are all continous, i just removed the cronjob on 17th February, because it didnt made sense to get always the same results...

But the speedtest from Today shows what im saying...
i never had before that latency or that speed before at home, it tops out now my 250mbit vdsl connection.
And this happened definitively after updating to 6.8!
 
Last edited:
  • Like
Reactions: vesalius
dona310 said:
Just updated and noticed that r8125-dkms drivers won't compile with the new kernel.

A fix has already been provided for ubuntu users: https://bugs.launchpad.net/ubuntu/+source/r8125/+bug/2059256
Click to expand...
I checked this out a bit more closely, and it seems that the source code of the latest upstream versions 9.013.02
has some explicit checks for the 6.8 kernel (or newer), so I'm not sure if it's best to just updating the str-compare method to make it compile is enough to actually make it fully working...
 
Last edited:
mac.linux.free said:
Yes, thank you.

bnxt_en 0000:3d:00.0 (unnamed net_device) (uninitialized): Device requests max timeout of 100 seconds, may trigger hung task watchdog
bnxt_en 0000:3d:00.0: Unable to read VPD

Apr 07 00:07:44 fbo-vmh-024 kernel: ------------[ cut here ]------------
Apr 07 00:07:44 fbo-vmh-024 kernel: UBSAN: shift-out-of-bounds in ./include/linux/log2.h:57:13
Apr 07 00:07:44 fbo-vmh-024 kernel: shift exponent 64 is too large for 64-bit type 'long unsigned int'
Apr 07 00:07:44 fbo-vmh-024 kernel: CPU: 45 PID: 1471 Comm: (udev-worker) Tainted: P O 6.8.1-1-pve #1
Apr 07 00:07:44 fbo-vmh-024 kernel: Hardware name: Supermicro Super Server/X13DEI-T, BIOS 2.1 12/13/2023
Apr 07 00:07:44 fbo-vmh-024 kernel: Call Trace:
Apr 07 00:07:44 fbo-vmh-024 kernel: <TASK>
Apr 07 00:07:44 fbo-vmh-024 kernel: dump_stack_lvl+0x48/0x70
Apr 07 00:07:44 fbo-vmh-024 kernel: dump_stack+0x10/0x20
Apr 07 00:07:44 fbo-vmh-024 kernel: __ubsan_handle_shift_out_of_bounds+0x1ac/0x360
Apr 07 00:07:44 fbo-vmh-024 kernel: bnxt_qplib_alloc_init_hwq.cold+0x8c/0xd7 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: bnxt_qplib_create_qp+0x1d5/0x8c0 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: bnxt_re_create_qp+0x71d/0xf30 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: ? bnxt_qplib_create_cq+0x247/0x330 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: ? __kmalloc+0x1ab/0x400
Apr 07 00:07:44 fbo-vmh-024 kernel: create_qp+0x17a/0x290 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: ? create_qp+0x17a/0x290 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: ib_create_qp_kernel+0x3b/0xe0 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: create_mad_qp+0x8e/0x100 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: ? __pfx_qp_event_handler+0x10/0x10 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: ib_mad_init_device+0x2c2/0x8a0 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: add_client_context+0x127/0x1c0 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: enable_device_and_get+0xe6/0x1e0 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: ib_register_device+0x506/0x610 [ib_core]
Apr 07 00:07:44 fbo-vmh-024 kernel: bnxt_re_probe+0xe7d/0x11a0 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: ? __pfx_bnxt_re_probe+0x10/0x10 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: auxiliary_bus_probe+0x3e/0xa0
Apr 07 00:07:44 fbo-vmh-024 kernel: really_probe+0x1c9/0x430
Apr 07 00:07:44 fbo-vmh-024 kernel: __driver_probe_device+0x8c/0x190
Apr 07 00:07:44 fbo-vmh-024 kernel: driver_probe_device+0x24/0xd0
Apr 07 00:07:44 fbo-vmh-024 kernel: __driver_attach+0x10b/0x210
Apr 07 00:07:44 fbo-vmh-024 kernel: ? __pfx___driver_attach+0x10/0x10
Apr 07 00:07:44 fbo-vmh-024 kernel: bus_for_each_dev+0x8a/0xf0
Apr 07 00:07:44 fbo-vmh-024 kernel: driver_attach+0x1e/0x30
Apr 07 00:07:44 fbo-vmh-024 kernel: bus_add_driver+0x156/0x260
Apr 07 00:07:44 fbo-vmh-024 kernel: driver_register+0x5e/0x130
Apr 07 00:07:44 fbo-vmh-024 kernel: __auxiliary_driver_register+0x73/0xf0
Apr 07 00:07:44 fbo-vmh-024 kernel: ? __pfx_bnxt_re_mod_init+0x10/0x10 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: bnxt_re_mod_init+0x3e/0xff0 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: ? __pfx_bnxt_re_mod_init+0x10/0x10 [bnxt_re]
Apr 07 00:07:44 fbo-vmh-024 kernel: do_one_initcall+0x5b/0x340
Apr 07 00:07:44 fbo-vmh-024 kernel: do_init_module+0x97/0x290
Apr 07 00:07:44 fbo-vmh-024 kernel: load_module+0x213a/0x22a0
Apr 07 00:07:44 fbo-vmh-024 kernel: init_module_from_file+0x96/0x100
Apr 07 00:07:44 fbo-vmh-024 kernel: ? init_module_from_file+0x96/0x100
Apr 07 00:07:44 fbo-vmh-024 kernel: idempotent_init_module+0x11c/0x2b0
Apr 07 00:07:44 fbo-vmh-024 kernel: __x64_sys_finit_module+0x64/0xd0
Apr 07 00:07:44 fbo-vmh-024 kernel: do_syscall_64+0x84/0x180
Apr 07 00:07:44 fbo-vmh-024 kernel: ? syscall_exit_to_user_mode+0x86/0x260
Apr 07 00:07:44 fbo-vmh-024 kernel: ? do_syscall_64+0x93/0x180
Apr 07 00:07:44 fbo-vmh-024 kernel: entry_SYSCALL_64_after_hwframe+0x6e/0x76
Apr 07 00:07:44 fbo-vmh-024 kernel: RIP: 0033:0x7ac146137719
Apr 07 00:07:44 fbo-vmh-024 kernel: Code: 08 89 e8 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 f>
Apr 07 00:07:44 fbo-vmh-024 kernel: RSP: 002b:00007ffc8a83b208 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
Apr 07 00:07:44 fbo-vmh-024 kernel: RAX: ffffffffffffffda RBX: 00005f4b75018a80 RCX: 00007ac146137719
Apr 07 00:07:44 fbo-vmh-024 kernel: RDX: 0000000000000000 RSI: 00007ac1462caefd RDI: 000000000000000f
Apr 07 00:07:44 fbo-vmh-024 kernel: RBP: 00007ac1462caefd R08: 0000000000000000 R09: 00005f4b74fd8720
Apr 07 00:07:44 fbo-vmh-024 kernel: R10: 000000000000000f R11: 0000000000000246 R12: 0000000000020000
Apr 07 00:07:44 fbo-vmh-024 kernel: R13: 0000000000000000 R14: 00005f4b7500f170 R15: 00005f4b74858ec1
Apr 07 00:07:44 fbo-vmh-024 kernel: </TASK>
Apr 07 00:07:44 fbo-vmh-024 kernel: ---[ end trace ]---

Apr 07 00:08:45 fbo-vmh-024 systemd-udevd[1463]: bnxt_en.rdma.0: Worker [1642] processing SEQNUM=18223 is taking a long time
Apr 07 00:08:45 fbo-vmh-024 systemd-udevd[1463]: bnxt_en.rdma.1: Worker [1471] processing SEQNUM=18226 is taking a long time
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.0: QPLIB: bnxt_re_is_fw_stalled: FW STALL Detected. cmdq[0xe]=0x3 waited (102422 > 100000) msec active 1
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.0 bnxt_re0: Failed to modify HW QP
Apr 07 00:09:26 fbo-vmh-024 kernel: infiniband bnxt_re0: Couldn't change QP1 state to INIT: -110
Apr 07 00:09:26 fbo-vmh-024 kernel: infiniband bnxt_re0: Couldn't start port
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.0 bnxt_re0: Failed to destroy HW QP
Apr 07 00:09:26 fbo-vmh-024 kernel: ------------[ cut here ]------------
Apr 07 00:09:26 fbo-vmh-024 kernel: WARNING: CPU: 11 PID: 1471 at drivers/infiniband/core/cq.c:322 ib_free_cq+0x109/0x150 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: Modules linked in: ipmi_ssif intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs i10nm_edac nfit x86_pk>
Apr 07 00:09:26 fbo-vmh-024 kernel: nvme_auth i2c_i801 spi_intel_pci megaraid_sas xhci_hcd libahci i2c_smbus spi_intel i2c_ismt wmi pinctrl_emmitsburg
Apr 07 00:09:26 fbo-vmh-024 kernel: CPU: 11 PID: 1471 Comm: (udev-worker) Tainted: P O 6.8.1-1-pve #1
Apr 07 00:09:26 fbo-vmh-024 kernel: Hardware name: Supermicro Super Server/X13DEI-T, BIOS 2.1 12/13/2023
Apr 07 00:09:26 fbo-vmh-024 kernel: RIP: 0010:ib_free_cq+0x109/0x150 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: Code: e8 fc 9c 02 00 65 ff 0d 9d 87 e5 3e 0f 85 70 ff ff ff 0f 1f 44 00 00 e9 66 ff ff ff 48 8d 7f 50 e8 0c 3a 33 df e9 35 ff ff ff <0f> 0b 31 c0 3>
Apr 07 00:09:26 fbo-vmh-024 kernel: RSP: 0018:ff6fb876ceb3b6f0 EFLAGS: 00010202
Apr 07 00:09:26 fbo-vmh-024 kernel: RAX: 0000000000000002 RBX: 0000000000000001 RCX: 0000000000000000
Apr 07 00:09:26 fbo-vmh-024 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ff4118c220ef4400
Apr 07 00:09:26 fbo-vmh-024 kernel: RBP: ff6fb876ceb3b760 R08: 0000000000000000 R09: 0000000000000000
Apr 07 00:09:26 fbo-vmh-024 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ff4118c235c00000
Apr 07 00:09:26 fbo-vmh-024 kernel: R13: ff4118c209bb8500 R14: 00000000ffffff92 R15: ff4118c22e88f000
Apr 07 00:09:26 fbo-vmh-024 kernel: FS: 00007ac145a2a8c0(0000) GS:ff4118e0ff780000(0000) knlGS:0000000000000000
Apr 07 00:09:26 fbo-vmh-024 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 07 00:09:26 fbo-vmh-024 kernel: CR2: 00005f4b7509f1e8 CR3: 0000000131da2003 CR4: 0000000000f71ef0
Apr 07 00:09:26 fbo-vmh-024 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 07 00:09:26 fbo-vmh-024 kernel: DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
Apr 07 00:09:26 fbo-vmh-024 kernel: PKRU: 55555554
Apr 07 00:09:26 fbo-vmh-024 kernel: Call Trace:
Apr 07 00:09:26 fbo-vmh-024 kernel: <TASK>
Apr 07 00:09:26 fbo-vmh-024 kernel: ? show_regs+0x6d/0x80
Apr 07 00:09:26 fbo-vmh-024 kernel: ? __warn+0x89/0x160
Apr 07 00:09:26 fbo-vmh-024 kernel: ? ib_free_cq+0x109/0x150 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: ? report_bug+0x17e/0x1b0
Apr 07 00:09:26 fbo-vmh-024 kernel: ? handle_bug+0x46/0x90
Apr 07 00:09:26 fbo-vmh-024 kernel: ? exc_invalid_op+0x18/0x80
Apr 07 00:09:26 fbo-vmh-024 kernel: ? asm_exc_invalid_op+0x1b/0x20
Apr 07 00:09:26 fbo-vmh-024 kernel: ? ib_free_cq+0x109/0x150 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: ? ib_mad_init_device+0x54c/0x8a0 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: add_client_context+0x127/0x1c0 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: enable_device_and_get+0xe6/0x1e0 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: ? ib_mad_init_device+0x54c/0x8a0 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: add_client_context+0x127/0x1c0 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: enable_device_and_get+0xe6/0x1e0 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: ib_register_device+0x506/0x610 [ib_core]
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_re_probe+0xe7d/0x11a0 [bnxt_re]
Apr 07 00:09:26 fbo-vmh-024 kernel: ? __pfx_bnxt_re_probe+0x10/0x10 [bnxt_re]
Apr 07 00:09:26 fbo-vmh-024 kernel: auxiliary_bus_probe+0x3e/0xa0
Apr 07 00:09:26 fbo-vmh-024 kernel: really_probe+0x1c9/0x430
Apr 07 00:09:26 fbo-vmh-024 kernel: __driver_probe_device+0x8c/0x190
Apr 07 00:09:26 fbo-vmh-024 kernel: driver_probe_device+0x24/0xd0
Apr 07 00:09:26 fbo-vmh-024 kernel: __driver_attach+0x10b/0x210
Apr 07 00:09:26 fbo-vmh-024 kernel: ? __pfx___driver_attach+0x10/0x10
Apr 07 00:09:26 fbo-vmh-024 kernel: bus_for_each_dev+0x8a/0xf0
Apr 07 00:09:26 fbo-vmh-024 kernel: driver_attach+0x1e/0x30
Apr 07 00:09:26 fbo-vmh-024 kernel: bus_add_driver+0x156/0x260
Apr 07 00:09:26 fbo-vmh-024 kernel: driver_register+0x5e/0x130
Apr 07 00:09:26 fbo-vmh-024 kernel: __auxiliary_driver_register+0x73/0xf0
Apr 07 00:09:26 fbo-vmh-024 kernel: ? __pfx_bnxt_re_mod_init+0x10/0x10 [bnxt_re]
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_re_mod_init+0x3e/0xff0 [bnxt_re]
Apr 07 00:09:26 fbo-vmh-024 kernel: ? __pfx_bnxt_re_mod_init+0x10/0x10 [bnxt_re]
Apr 07 00:09:26 fbo-vmh-024 kernel: do_one_initcall+0x5b/0x340
Apr 07 00:09:26 fbo-vmh-024 kernel: do_init_module+0x97/0x290
Apr 07 00:09:26 fbo-vmh-024 kernel: load_module+0x213a/0x22a0
Apr 07 00:09:26 fbo-vmh-024 kernel: init_module_from_file+0x96/0x100
Apr 07 00:09:26 fbo-vmh-024 kernel: ? init_module_from_file+0x96/0x100
Apr 07 00:09:26 fbo-vmh-024 kernel: idempotent_init_module+0x11c/0x2b0
Apr 07 00:09:26 fbo-vmh-024 kernel: __x64_sys_finit_module+0x64/0xd0
Apr 07 00:09:26 fbo-vmh-024 kernel: do_syscall_64+0x84/0x180
Apr 07 00:09:26 fbo-vmh-024 kernel: ? syscall_exit_to_user_mode+0x86/0x260
Apr 07 00:09:26 fbo-vmh-024 kernel: ? do_syscall_64+0x93/0x180
Apr 07 00:09:26 fbo-vmh-024 kernel: ? exc_page_fault+0x94/0x1b0
Apr 07 00:09:26 fbo-vmh-024 kernel: entry_SYSCALL_64_after_hwframe+0x6e/0x76
Apr 07 00:09:26 fbo-vmh-024 kernel: RIP: 0033:0x7ac146137719
Apr 07 00:09:26 fbo-vmh-024 kernel: Code: 08 89 e8 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 f>
Apr 07 00:09:26 fbo-vmh-024 kernel: RSP: 002b:00007ffc8a83b208 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
Apr 07 00:09:26 fbo-vmh-024 kernel: RAX: ffffffffffffffda RBX: 00005f4b75018a80 RCX: 00007ac146137719
Apr 07 00:09:26 fbo-vmh-024 kernel: RDX: 0000000000000000 RSI: 00007ac1462caefd RDI: 000000000000000f
Apr 07 00:09:26 fbo-vmh-024 kernel: RBP: 00007ac1462caefd R08: 0000000000000000 R09: 00005f4b74fd8720
Apr 07 00:09:26 fbo-vmh-024 kernel: R10: 000000000000000f R11: 0000000000000246 R12: 0000000000020000
Apr 07 00:09:26 fbo-vmh-024 kernel: R13: 0000000000000000 R14: 00005f4b7500f170 R15: 00005f4b74858ec1
Apr 07 00:09:26 fbo-vmh-024 kernel: </TASK>
Apr 07 00:09:26 fbo-vmh-024 kernel: ---[ end trace 0000000000000000 ]---
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.0 bnxt_re0: Free MW failed: 0xffffff92
Apr 07 00:09:26 fbo-vmh-024 kernel: infiniband bnxt_re0: Couldn't open port 1


Apr 07 00:11:09 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.1: QPLIB: bnxt_re_is_fw_stalled: FW STALL Detected. cmdq[0xe]=0x3 waited (102345 > 100000) msec active 1
Apr 07 00:11:09 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.1 bnxt_re1: Failed to modify HW QP
Apr 07 00:11:09 fbo-vmh-024 kernel: infiniband bnxt_re1: Couldn't change QP1 state to INIT: -110
Apr 07 00:11:09 fbo-vmh-024 kernel: infiniband bnxt_re1: Couldn't start port
Apr 07 00:11:09 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.1 bnxt_re1: Failed to destroy HW QP
Apr 07 00:11:09 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.1 bnxt_re1: Free MW failed: 0xffffff92
Apr 07 00:11:09 fbo-vmh-024 kernel: infiniband bnxt_re1: Couldn't open port 1
Click to expand...
Auch mit Kernel 6.5 Probleme


[Mon Apr 8 10:32:27 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:32:27 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:32:30 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Up, 10000 Mbps full duplex, Flow control: none
[Mon Apr 8 10:32:30 2024] bnxt_en 0000:3d:00.0 eno1np0: EEE is not active
[Mon Apr 8 10:32:30 2024] bnxt_en 0000:3d:00.0 eno1np0: FEC autoneg off encoding: None
[Mon Apr 8 10:32:31 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Down
[Mon Apr 8 10:32:31 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:32:31 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:32:33 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Up, 10000 Mbps full duplex, Flow control: none
[Mon Apr 8 10:32:33 2024] bnxt_en 0000:3d:00.0 eno1np0: EEE is not active
[Mon Apr 8 10:32:33 2024] bnxt_en 0000:3d:00.0 eno1np0: FEC autoneg off encoding: None
[Mon Apr 8 10:32:35 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Down
[Mon Apr 8 10:32:35 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:32:35 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:32:38 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Up, 10000 Mbps full duplex, Flow control: none
[Mon Apr 8 10:32:38 2024] bnxt_en 0000:3d:00.0 eno1np0: EEE is not active
[Mon Apr 8 10:32:38 2024] bnxt_en 0000:3d:00.0 eno1np0: FEC autoneg off encoding: None
[Mon Apr 8 10:37:50 2024] perf: interrupt took too long (3246 > 3238), lowering kernel.perf_event_max_sample_rate to 61500
[Mon Apr 8 10:41:50 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Down
[Mon Apr 8 10:41:50 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:41:50 2024] eno1np0 speed is unknown, defaulting to 1000
[Mon Apr 8 10:41:52 2024] bnxt_en 0000:3d:00.0 eno1np0: NIC Link is Up, 10000 Mbps full duplex, Flow control: none
[Mon Apr 8 10:41:52 2024] bnxt_en 0000:3d:00.0 eno1np0: EEE is not active
[Mon Apr 8 10:41:52 2024] bnxt_en 0000:3d:00.0 eno1np0: FEC autoneg off encoding: None
 
mac.linux.free said:
Apr 07 00:09:26 fbo-vmh-024 kernel: bnxt_en 0000:3d:00.0: QPLIB: bnxt_re_is_fw_stalled: FW STALL Detected. cmdq[0xe]=0x3 waited (102422 > 100000)
Click to expand...
from a quick search (don't have this card lying around) - there might be a new firmware available (based on the version printed in the lshw output you shared above...
https://www.broadcom.com/products/ethernet-connectivity/network-adapters/bcm57416-1gbase-t-ic
https://www.broadcom.com/products/ethernet-connectivity/network-adapters/n210tp

maybe updating the firmware to the latest version provided by the manufacturer could help here
 
Tom1972 said:
network did not came up after kernel update, due to interface name change. My 10G Intel NIC's suffered a name change,
from enp2s0f0 to enp2s0f0np0, had to adapt /etc/network/interfaces accordingly.

Code: 
[root@pve-mini ~]$ ethtool -i enp2s0f0np0
driver: i40e
version: 6.8.1-1-pve
firmware-version: 9.20 0x8000d8c5 0.0.0
expansion-rom-version:
bus-info: 0000:02:00.0
Click to expand...
Is there anyway to prevent this from happing when doing major updates? It is a bit of a pain point.

Mat
 
mat.murdock said:
Is there anyway to prevent this from happing when doing major updates? It is a bit of a pain point.
Click to expand...
Yes, see my reply here:
t.lamprecht said:
One thing to avoid such changes is to pin the name of the interfaces manually. E.g., one could name a network interface net0 through matching their MAC address in a /etc/systemd/network/00-net0.link configuration like:
Code: 
[Match]
MACAddress=aa:bb:cc:12:34:56
[Link]
Name=net0
Click to expand...
 
This is not something Proxmox can fix, but please be aware that vendor-reset is currently not compatible with kernel 6.8: https://github.com/gnif/vendor-reset/pull/77
PCIe passthrough of my AMD Radeon GPUs (those that don't need vendor-reset), Ryzen USB controllers, audio controller and a SATA controller for the optical drive appear to (still) work fine. No regressions at this time.

EDIT: vendor-reset has been updated and compiles and installs fine on the Proxmox kernel 6.8 now.
 
Last edited:
are some prerequisites required for pve 8 version ?
because here vanilla version 8.0 ( June 2023 iso) , zfs + systemd-boot, kernel 6.8 install without error, but systemd-boot don't use it.
once upgraded to 8.1 , kernel 6.8 boot.
 
Last edited:
Just installed 6.8 on one of my Fujitsu D756/E85+.
It hangs at the boot screen.

PVE is installed on LVM/ext4, booting with EFI. Ceph and the Intel CPU microcode package are installed. The "quiet" parameter was deleted from boot parameters.

If I boot with the advanced options and choose one of the 6.8.1-1 options, the system boots normally as it should.
Any idea how to debug this?

1712615561637.png
 
This is exciting. Lots of cool stuff in 6.8. :)

I had a question about iGPU support in this kernel. I've read the kernel docs, but I'm a bit confused and wanted to ask here since I'm hoping to be able to use it with SR-IOV.

I'm running an Alder Lake i5-12500T in a mini PC. My initial understanding was that kernel 6.8 included new "Xe" kernel drivers for the Alder Lake iGPU that replaced the older i915 drivers. At first, I thought that would enable SR-IOV GPU passthrough on my machine, but it looks like they're still in the early testing phase. https://www.phoronix.com/news/Intel-Xe-DRM-Linux-6.9-Pull

I wanted to clarify: Will SR-IOV support for Alder Lake iGPUs not be available until the 6.9 kernel is available for Proxmox? It sounds like even then the kernel will default to i915 and I'll have to force it to load the newer driver.

I've got some VMs I need to use a remote desktop on that need 2D GPU acceleration for things like software defined radio displays. SR-IOV-based passthrough would be perfect. I'm excited to see it closer than ever. :)
 
with kernel 6.8, i having problem to run a grafana container. error code below, not sure if this related to change of apparmor, any suggestion?

Code: 
Starting btb_grafana_dashboard ... error

ERROR: for btb_grafana_dashboard  Cannot start service btb_grafana_dashboard: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error jailing process inside rootfs: pivot_root .: permission denied: unknown

ERROR: for btb_grafana_dashboard  Cannot start service btb_grafana_dashboard: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error jailing process inside rootfs: pivot_root .: permission denied: unknown
ERROR: Encountered errors while bringing up the project.
 
Ramalama said:
PS: Its extremely weird, maybe im somewhat confused, but everything on my Hetzner Server with an E3-1275 v5, is like 10x faster with the 6.8 kernel.
Updating docker images and some scripts that i wrote, run like 100x faster.
Click to expand...
Can't wait!
 
I've updated in the last few days more than 20 nodes. Except for two Thinkserver sr530 and systemd idiotic interface change(eno1 to eno1np2???)
 
ness1602 said:
I've updated in the last few days more than 20 nodes. Except for two Thinkserver sr530 and systemd idiotic interface change(eno1 to eno1np2???)
Click to expand...
I think tbh, that this change isn't that bad. Sure it nerves but it takes 2 seconds to fix /etc/network/interfaces over bmc.
However, the reason why i think that it isn't that bad, if im not wrong, you probably see now directly that your NIC supports some sort of SR-IOV.
Or some sort of special virtualization support because of the np2 addition.
So you don't have to dig if you have a new server, you see directly on the nics name if its sr-iov capable. Sure you can grep with lspci -vvv the model and google... But yeah.
 
Well, It still doesn'y fix my USB3 external 4TB drive in NTFS preformance. For some reason, any kernel higher than 6.2.16-20-pve, and I can't transfer files over SMB from my container. I have it bind mounted. It works perfectly on 6.2.16-20-pve, but anything higher, it will transfer for a little bit, maybe a gig, then it will just stop. So I have my Proxmox pinned at 6.2.16-20-pve.

My only though is that there is some incompatibility with the NTFS? I have not tried to transfer all files off of it, then reformating it to a different filesystem to see if that is the issue.

My fstab entry:
Code: 
UUID=88726111726XXXXX    /mnt/Storage    ntfs-3g    umask=000,allow_other    0    0


From lspci:
Code: 
00:14.0 0c03: 8086:9c31 (rev 04) (prog-if 30 [XHCI])
    Subsystem: 8086:7270
    Flags: bus master, medium devsel, latency 0, IRQ 52, IOMMU group 3
    Memory at f7d00000 (64-bit, non-prefetchable) [size=64K]
    Capabilities: [70] Power Management version 2
    Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci

Here is the lsusb -vv of the Drive:
Code: 
Bus 003 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            0
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         9
  idVendor           0x174c ASMedia Technology Inc.
  idProduct          0x55aa ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
  bcdDevice            1.00
  iManufacturer           2 FD
  iProduct                3 Fantom
  iSerial                 1 XXXXXXXXXXXX
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0079
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xc0
      Self Powered
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           4
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     98
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15
        MaxStreams             32
        Data-in pipe (0x03)
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15
        MaxStreams             32
        Data-out pipe (0x04)
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15
        MaxStreams             32
        Status pipe (0x02)
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x04  EP 4 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst               0
        Command pipe (0x01)
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength       0x0016
  bNumDeviceCaps          2
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x0000f41e
      BESL Link Power Management (LPM) Supported
    BESL value     1024 us
    Deep BESL value    61440 us
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000e
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat          10 micro seconds
    bU2DevExitLat        2047 micro seconds
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x000d
  Self Powered
  U1 Enabled
  U2 Enabled

SMART info:
Code: 
~# smartctl --all /dev/sdc
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.2.16-20-pve] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     HGST Ultrastar 7K6000
Device Model:     HGST HUS726040ALN610
Serial Number:    XXXXXXXX
LU WWN Device Id: 5 000cca 269ec608d
Firmware Version: APGNTD05
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Size:      4096 bytes logical/physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5319
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Wed Apr 10 04:11:14 2024 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82)    Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:         (  113) seconds.
Offline data collection
capabilities:              (0x5b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    No Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 571) minutes.
SCT capabilities:            (0x003d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   136   136   054    Pre-fail  Offline      -       108
  3 Spin_Up_Time            0x0007   133   133   024    Pre-fail  Always       -       402 (Average 402)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       2269
  5 Reallocated_Sector_Ct   0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   067    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   140   140   020    Pre-fail  Offline      -       15
  9 Power_On_Hours          0x0012   096   096   000    Old_age   Always       -       30385
 10 Spin_Retry_Count        0x0013   100   100   060    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       56
192 Power-Off_Retract_Count 0x0032   099   099   000    Old_age   Always       -       2286
193 Load_Cycle_Count        0x0012   099   099   000    Old_age   Always       -       2286
194 Temperature_Celsius     0x0002   139   139   000    Old_age   Always       -       43 (Min/Max 14/55)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   200   200   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%       490         -
# 2  Short offline       Completed without error       00%       468         -
# 3  Short offline       Completed without error       00%       444         -
# 4  Short offline       Completed without error       00%       420         -
# 5  Short offline       Completed without error       00%       396         -
# 6  Short offline       Completed without error       00%       372         -
# 7  Short offline       Completed without error       00%       348         -
# 8  Short offline       Completed without error       00%       324         -
# 9  Short offline       Completed without error       00%       300         -
#10  Short offline       Completed without error       00%       276         -
#11  Short offline       Completed without error       00%       252         -
#12  Short offline       Completed without error       00%       228         -
#13  Short offline       Aborted by host               10%       193         -
#14  Short offline       Aborted by host               10%       169         -
#15  Short offline       Aborted by host               10%       145         -
#16  Short offline       Aborted by host               10%       121         -
#17  Short offline       Aborted by host               10%        97         -
#18  Short offline       Aborted by host               10%        73         -
#19  Short offline       Aborted by host               10%        49         -
#20  Short offline       Aborted by host               10%        25         -
#21  Short offline       Aborted by host               10%         2         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
 
You must log in or register to reply here.
Top Bottom