Search results

  1. A

    Crashed VM Too many open files

    I will give that a try. Do you guys plan on applying the patch?
  2. A

    Crashed VM Too many open files

    This is a new one for us. Jan 23 15:29:39 QEMU[284972]: kvm: virtio_bus_set_host_notifier: unable to init event notifier: Too many open files (-24) Jan 23 15:29:39 QEMU[284972]: virtio-blk failed to set host notifier (-24) Jan 23 15:29:39 QEMU[284972]: kvm: virtio_bus_start_ioeventfd...
  3. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Another update. Hit lockups last night again, except this time the host had some interesting lines in the logs. These 3 lines took place right before the VM hit all kinds of kernel panics. [Thu Nov 9 02:47:05 2023] workqueue: blk_mq_run_work_fn hogged CPU for >10000us 4 times, consider...
  4. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Here is some more interesting stuff. This has been a issue since the 6.x kernel hit the streets for proxmox. Host: HP DL 560 Gen10 Storage: iSCSI Alletra 6000 NVMe I have 2 very active CentOS7 VM's running httpd/java/tomcat, typically seeing 2k-3k httpd sessions at any given time. As...
  5. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Unfortunately still getting soft lockups even with the latest microcode. Doesn't seem to matter what way I have the VM configured.
  6. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    We hit soft lockups again with the VM set to 192 cores and our compiles set to use 128 of those cores (NUMA on and CPU hotplug disabled). No benchmark's, these are production, we don't have time to do that kind of stuff. Hence the reason we pay for enterprise repo's. Just updated the...
  7. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Good questions. Just checked Supermicro's site and the host is on the latest Bios. They don't seem to offer any microcode updates as of yet for this model. Doing some testing without CPU hotplug enabled and I found the following. CPU Hotplug Enabled - VM Boots aok with all the cores from the...
  8. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Heavy compile load. Cc1plus is the vast majority of it. I will do some testing shortly and report back.
  9. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Its a Debian 12 VM. root@progmaindeb:~# uname -r 6.1.0-13-amd64
  10. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Running on the 4th Gen Intel with any of the 6.5.x kernels results in the following within a matter of an hour or so. I can drop core counts, memory, enable numa etc, none of it matters and we still hit CPU lockups. Move the VM back to a 5.15.x kernel and the VM is rock solid. Move the VM...
  11. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Appreciate the input, but thats not new hardware. That is 3rd gen intel. We have alot of 3rd gen's with 0 issues.
  12. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    The use cases are large VM's, from LAMP to very proprietary databases. Numa on or off doesn't matter, it just happened to be the option that was left after testing. We typically have numa on, but we wanted to try every option. Like I said, this is a Intel XEON 4th Gen CPU issue. I can't...
  13. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Also noticing this being repeated in the logs on any host I move to the 6.5.x kernel. [Fri Nov 3 06:51:50 2023] bpfilter: read fail 0 [Fri Nov 3 06:51:50 2023] bpfilter: Loaded bpfilter_umh pid 72492 [Fri Nov 3 06:51:50 2023] bpfilter: read fail 0 [Fri Nov 3 06:51:50 2023] bpfilter: Loaded...
  14. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    I do appreciate the information, but this has nothing to do with that. This is a Intel Gen 4 XEON issue. We have Gen2 and Gen3 quad socket setups running 6.x.x kernel with no performance issues.
  15. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    We have been doing this for a decade with no issues, its not worth the debate.
  16. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    I have all kinds of hardware that I test on. For the KSM issue I have been testing on a HP DL 380 Gen9. That is dual socket. For the performance issues on the latest 4th Gen Intel CPU's I am testing on a newer Supermicro chassis. Its a quad socket...
  17. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    So far its also looking like KSM is still completely broke in 6.5. All of this is such a bummer for real enterprise environments. EDIT: Might have spoke to soon. We will see were this goes within the next 45 minutes. Went from 0 to 981MB in a matter of a minute or so. EDIT #2...
  18. A

    Opt-in Linux 6.5 Kernel with ZFS 2.2 for Proxmox VE 8 available on test & no-subscription

    Here is the VM config. Its a Debian 12 VM. root@ccsprogmiscrit1:~# cat /etc/pve/nodes/ccsprogmiscrit1/qemu-server/154.conf agent: 1 bios: ovmf boot: order=scsi0;net0 cores: 62 cpu: host,flags=+pdpe1gb efidisk0: MissionCrit-Alletra:vm-154-disk-0,efitype=4m,pre-enrolled-keys=1,size=528K...