Recent content by crabbypup

I do prefer zfs over mdraid, but I don't believe the bug linked is actually an issue most users are likely to run into. I've run many mdraid arrays in many systems without issue, some of them even qemu-kvm hosts. This specific edge case seems to be easily solvable by one or more of: 1) use...

Hardware raid is basically dead on Linux in a wold where zfs and mdadm exist, they are many times more reliable and faster than hardware raid, which is why almost nobody running proxmox is going to be using a raid card with a battery backed cache.

My point was, you only have to pass through one device. if you're passing through two, then you are probably also passing through the controller your boot drives are connected to. my system is a supermico X10DRI-F board, with 2x e5-2680 v4 cpus 128gb of ram Mellanox cx3 dual port card...

The 9207-8E is only a single controller card, with two ports exposing 4 lanes of SAS on each connector. The 9206-16e (which I have in my system at the moment) is basically two 9207-8E cards on the same board with a pcie switch.

That card should be fine. Are you certain that you've selected the card properly? And that the boot devices are definitely not attached to that device? Try booting the system from a live debian or Ubuntu USB drive.

Yes, that's correct. If you're doing pcie passthrough for the controller to truenas, you'll want to make sure the correct controller is being claimed by the vfio stub driver during boot, since the OS usually doesn't like it when the device it's booting off of disappears suddenly. That's out of...

Have you tried the documented fix that's shared in this thread?

Excellent! To summarize for anyone else, yes it's the exact same issue. It appears to be specifically related to the use of external SAS enclosures with certain controllers, of which the 2008 is one. The edge kernel doesn't appear to be necessary to resolve the issue - it's enough to add that...

Can you post your dmesg log segments - dmesg | grep mpt3 to verify it's the same issue? Can you try booting with the option mpt3sas.max_queue_depth=8000 appended to your kernel boot line and see if the issue persists? Lastly, have you tried the edge kernel?

If you get a chance to try 5.13 again, it would be good to see what the output is.

Which kernel are you running?

I'm unfortunately not familiar with how system-boot is configured. Did you recall having to change any options to enable passthrough for your system? If you run dmesg | grep mpt3 right now, what does that return?

You'd have to remove the vfio options in /etc/modprobe.d/ to make sure proxmox tried to claim the device. And un-blacklist the mpt3sas module if you've blacklisted it. You may have to also edit your grub boot line depending on how you've configured it. But your iommu conf can stay as is

Hey, can you post the relevant parts of your dmesg output (with passthrough disabled for the card)? It sounds like your issue might be slightly different if the edge kernel isn't helping, but might be related.

For comparison, the boot log with the edge kernel: ### Edge kernel # journalctl -b 10e72e848a5c4a9fb2abdbd5901e353a | egrep -e '(Linux version|sas|SAS|mptctl)' Aug 14 17:16:58 Atlas kernel: Linux version 5.13.9-1-edge (github@pve-bullseye) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU...