zfs installation fails on dell with perc h730

[goty]

Hello everyone!

I have tried to install proxmox with zfs on a Dell poweredge 730 with a h730 controller.

I can not install with the drives in raid mode neither with HBA configuration.

I have tried to install it with raid5, I have tried to install it in hba mode and I always have the same result:

Installing for i386-pc platform.
/usr/sbin/grub-install: error unknown filesystem.
unable to install the i386-pc boot loader on ´/dev/sda´
umount: /rpool/ROOT/pve-1: target is busy

(In some cases useful info about processes that use the device is found by lsoft(8) of fuser(1).)​

cannot unmount ´/rpool/ROOT/pve-1´: umount failed

as well I have tried with uefi bot and bios boot...

I do not have any problem installing proxmox with ext4.

is there some problem with h730 controller and zfs?

 

[bobmc]

I believe you will need to change the config on the h730 controller to non-raid mode

http://www.mrvsan.com/configuring-the-dell-perc-h730-controller-for-passthrough-and-raid/

 

[goty]

I have all the disk in no raid mode and it does not work.



I have installed it as well with raid-5 (I know it is not recommended, only for test purpose) and it does not work either. If I install it in ext4 all works correctly.

 

[bobmc]

maybe it's not possible for the controller to be bootable in zfs mode.

 

[fireon]

Any news about it?

 

[goty]

Any news about it?

I could not install proxmox with zfs. Finally I have installed proxmox in two servers with ext4 and DRBD9 storage.

 

[fireon]

I could not install proxmox with zfs. Finally I have installed proxmox in two servers with ext4 and DRBD9 storage.

Thanks for the Information.

 

[velocity08]

I could not install proxmox with zfs. Finally I have installed proxmox in two servers with ext4 and DRBD9 storage.

Hi @goty

How are you finding the DRBD9 solution?

""Cheers
G

 

[fireon]

DRBD9 is really not good document documented. You should have a look at https://www.linbit.com/drbd/

 

[velocity08]

DRBD9 is really not good document documented. You should have a look at https://www.linbit.com/drbd/

But is the solution working well?

 

[fireon]

But is the solution working well?

Since they changing whole things/licence... for us it is not a solution anymore. We are changed to ceph.

 

[velocity08]

Since they changing whole things/licence... for us it is not a solution anymore. We are changed to ceph.

ok i wasn't aware of the license changes sorry to hear that.

How is Ceph going for you?

 

[fireon]

How i should describe Ceph? ...ok... "Fire up and forget" infinitely scalable

 

[velocity08]

How i should describe Ceph? ...ok... "Fire up and forget" infinitely scalable

Are you running all SSD or Hybrid SSD with Spinning disk?

 

[fireon]

Are you running all SSD or Hybrid SSD with Spinning disk?

Yes, only SSD's. With this prices... really... don't need SAS HDD. For example here an Intel S4510. https://geizhals.de/intel-ssd-d3-s4510-960gb-ssdsc2kb960g801-a1866192.html

 

[yjjoe]

IOMMU seems to stop working with Dell r730 with PERC h730 mini in HBA mode.
I'm not using efi (tried everything findable on the net and was not working). So now I'm in bios mode using Grub.
There is an option in the r730 that is x2apic, which is enabled (kinda optimized enabled thing) and then
'dmesg | grep -e DMAR -e IOMMU' outputs:

root@stoe1:~# dmesg | grep -e DMAR -e IOMMU
[    0.008749] ACPI: DMAR 0x000000007BAFE000 0001D8 (v01 DELL   PE_SC3   00000001 DELL 00000001)
[    0.008774] ACPI: Reserving DMAR table memory at [mem 0x7bafe000-0x7bafe1d7]
[    1.111934] DMAR: Host address width 46
[    1.111935] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[    1.111941] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0466 ecap f020df
[    1.111944] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1
[    1.111947] DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap d2078c106f0466 ecap f020df
[    1.111949] DMAR: RMRR base: 0x0000007ae07000 end: 0x0000007af06fff
[    1.111950] DMAR: ATSR flags: 0x0
[    1.111952] DMAR: ATSR flags: 0x0
[    1.111955] DMAR-IR: IOAPIC id 10 under DRHD base  0xfbffc000 IOMMU 0
[    1.111957] DMAR-IR: IOAPIC id 8 under DRHD base  0xc7ffc000 IOMMU 1
[    1.111958] DMAR-IR: IOAPIC id 9 under DRHD base  0xc7ffc000 IOMMU 1
[    1.111960] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000
[    1.111961] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    1.112163] DMAR-IR: IRQ remapping was enabled on dmar0 but we are not in kdump mode
[    1.112371] DMAR-IR: IRQ remapping was enabled on dmar1 but we are not in kdump mode
[    1.112491] DMAR-IR: Enabled IRQ remapping in x2apic mode

Usual method used:

# Grub
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel=iommu=on iommu=pt"
update-grub

/etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

update-initramfs -u -k all
reboot

But here's something that sounds promising even if it does not work on my end.

# kernel fix
apt update && apt install pve-kernel-5.15

After rebooting, adding a pci throughput doesn't tell no IOMMU present anymore. Being able to add the HBA controller, I restart TrueNas, but well, system freeze, I have to hard-boot.
I've heard that the that kernel should be in release Q2 this year. In the meantime.. I didn't find a solution

 

[Neobin]

Not sure, what your problem has to do with the original post, but:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel=iommu=on iommu=pt"

has to be: intel_iommu=on

Would suggest to open a new thread for your problem, also with more informations.

 

[yjjoe]

Not sure, what your problem has to do with the original post, but:



has to be: intel_iommu=on

Would suggest to open a new thread for your problem, also with more informations.

Yeah, I've digressed a bit on my post, but you saved me. I'm so aware of the syntax but that typo in my notepad cause me days of struggling. Even after more then 20 years in IT, the biggest showstopper are always in you face. Thx mister!