Boot Proxmox with zfs raid1 root partition

[StaLeWaR]

Good day!
For the second day, I try to solve the problem booting.
I installed Proxmox 4.3. When installing I chose the ZFS raid1, use disk sda sdb.
The installation was successful and the system is booted.
I simulated failure of one of the disks, turning off the system and removing one disc.
But the system does not start.
Just a black screen, no flashing cursor, nothing!
I tried to repeat the operation with another drive, and also failed.

I found a link to problem
https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Grub_boot_ZFS_problem
https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Grub_boot_ZFS_problem
Hardware:
MB - SuperMicro X7DAE (I updated the BIOS to the latest version 2.1с)
HDD - WD5000AAKX


Is there any way to solve?
Any help!

 

[fireon]

I have all my private servers on Supermicro. Had some similar problem for a while. But not yours directly. I put on all my installations this do the grubline: "rootdelay=10" Maybe this will solve your issue.

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on rootdelay=10"

 

[StaLeWaR]

thanks you for the answer!
The proposed options are to download the kernel, grub stop before.
I did not even see the grub boot menu!

 

[StaLeWaR]

I added in grub.conf:

set pager=1
set debug=all

No results.
That is grub to boot not come.
I'm confused

 

[fabian]

is it possible that your bios is configured to boot only from one disk? have you tried removing the other one instead and see if it boots then? the PVE installer sets the bootloader up on all disks of the first vdev, so if you have a mirror ("RAID1"), you should be able to boot from both/all disks.

 

[StaLeWaR]

Yes, I checked the BIOS settings!
The system is loaded only if both disks are inserted
very strange behavior

I can imagine that:
Boot partition can not be mounted by the bootloader and grub did not get control.

Question - why?

 

[fabian]

Yes, I checked the BIOS settings!
The system is loaded only if both disks are inserted
very strange behavior

I can imagine that:
Boot partition can not be mounted by the bootloader and grub did not get control.

Question - why?

that sounds like a bios bug - GRUB does not care about having all vdevs, as long as there are enough to import the pool.

 

[StaLeWaR]

Maybe you're right.
Can I get debug information from the grub?
Maybe there debug version of the grub?
What would loader display all the information.

 

[fabian]

Maybe you're right.
Can I get debug information from the grub?
Maybe there debug version of the grub?
What would loader display all the information.

if grub loads at all, it should drop you into a rescue shell if it fails to load the device where grub.cfg is (the zpool in your case). you can try upgrading to the current version if you haven't already, there was a round of grub updates to a newer version recently.

 

[StaLeWaR]

grub does not show the rescue shell.
in grub.conf I added options for debugging.
in the case of both discs I see debug output, but if one disk is not I see a black screen

 

[StaLeWaR]

root@pve:~# zpool status rpool
  pool: rpool
state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda2    ONLINE       0     0     0
            sdb2    ONLINE       0     0     0

errors: No known data errors
root@pve:~# fdisk -l /dev/sda

Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B83966BB-1A46-4909-B4A9-474F5C5981B4

Device         Start       End   Sectors   Size Type
/dev/sda1         34      2047      2014  1007K BIOS boot
/dev/sda2       2048 976756749 976754702 465.8G Solaris /usr & Apple ZFS
/dev/sda9  976756750 976773134     16385     8M Solaris reserved 1

Partition 2 does not start on physical sector boundary.

Partition 10 does not start on physical sector boundary.

root@pve:~# fdisk -l /dev/sdb

Disk /dev/sdb: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: A166A6DA-8B5B-4646-AEBA-3C3C6587A512

Device         Start       End   Sectors   Size Type
/dev/sdb1         34      2047      2014  1007K BIOS boot
/dev/sdb2       2048 976756749 976754702 465.8G Solaris /usr & Apple ZFS
/dev/sdb9  976756750 976773134     16385     8M Solaris reserved 1

root@pve:~#

It's all good.

 

[StaLeWaR]

UP!