Proxmox 4 Won't Boot Automatically

Discussion in 'Proxmox VE: Installation and configuration' started by NerdUno, Oct 14, 2015.

Thread Status:
Not open for further replies.
  1. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
    Not sure how to say it any more clearly. It's a bug because the Proxmox installer does not honor the BIOS setting that has disabled UEFI on the machine. Proxmox creates a GPT disk label type that only can be used on a UEFI-enabled machine. Thus, it will not boot from power on. And, no, "all known BIOS can deal with that." Intel BIOS cannot, and it makes up a lion's share of the server market. Proxmox boot fails on any server with UEFI capability that does not have a UEFI-compatible drive. RedHat, CentOS, Scientific Linux, and Ubuntu all handle this properly. I suspect Debian does as well since Ubuntu is a derivative. Proxmox does not. Simple as that.
     
  2. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    S2612-UR (*Intel* server) with S5520UR motherboard happily boots PVE4. This server has UEFI.

    Code:
    # fdisk -l /dev/sda
    
    
    Disk /dev/sda: 37.3 GiB, 40020664320 bytes, 78165360 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: D92C9D6C-42F1-4939-9779-EABA29230187
    
    
    Device      Start      End  Sectors  Size Type
    /dev/sda1      34     2047     2014 1007K BIOS boot
    /dev/sda2    2048   262143   260096  127M EFI System
    /dev/sda3  262144 78165326 77903183 37.2G Linux LVM
    
     
  3. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
    The issue is whether you have a UEFI-compatible drive. If so, it obviously will work fine. If not, it won't... even with UEFI disabled in the BIOS.
     
  4. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    What is a "UEFI-compatible" drive?
     
  5. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
  6. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    So, GPT instead of MBR, which is clearly shown in my fdisk output.
     
  7. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
    No, GPT is set by Proxmox during the install based upon the fact that your motherboard has (not uses) UEFI to control the drive. Whether the drive itself can function as a UEFI device without further tweaking in the BIOS is what makes the drive UEFI-compatible. Some can and many cannot. So it makes more sense to honor the BIOS setting as to whether to rely on UEFI or Legacy BIOS. That's why the legacy setting is in BIOS in the first place.
     
    #27 NerdUno, Oct 18, 2015
    Last edited: Oct 18, 2015
  8. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    No, you are wrong. Proxmox *always* sets the partition as GPT. This is the output on a HP Microserver Gen8 that is *NOT* UEFI:

    Code:
    root@gen8:~# fdisk -l /dev/sdf
    
    
    Disk /dev/sdf: 30.1 GiB, 32296140800 bytes, 63078400 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: A5ECD13D-5476-4CB8-B621-57E53A836AFA
    
    
    Device      Start      End  Sectors  Size Type
    /dev/sdf1      34     2047     2014 1007K BIOS boot
    /dev/sdf2    2048   262143   260096  127M EFI System
    /dev/sdf3  262144 63078366 62816223   30G Linux LVM
    
    See? The exact partitioning scheme with GPT. Both servers boot out of the box.
     
  9. pezi

    pezi New Member

    Joined:
    Aug 2, 2011
    Messages:
    28
    Likes Received:
    0
    Same layout - but my BIOS (last available update 2013) isn't happy with this layout - with UEFI support enabled, or disabled.

    PHP:
    Disk /dev/sda111.8 GiB120034123776 bytes234441648 sectors
    Units
    sectors of 1 512 512 bytes
    Sector size 
    (logical/physical): 512 bytes 512 bytes
    I
    /O size (minimum/optimal): 512 bytes 512 bytes
    Disklabel type
    gpt
    Disk identifier
    1D471F05-32DC-43B1-BE15-99AEA5C84BFB

    Device      Start       End   Sectors   Size Type
    /dev/sda1      34      2047      2014  1007K BIOS boot
    /dev/sda2    2048    262143    260096   127M EFI System
    /dev/sda3  262144 234441614 234179471 111.7G Linux LVM
     
  10. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
  11. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    From what I can see, the structure is this:

    1 - the embedded GRUB stuff
    2 - a FAT32 (EFI) partition that has grubx64.efi in it (UEFI boot should pick and execute it in UEFI mode)
    3... - OS

    To me the structure looks correct. I think the issue happens because the "EFI" partition is not the first partition on the disk. I've found something related here http://askubuntu.com/questions/6182...-must-or-should-be-first-for-some-reason-if-s and it looks like Windows is advising to be the first partition. Considering that NUC is not Linux certified they must've been followed Microsoft advice and their firmware requires the partition to be the first, therefore it is a firmware bug.

    Even more, considering that there is at least one server with UEFI firmware (ironically an Intel server, not just chipset) that boots Proxmox's structure in CSM mode, I would say that it confirms the firmware bug even more.

    LE: Yes, you are wrong. The NUC firmware is not Linux certified, therefore buggy in this scenario.
    @pezi: people had succes with downgraded UEFI firmware versions, as Google searches suggest.
     
  12. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
    The problem is Proxmox not honoring the Legacy Boot setting in the BIOS. You should never have a GPT-partitioned disk on a non-UEFI or UEFI-disabled machine, and Proxmox should honor that just as all the other operating systems do. Having spent two days on this, it doesn't matter to me personally since I (now) know how to fix the problem. But it's going to be an unwelcome surprise to those that install Proxmox only to find their shiny new server will no longer reboot because "no boot device found."
     
    #32 NerdUno, Oct 18, 2015
    Last edited: Oct 18, 2015
  13. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    16,173
    Likes Received:
    268
    Don't get me wrong, but what you claim is simply not true.
     
  14. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    If you would know how this works, you would realize that shouldn't be a problem unless the firmware is trying to "help".

    In CSM/BIOS mode, booting from a hard drive, firmware loads the code at disk offset 0 (446 bytes at most). Sometimes there is a standard code that loads the active partition boot code and executes it, sometimes it is GRUB's stage1 (when GRUB is installed in the partition table). That's the whole job of the BIOS boot sequence. Executing stage1 will load stage2 and things will get smarter and smarter, but firmware finished its job after loading and executing the MBR.

    So the whole thing that BIOS/CSM should do is to load few hundred bytes and execute. This is why it works with dumb firmware (BIOS).

    I assume that NUC (and fozi's MB) try to do something more than that and shoots itself in the head.

    When you know how simple is the BIOS booting, it is very hard to put the blame on Proxmox. They took the clever path to make the OS drive dual booting aware. When I will have the chance I will try this on a UEFI enabled Dell server and report if it works or not. Btw, this also works on an old HP DL160, non-UEFI.
     
  15. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    #35 sigxcpu, Oct 18, 2015
    Last edited: Oct 18, 2015
  16. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    16,173
    Likes Received:
    268
    Also, please make sure that you test with latest available firmware for you MB (maybe they already fixed the BUG). If it is still buggy, please report a BUG to the firmware vendor.
     
  17. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
  18. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
    Let's try this from a different angle. You said Proxmox always labels a drive as GPT whether it is or not. As you observed, this doesn't cause a problem with a legacy machine such as your HP DL160 which is non-UEFI. The reason is because that machine only knows how to boot using the MBR so it doesn't care what label appears on the drive. It could say GPT or XYZ or whatever. The machine still boots using its legacy MBR routine. Now let's consider a newer UEFI motherboard. Intel developed the UEFI specification so it's more than a little amusing that you now have labeled the Intel UEFI implementation as buggy. That's kinda like telling General Motors that they put the GM hood ornament in the wrong place. With UEFI motherboards, the GPT drive label type means something. It is a word of art. By definition, it means the UEFI machine should look in the GUID Partition Table (GPT) to find the disk drive characteristics. For a UEFI-compatible drive, the server will find the information it needs and boot from the drive. So, yes, some drives boot fine with a UEFI motherboard because the drive is recognized based upon the information in the GUID Partition Table. But many drives don't have GUID Partition Table support yet so the drive is unrecognized. The motherboard manufacturers provided a workaround. For these drives, you could turn off UEFI in the BIOS or at least turn ON Legacy Boot Mode to tell the operating system not to treat the drive as a UEFI/GPT device. With either of those settings (UEFI OFF or Legacy Boot ON) the operating system installation routine needs to honor the BIOS settings and partition the drive(s) without the GPT label because that label tells the bootloader what to do next. That's where Proxmox flunks the test while Debian, Ubuntu, and RedHat all pass. If you're going to use the GPT designation, then you have to play by the GPT rules for UEFI devices and honor the UEFI convention which says, if the Legacy Boot Mode flag is set, configure the drive for bootup using MBR as part of the install. That could be as simple as inserting fs0:\EFI\proxmox\grubx64.efi in startup.nsh or removing the GPT disk label which Proxmox incorrectly assigned to attached non-GPT drives.
     
    #38 NerdUno, Oct 18, 2015
    Last edited: Oct 18, 2015
  19. sigxcpu

    sigxcpu Member

    Joined:
    May 4, 2012
    Messages:
    433
    Likes Received:
    8
    No. UEFI mandates GPT. GPT does not mandate UEFI.

    Please read the post above where my Intel UEFI-enabled server boots the Proxmox partition. You keep ignoring that. Intel wrote the specs, Intel made the server, Intel made the firmware, Intel boots the hybrid drive without a hitch. The constant here is the drive partition format. The variable is Intel firmware (NUC vs S2612UR).
    You keep saying the same thing with different words all the time, but you fail to understand simple things:

    1. UEFI requires GPT drive. GPT drive does not require or imply UEFI
    2. there are both UEFI and non-UEFI machines that boot successfully so the drive format is not an issue
    3. there are machines that do not boot. To me it looks like the firmware is buggy, following your logic: GPT means UEFI only (which is not the case, not mandated by any spec)
    4. NUC has Windows as the only supported OS. That means that probably it makes some assumptions (see 3)

    I understand that you're upset about your specific configuration failing, but you cannot ask to change a system that works and fits most of the people to make a NUC happy. If you search about NUC boot problems with Linux you will find lots of pages and discussions about firmware updates that sometimes fixed, sometime didn't fix the issue.
    If you have a non-working configuration, you have the workaround to install Jessie and then Proxmox on top of it. That is a luxury that you don't have in other virtualization platforms.
     
  20. NerdUno

    NerdUno Member

    Joined:
    Sep 13, 2009
    Messages:
    30
    Likes Received:
    0
    1. True as long as the GPT drive on a non-UEFI platform also has a master boot record. Non-UEFI platforms know nothing about a GUID Partition Table (GPT) so a GPT-only drive would not boot without UEFI.
    2. True up to "so..." The drive format matters because the GPT disk label tells a UEFI-machine to use the GUID Partition Table (GPT). If a drive has GPT-support, then it will boot. Otherwise, it won't.
    3. The GPT drive label by definition means there is a GUID Partition Table. When the UEFI bootloader can't find one, the drive won't boot. You can trick UEFI/GPT drive into booting from MBR using grubx64.efi in startup.nsh.
    4. If you mean Windows is the only OS with Intel commercial tech support, that's true. However, Intel has documented NUC compatibility for a broad range of Linux platforms. What's your point? CentOS, Scientific Linux, and Ubuntu install and boot correctly. And those are the only ones we've tested. Sounds like Debian 7 and 8 work fine as well. If we hadn't stumbled into grubx64.efi, loading Proxmox on top of Jessie was next on our list.
     
    #40 NerdUno, Oct 18, 2015
    Last edited: Oct 18, 2015
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice