1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Proxmox installation via PXE: solution

Discussion in 'Proxmox VE: Installation and configuration' started by Krokodox, Feb 24, 2012.

  1. Krokodox

    Krokodox New Member

    Joined:
    Feb 15, 2012
    Messages:
    11
    Likes Received:
    0
    Hi there,

    I would like to contribute to Proxmox with a simple(?) how-to that makes it possible to install PVE 2.0 via PXE.



    PREREQUSITES

    The following needs to be up and running for any kind of PXE + iPXE / gPXE installation, the setup of these services are specific for your environment, make sure that they all work as expected:

    1. DHCP Server (see http://ipxe.org/howto/chainloading and http://ipxe.org/howto/dhcpd for how-to)
    2. TFTP Server, I assume that it uses the "/var/www/boot/" directory as it's root
    3. iPXE (http://ipxe.org/download)
    4. HTTP Server, I assume that it uses the "/var/www/" directory as it's root
    5. Files from the Syslinux project (http://www.syslinux.org/wiki/index.php/The_Syslinux_Project), i.e. "menu.c32" in the "/var/www/boot/" directory as well as a "deafult" menu file in the "/var/www/boot/pxelinux.cfg/" directory



    NEW INITRD with embedded ISO

    In order to have the installer find the .ISO image it needs to be embedded in the initrd.img that will be downloaded via PXE. Also, the init script in this new initrd.img needs to be modified so that it finds and mounts the embedded .ISO image instead of searching for a physical CD.

    An overview of the required steps:

    1. Extract initrd.img and linux26 from the .ISO image
    2. Extract initrd.img to a directory
    3. Copy the .ISO image to the initrd directory
    4. Modify init in the initrd directory
    5. Create a new initrd.img from the directory
    6. Copy the new initrd.img to the PXE server

    Detailed step-by-step guide:

    1. Log in to your Linux workstation, copy the Proxmox .ISO image to your working directory and rename it to "proxmox.iso" for simplicity. Extract the files "\boot\isolinux\initrd.img" and "\boot\isolinux\linux26" from the .ISO image.
    2. Rename "initrd.img" to "initrd.org.img" and unzip it:

      Code:
      # mv initrd.img initrd.org.img
      # gzip -d -S ".img" ./initrd.org.img
      Create a directory for the files in the initrd, extract initrd to this directory:

      Code:
      # mkdir initrd.tmp
      # cd initrd.tmp
      # cpio -i -d < ../initrd.org
    3. Copy the .ISO image to this directory:

      Code:
      # cp ../proxmox.iso .
    4. Modify the startup script so that it will mount the embedded .ISO image instead of searching for a physical CD.

      Code:
      # nano init
      At the end of the init script it searches for a CD/DVD reader containing a physical removeable CD disc. Modify the script so that it first checks whether there is an .ISO image ("proxmox.iso") in the root of the filesystem, and if so, mounts it. The colored + bolded text is what needs to be added between the "done" and "else" statements that are already present in the script:

      Code:
      [...]
          done
      
      [B][COLOR=#b22222]elif [ -f /proxmox.iso ]; then
          echo "found proxmox cdrom ISO image"
          echo "mounting /proxmox.iso image"
          mount -t iso9660 -o loop /proxmox.iso /mnt
          echo "mounting ISO done"[/COLOR][/B]
      
      else
      
          echo "searching for cdrom"    
      [...]
      Save the file and exit the editor, no more changes are needed.
    5. Create a new initrd.img:

      Code:
      # find . | cpio -H newc -o > ../initrd
      Zip this new initrd:

      Code:
      # cd ..
      # gzip -9 -S ".img" initrd
    6. You should now have an much enlarged "initrd.img" file containing all original files from the original "initrd.org.img" as well as the .ISO image and the modified init script. Copy it to your HTTP server together with the "linux26" file:

      Code:
      # mkdir /var/www/boot/proxmox
      # cp linux26 /var/www/boot/proxmox/
      # cp initrd.img /var/www/boot/proxmox/
      Make sure that these files can be served via HTTP, i.e. test to download from a browser!



    PXE MENU

    I assume that your TFTP server's root is "/var/www/boot/" and that it contains the "menu.c32" file from the Syslinux project. You need to have your DHCP Server setup so that it serves iPXE or gPXE to your PXE enabled host and have iPXE / gPXE read the "pxelinux.cfg/default" file from the TFTP server's root.

    Modify "/var/www/boot/pxelinux.cfg/default" so that it looks something like this:

    Code:
    UI http://<HTTP Server name or IP>/boot/menu.c32
    PROMPT 0
    DEFAULT Proxmox
    
    
    MENU TITLE ..:: My PXE Menu ::..
    MENU CLEAR
    
    
    LABEL Proxmox
        MENU LABEL Proxmox
        LINUX http://<HTTP Server name or IP>/boot/proxmox/linux26
        APPEND vga=791 video=vesafb:ywrap,mtrr ramdisk_size=16777216 
        INITRD http://<HTTP Server name or IP>/boot/proxmox/initrd.img splash=verbose 


    Now everything should be setup for a successful installation of Proxmox VE via PXE to any PXE enabled host. These changes in the init file of the initrd.img image could perhaps be implement into the distribution of Proxmox, it would certainly make it easier to setup a PXE install environment ;)



    Either way, thank you for a very, very good product. keep up the good work!
     
    #1 Krokodox, Feb 24, 2012
    Last edited: Feb 24, 2012
  2. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    14,159
    Likes Received:
    69
    OK, will include such test in the next release.
     
  3. Kjeld Flarup

    Kjeld Flarup New Member

    Joined:
    Feb 29, 2012
    Messages:
    3
    Likes Received:
    0
    I tried your instructions, but my kernel panic'ed with this message

    VFS: Cannot open root device "<NULL>" or unknown-block(8,1)
    Please append a correct "root=" boot option; here are the available partions:
    Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1)

    Is there something in your setup which I also need to know.
    I'm trying to run this on a HP Proliant Blade with 64 bit Xeon
     
  4. Krokodox

    Krokodox New Member

    Joined:
    Feb 15, 2012
    Messages:
    11
    Likes Received:
    0
    That's odd, I did not encounter anything like that. My setup is with DELL PowerEdge R200 Servers / Quadcore / 4 GB RAM / QLA2xxx FC HBA / no local drives.

    What you could try is to boot the original initrd.img extracted from the original .ISO and see if you get the same error message.

    If the original initrd.img downloads / boots OK and halts with a message stating that it can not find a CD then your PXE / iPXE / TFTP / DHCP, etc. setup is OK. If not then the problem is somewhere in your TFTP / HTTP / DHCP / iPXE / Syslinux setup. Check for more recent versions, try PXE booting something else and see that it works, etc.

    If you get the original initrd.img booting from PXE then the problem is with the way your new initrd.img was created. Try to build one without editing the init file but with the .ISO image embedded. If this boots then you can manually mount the image as per the instructions above, i.e.:

    Code:
    mount -t iso9660 -o loop /proxmox.iso /mnt
    and start the setup:

    Code:
    chroot /mnt sbin/unconfigured.sh
    Happy hunting! :)
     
  5. Maxnet

    Maxnet New Member

    Joined:
    Aug 6, 2009
    Messages:
    14
    Likes Received:
    0
    We have also experienced issues with iPXE and large image files on some systems, while it does work fine on others.
    E.g. Vmware vSphere 5 also comes with a very large ramdisk.
    When you try to pxe boot that on a problematic system, iPXE/syslinux either fails with an "out of resources" error during downloading the file (even though the system has 8 GB memory), or it crashes shortly after booting.
    Not sure if it is an actual iPXE bug, or if it's an external issue like a bug in the BIOS or the network card's firmware, but fact is that large images simply do not work properly on some systems.

    A more reliable way to install Proxmox 2 over the network, is to install Debian Squeeze first, and then install the Proxmox packages.
     
  6. HarusKG

    HarusKG New Member

    Joined:
    May 28, 2012
    Messages:
    3
    Likes Received:
    0
    Hello. I got same issue as Kjeld Flarup. I used VE ver. 2.1. I put ISO file into initrd.img by instruction and did not edit the script, tried initrd.img with iso and without it (original image) and got same error. Tried to boot by PXE VirtualBox guest and my workstation, all the same. Need some help here =)


    Best regards, Konstantin.
     
  7. Pilat

    Pilat New Member

    Joined:
    Aug 21, 2010
    Messages:
    21
    Likes Received:
    0
    Krokodox, thank you for the excellent instruction. I made ​​a small modification to it. Here are my results:

    This is instruction for modification of Debian Squeeze standard PXE network install (http://www.debian.org/releases/stable/amd64/ch04s05.html.en).

    menu.c32 from first post replaced by a standard vesamenu.c32, initrd.img and linux26 are loaded from a TFTP server.

    initrd.img and linux26 prepared as written in the first post.

    10.10.10.10 - dhcp, tftp, dns server address. They may be different, but I have everything on one server.

    10.10.1.1 - default router

    Set up a DHCP server, configure the address pool

    Code:
    root@dhcp:/srv# aptitude install dhcp3-server
    root@dhcp:/srv# nano /etc/dhcp/dhcpd.conf:
    ...
    root@dhcp:/srv# cat /etc/dhcp/dhcpd.conf:
    
    Code:
    option domain-name "local";
    option domain-name-servers 10.10.10.10;
    option subnet-mask 255.255.0.0;
    
    subnet 10.10.0.0 netmask 255.255.0.0 {
        pool {
                range 10.10.1.2 10.10.1.200;
                next-server 10.10.10.10;
                filename "pxelinux.0";
                option routers 10.10.1.1;
                option domain-name-servers 10.10.10.10;
        }
    }
    
    Set up a TFTP server.

    Read http://www.debian.org/releases/stable/amd64/ch04s02.html.ru#where-files,
    download and unpack netboot.tar.gz

    Code:
    root@dhcp:/srv# aptitude install tftpd-hpa
    root@dhcp:/srv# cd /srv/tftp
    root@dhcp:/srv/tftp# wget http://http.us.debian.org/debian/dists/squeeze/main/installer-amd64/current/images/netboot/netboot.tar.gz
    root@dhcp:/srv/tftp# tar zxf netboot.tar.gz
    
    Code:
    root@dhcp:/srv/tftp# ls -la
    total 9180
    drwxr-xr-x 3 root root    4096 May  8 21:19 .
    drwxr-xr-x 3 root root    4096 Jun  2 22:32 ..
    drwxr-xr-x 3 root root    4096 May  8 21:19 debian-installer
    -rw-r--r-- 1 root root 9366606 Jun  2 22:35 netboot.tar.gz
    lrwxrwxrwx 1 root root      33 Jun  2 22:37 pxelinux.0 -> debian-installer/amd64/pxelinux.0
    lrwxrwxrwx 1 root root      35 Jun  2 22:37 pxelinux.cfg -> debian-installer/amd64/pxelinux.cfg
    -rw-r--r-- 1 root root      69 May  8 21:19 version.info
    
    Put on the TFTP server previously created files:

    Code:
    cp initrd.img /srv/tftp/
    cp linux26 /srv/tftp/
    

    Edit /srv/tftp/debian-installer/amd64/pxelinux.cfg/default.

    This is a standard file from the distribution Debian, need to comment out the line "include debian-installer/amd64/boot-screens/menu.cfg"
    and add a boot Proxmox, as described in the first post. The only difference is that the files are taken not from the HTTP, and from a TFTP server.

    Code:
    root@dhcp:/srv/tftp# cat /srv/tftp/debian-installer/amd64/pxelinux.cfg/default
    
    Code:
    # D-I config version 2.0
    #include debian-installer/amd64/boot-screens/menu.cfg
    default debian-installer/amd64/boot-screens/vesamenu.c32
    
    PROMPT 0
    DEFAULT Proxmox
    
    MENU TITLE ..:: My PXE Menu ::..
    MENU CLEAR
    
    LABEL Proxmox
        MENU LABEL Proxmox
        LINUX /linux26
        APPEND vga=791 video=vesafb:ywrap,mtrr ramdisk_size=16777216
        INITRD /initrd.img splash=verbose
    
    Reatart DHCP server...

    Code:
    root@dhcp:/srv/tftp# /etc/init.d/isc-dhcp-server restart
    
    Code:
    Stopping ISC DHCP server: dhcpd failed!
    Starting ISC DHCP server: dhcpd.
    
    Now you can load the server to install Proxmox - it should boot from PXE
     
  8. Kjeld Flarup

    Kjeld Flarup New Member

    Joined:
    Feb 29, 2012
    Messages:
    3
    Likes Received:
    0
    got past this problem with a cfg from the ubuntu installer
    LABEL Proxmox
    MENU LABEL Proxmox
    LINUX /proxmox/linux26
    append vga=normal initrd=/proxmox/initrd.img -- quiet

    Now it aborts after
    Not really much information to go on there
     
  9. 6opo9a

    6opo9a New Member

    Joined:
    May 15, 2012
    Messages:
    9
    Likes Received:
    0
    Why do not use the original iso with the memdisk?
     
  10. Kjeld Flarup

    Kjeld Flarup New Member

    Joined:
    Feb 29, 2012
    Messages:
    3
    Likes Received:
    0
    It is a lot easier to deploy in data centers with PXE
     
  11. mrballcb

    mrballcb New Member

    Joined:
    Sep 30, 2011
    Messages:
    24
    Likes Received:
    0
    I experienced that same error this morning when I was setting up PXE boot for Proxmox 2.1. That happens because the first thing the installer tries to do is start up X, and since you've told it to go into a lower res mode than what the installer wants, it just aborts. (I agree it would be nice if it would explain that it couldn't start up X.) You need to use the correct vga and video parameters for X to be able to start. It would also help for you to include splash=verbose because it's a good indicator of things working properly when you see actual output from the kernel and the splash screen.

    I too built the new initrd.img as described in the original post with the iso included. I named it "initrd.iso.img" to make it clear to other humans who might be looking at it that it serves a different purpose from the default proxmox initrd.img (as if you couldn't tell from the much larger size). This is what my pxelinux.cfg/default looks like:
    Code:
    label proxmox
       kernel proxmox/linux26
       append initrd=proxmox/initrd.iso.img ramdisk_size=524288 vga=791 video=vesafb:ywrap,mtrr splash=verbose
    #   append initrd=proxmox/initrd.iso.img ramdisk_size=524288 proxdebug
    I'll also note that some others have been using some large numbers for ramdisk_size. That value is the number of KB, so in my example above, 524288 KB => 512 MB, which is plenty large enough for the initrd with the iso to be expanded:

    Code:
    [root@dump02 build]# du -sh initrd.tmp/
    410M    initrd.tmp/
     
  12. mrballcb

    mrballcb New Member

    Joined:
    Sep 30, 2011
    Messages:
    24
    Likes Received:
    0
    I wrote a little script to be able to do this quickly in the future. Sample output:
    Code:
    [root@dump02 /]# build_proxmox_iso_initrd.sh /download/Proxmox/
    Using proxmox-2.1.iso
    Mounting iso image
    Unmounted iso, extracting contents of initrd...
    Added iso, creating and compressing the new initrd...
    Cleaning up temp files...
    Done!  Look in /download/Proxmox/pxeboot for pxeboot files.
    And here's the script:
    Code:
    #!/bin/bash
    
    BASEDIR=${1:-./}
    pushd $BASEDIR >/dev/null
    
    [ -L "proxmox.iso" ] && rm proxmox.iso &>/dev/null
    
    for ISO in *.iso; do
        if [ "$ISO" = "*.iso" ]; then continue; fi
        if [ "$ISO" = "proxmox.iso" ]; then continue; fi
        echo "Using $ISO"
        ln -s "$ISO" proxmox.iso
    done
    if [ ! -f "proxmox.iso" ]; then
        echo "Couldn't find a proxmox iso, aborting."
        echo "Add /path/to/iso_dir to the commandline."
        exit 2
    fi
    [ -d pxeboot ] || mkdir  pxeboot
    pushd pxeboot >/dev/null
    [ -d mnt ] || mkdir  mnt
    echo "Mounting iso image"
    if ! mount -t iso9660 -o ro,loop ../proxmox.iso mnt/ ; then
        echo "Failed to mount iso image, aborting."
        exit 3
    fi
    cp  mnt/boot/isolinux/linux26 .
    rm -f  initrd.orig initrd.orig.img initrd.img
    cp  mnt/boot/isolinux/initrd.img initrd.orig.img
    if ! umount  mnt ; then
        echo "Couldn't unmount iso image, aborting."
        exit 4
    fi
    echo "Unmounted iso, extracting contents of initrd..."
    gzip -d -S ".img" ./initrd.orig.img
    rm -rf initrd.tmp
    mkdir  initrd.tmp
    pushd initrd.tmp >/dev/null
    echo "Added iso, creating and compressing the new initrd..."
    cpio -i -d < ../initrd.orig 2>/dev/null
    cp ../../proxmox.iso proxmox.iso
    (find . | cpio -H newc -o > ../initrd.iso) 2>/dev/null
    popd >/dev/null
    rm -f initrd.iso.img
    gzip -9 -S ".img" initrd.iso
    
    # Now clean up temp stuff
    echo "Cleaning up temp files..."
    rmdir  mnt
    rm -rf initrd.tmp
    rm  ./initrd.orig
    
    echo "Done!  Look in $PWD for pxeboot files."
    popd >/dev/null
    popd >/dev/null
     
  13. 6opo9a

    6opo9a New Member

    Joined:
    May 15, 2012
    Messages:
    9
    Likes Received:
    0
    I know. I meant using pxe to load pxelinux to load memdisk with original iso-installer. Or do you want to make diskless nodes?
     
  14. mrballcb

    mrballcb New Member

    Joined:
    Sep 30, 2011
    Messages:
    24
    Likes Received:
    0
    I wanted to add that I am using Dell PowerEdge R210's, but with 8 GB of RAM and local disks. The Dell machines just seem to work with little to no issues, I just make sure that the virtualization features of the CPU's are enabled in the BIOS.
     
  15. delicatepc

    delicatepc New Member

    Joined:
    Nov 4, 2011
    Messages:
    3
    Likes Received:
    0
    I used a Linux Mint Live boot CD (which I booted over PXE) to create the resulting files of the script. This method works. Tested on a virtualbox VM.

    MANY THANKS TO mrballcb for this.

    My entries in my "menu cfg":
    Code:
    LABEL -    
    MENU LABEL Linux Mint 12 Live CD
        KERNEL linux/mint12/casper/vmlinuz
        APPEND root=/dev/nfs boot=casper netboot=nfs nfsroot=10.9.9.10:/export/tftpboot/linux/mint12 quiet splash --
        INITRD /linux/mint12/casper/initrd.lz
        
    LABEL -    
    MENU LABEL ProxMox (latest)
        KERNEL linux/proxmox/linux26
        APPEND ramdisk_size=524288 vga=791 video=vesafb:ywrap,mtrr splash=verbose
        INITRD linux/proxmox/initrd.iso.img
    Would it be possible to have a section on ProxMox downloads called PXE which would be basically zips containing the two needed files (linux26 and initrd.iso.img)? Im sure the community will be happy to generate and upload them as needed. It took me 10 minutes to do this.... and redoing it will take 5.

    -dpc
     
  16. mrballcb

    mrballcb New Member

    Joined:
    Sep 30, 2011
    Messages:
    24
    Likes Received:
    0
    I just wanted to mention that the script creates a PXE bootable initrd for 2.2 as well, since I just ran it, deployed the images on our PXE servers, and tested the PXE boot. I did, however, make one small change to the script to handle multiple versions of proxmox iso that are found:

    if [ -L "proxmox.iso" ]; then
    rm -f "proxmox.iso";
    fi
    ln -s "$ISO" proxmox.iso

    Alternatively, you could have just changed the "ln -s" to "ln -sf", but that seems like the sloppy way to fix it.

    ...Todd
     
  17. 6opo9a

    6opo9a New Member

    Joined:
    May 15, 2012
    Messages:
    9
    Likes Received:
    0
    Thank you! Very useful topic!
     
  18. mrballcb

    mrballcb New Member

    Joined:
    Sep 30, 2011
    Messages:
    24
    Likes Received:
    0
    Everything written below is wrong. Ignore it. ...Todd

    I just wanted to follow up for anybody who stumbles across this thread. You do not *NEED* to run the script I posted earlier to extract the kernel/initrd and rebuild it with the iso in it. User 6opo9a's question above was such a quick reply that I read right past it. What he is implying, and I am confirming, is that it's easier to just use memdisk to boot the iso directly.

    PXE boot has a utility named memdisk which provides a virtual floppy/hd/iso mount capability. To use it, load memdisk as the kernel, and then specify the proxmox iso as the initrd (must be located in the /tftpboot file structure, you can't reference a file outside there since tftpd _normally_ chroots) in the append line. You need to also add "iso" to the end of the line so that it knows which type of file it's mounting. For reference, see the specs at http://www.syslinux.org/wiki/index.php/MEMDISK.

    This is what my pxelinux.cfg/default configuration looks like:
    Code:
    label proxmox
       kernel memdisk
       append initrd=proxmox/2.2/proxmox.iso iso
    
    label proxmox22
       kernel proxmox/2.2/linux26
       append initrd=proxmox/2.2/initrd.iso.img ramdisk_size=524288 vga=791 video=vesafb:ywrap,mtrr splash=verbose
    # Debug mode
    #   append initrd=proxmox/2.2/initrd.iso.img ramdisk_size=524288 proxdebug
    
    label proxmox21
       kernel proxmox/2.1/linux26
       append initrd=proxmox/2.1/initrd.iso.img ramdisk_size=524288 vga=791 video=vesafb:ywrap,mtrr splash=verbose
    # Debug mode
    #   append initrd=proxmox/2.1/initrd.iso.img ramdisk_size=524288 proxdebug
     
    #18 mrballcb, Mar 19, 2013
    Last edited: Mar 20, 2013
  19. 6opo9a

    6opo9a New Member

    Joined:
    May 15, 2012
    Messages:
    9
    Likes Received:
    0
    Sorry, but using the only memdisk doesn't work, because after the kernel is loaded it "forgets" about a fake cdrom. More details are here.
     
  20. mrballcb

    mrballcb New Member

    Joined:
    Sep 30, 2011
    Messages:
    24
    Likes Received:
    0
    Ugh, I hate it when I'm that wrong. I had just booted it to the cd boot screen and assumed all was well. Thanks for the correction. So you DO have to still run the script to generate the special initrd.
     

Share This Page