Proxmox on a diskless server

[Diabolik67]

Hello everyone,
I'm new to Proxmox and I installed a 3 nodes cluster with the latest version (6.1-8) on my former ESXi servers (Dell PowerEdge R610).
As they have neither hd nor conttroller, but only SD card reader, I installed Proxmox on new 32 GB SD. Datastores are provided by iSCSI SAN.
Now I'm experiencing very slow performance on the first VMs (Centos 7 and 8) created on this cluster, I must even wait several seconds after writing username and password, and I read on other topics that this mtght be due to slow disks on Proxmox hosts. Here is my pveperf result:

root@esxesa-5:~# pveperf
CPU BOGOMIPS: 134048.40
REGEX/SECOND: 1939069
HD SIZE: 6.83 GB (/dev/mapper/pve-root)
BUFFERED READS: 12.65 MB/sec
AVERAGE SEEK TIME: 1.54 ms
FSYNCS/SECOND: 84.79
DNS EXT: 6.75 ms
DNS INT: 0.81 ms (itndc.int.ingv.it)

Is it possible to prevent Proxmox from writing on local disks, in order not to suffer from slow devices ? Let's say similarly to vSphere, which works without any issue when installed on SD cards.
Otherwise any other hint/solution is very much appreciated.

Thanks and bye,
Dk

 

[alexskysilk]

why not use your iSCSI SAN for your boot volumes too?

 

[Diabolik67]

Hi,
yes that was exactly the same solution I was thinking about, but first I have to figure out whether it is possible with my "not so young" servers...

 

[alexskysilk]

it is there's a bios option for it along with iSCSI target specification.

 

[fortechitsolutions]

Footnotes in case of use-interest.

-- I believe dell servers since ~5+ years now would have iSCSI boot volume option hiding in the bios somewhere, so that may in fact be an easy option, if you enjoy playing with iSCSI Config and Dell bios. Should be possible to mount an iSCSI Lun as your boot device and tell bios to use that target as the boot device, basically. Would require you setup a dedicated iSCSI target/lun for each proxmox dell box / named in a way that allows clear recognition/configuration since you probably don't want to get them messed up

-- you can probably do some kind of unoffical setup otherwise on the dell boxes. For example if there are sata headers on the motherboard, you could use a pair of modest-sized SSD 2.5" SATA drives and use those as root volume mirror set for proxmox (ie, either via unoffical debian-then-mirror-SW-Raid-config method; or via the proxmox appliance ISO boot installer and ZFS 'mirror' config would also work). Or maybe just buy a few 3rd party drive sleds on ebay/reseller and pop drives in there, since most dell servers have a backplane for a couple of disks maybe. So you possibly could do a local SSD proxmox quite easily if desired.

Hope it works well, whatever route you take!

Tim

 

[Diabolik67]

Hi all, thank you for your replies, but I have been playing with iSCSI boot config for 2 days now without having a working setup.
In two words, the proxmox installer does not see the iSCSI lun.
Here are the screenshots of the various menus, please can you tell me where I'm going wrong ? The only official documentation found
says to choose "One time disabled" in General parameters -> Boot to iscsi target to allow the installation on the iscsi lun, otherwisw the boot fails because of no OS and the target is not mounted.

Thanks for any help,
Dk

PS: I also tried with static IP address on Initiator parameters, but again I cannot see the disk in Proxmox installer.







"

 

[Diabolik67]

I made other tests, and actually the connection between the server and the iSCSI SAN succeeds !
In the pictures I force the iSCSI boot, but it obviously fails because the remote disk is still empty.
So why the Proxmox installer doesn't list the disk in the possible installation targets ?





Here is the Proxmox installer log:

 

[alexskysilk]

The thing about doing things that are not supported by their vendor is that you have to be self sufficient.

Installing proxmox on an iSCSI target is not provided, so dont install it there. install it to a normal disk, then dd it to an iscsi target. Alternatively, you can boot debian, use debootstap to install it to the iscsi target, and then proceed to install proxmox as described here: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster

 

[Diabolik67]

Hi, thank you for the info, I'll try one of the procedures you mentioned (or both, if needed).
Bye,
Dk

 

[Ala de Dragon]

Hello, maybe is late to post. but....

I was able to manage this setup with a HP G7 DL280 server. So since I se you hace a iSCSI boot lun in your console (PETASAN ...) I'll tell you my solution, but you need to know that systemD tricked me and I'm not able to do a clean restart/shutdown. I'll comment soon.

install Debian 10 from live CD into your LUN:
I used expert setup, moved ASAP into a terminal, activate network interface related to iNET, the related to datastorage and assigned ip with the "ip" command from bussybox. pinged the datastore just to be sure connection is ok.
Them I moved into the standard setup and used the iSCSI disk option. reach the end of setup and place Bootloader into you MBR Lun (/dev/mapper/iscsi-lun-whatever)

Them moved into console before leave setup and follow proxmox manual:

https://pve.proxmox.com/wiki/Proxmox_ISCSI_installation


the problem with this setup is that is affected by a bug in systemd witch makes impossible to umount a filesystem and ifdown the interface related to iSCSI. So a loop is made and never ending shutdown process begins. I have no solution yet so any hint is welcome.

If you want more info realated to the bug:

https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdNFSMountShutdown

since is not a production server I shutdown at the end of the session, and each day I fscchk disk in the boot up.

Greets