[TUTORIAL] Install PVE directly on iSCSI target

[Pavel Vatolin]

Thank You for perfect post!

But please make me understand what is wrong in my setup!

I have multi NICs server. One NIC used for iSCSI boot, another for regular LAN. Installation went fine, respect to You, I've get working node.
But no visible address on interface commented as iSCSI. This is dedicated NIC for iSCSI boot and in the real it have the IP.
If I try to add IP on it through the Web-interface system becames unresponsive.
And I have messages on the screen identical to the Ray_'s comment (the first comment to yours post).
How can I resolve this? Without the second IP I can not join the node to cluster.

 

[deathblade]

Hi !

If I try to add IP on it through the Web-interface system becames unresponsive.

This is normal, when you click apply, ifupdown stops all links and restart them. By this way the iSCSI connection is interrupted.

I think you can set the same IP as you set in your NIC's iSCSI BIOS but don't apply settings online, just reboot the host to take settings. It will prevent the system to stops links while running.

 

[Pavel Vatolin]

I am very grateful for your answer!

I think you can set the same IP as you set in your NIC's iSCSI BIOS but don't apply settings online, just reboot the host to take settings. It will prevent the system to stops links while running.

But unfortunately it didn't help.

As in previous times, everything went well and iSCSI boot, and GRUB, and ramdisk worked perfectly. But when the main system starts to boot, the disk stops responding.

I am so sad...
Worst of all, the same but not identical configuration works fine for me on a pve2 node in the same cluster.
Node pve2 boots from iSCSI disk located on the same TrueNAS SCALE as a unlucky node pve3. But no problem to set IP to iSCSI NIC on pve2.

pve3 NIC commented as 'fake' has no even patch cord inserted and was needed only to join cluster.

Any think what to do?

 

[deathblade]

Is your iSCSI NIC the same on all your pve hosts ?

With my Intel NIC's I have no issues, but I never try with others brands/models.

 

[Pavel Vatolin]

One brand - yes. It's Intel of course.
The same - no.
Good node has onboard 2 NICs:

04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

The bad one has 2 NICs add-on PCIe card:

08:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

I've download and install latest Intel Boot Agent firmware 3.1.80 from 27.4 driver pack on I350, but still no luck (

 

[deathblade]

It is possible the iSCSI kernel module cannot communicate with this particular NIC fw to get IP at boot.

I dont have same hw config as yours and when we do some tricks like this, the results can change a lot depending of hardware.

I I think you should install the same NIC on all hosts.

 

[Pavel Vatolin]

Ok.
Unfortunately onboard NICs on "bad" node do not accept firmware. BootUtil shows this:

Port Network Address Location Series  WOL Flash Firmware                Version
==== =============== ======== ======= === ============================= =======
  1   002590E64FC0     8:00.0 Gigabit YES iSCSI Primary                 3.1.80
  2   002590E64FC1     8:00.1 Gigabit YES iSCSI                         3.1.80
  3   003048C92652    10:00.0 Gigabit YES FLASH Not Present
  4   003048C92653    10:00.1 Gigabit YES FLASH Not Present

And I can't flash NICs 3 and 4.

The way to modeling you setup is to find the same I350 add-on card, I think. But I don't have one...
May by I grub I350 from "bad" node and do some experiments on another motherboard with it later.

I still don't understand what's going wrong (

 

[deathblade]

To be sure to get a reliable system, it's recommanded to have the same hardware on you multiple hosts.

I recommand to get some intel X520-based copy in china shop.

 

[Pavel Vatolin]

X520! China shop! That's exactly what I've thinking about!

Great thanks to You!

And I'll keep trying...