Hello All,
I've spent far too much time getting PVE installed in our Cisco UCS blade environment. So in the hope that it helps others, I will share my installation instructions. Thanks to those who provided little nuggets of information to help along the way. Feel free to take and improve; I'm not an expert here; just determined
Our environment consists of the aforementioned Cisco UCS system, NetApp storage and Cisco switching. Each blade has four interfaces and the design intent is to use two of these for iSCSI A and B networks and bond the other two then overlay VLANs to separate traffic.
This guide just covers the installation of Debian 12 but once installed you can follow Proxmox's guide for installing PVE on Debian.
Broadly follow https://forums.debian.net/viewtopic.php?p=754711#p754711 but drop out to a shell after the network interfaces have been detected but before configuring the network interfaces. As we have VLANs on our interfaces you need to create the VLAN interface on the command line as follows:
Proceed with the installation until you reach the network configuration screen:
At this point, select Go Back, then select Execute a shell:
In the shell we need to setup the VLAN that will configure so that the installer can talk to the outside world:
You should now see an additional interface as shown below. Select this and configure accordingly.
Configure this network interface but don't worry if the network configuration reports an error, just continue until the disk detection fails, e.g.
At this point, select Go Back and Execute a shell. Then enter these commands:
You can then continue with the installation. When you reach the following screen, configure the iSCSI volumes:
Login to both interfaces on the storage system via the iscsi-a network (or whatever network was configured at boot time), for our storage system this is 10.31.5.101 and 10.31.5.102 for site-b or 10.31.5.103 and 10.31.5.104 for site-a hosts.
Then proceed with the installation.
If necessary, specify a proxy to access the outside world.
Before rebooting, Go Back and once more Execute a shell and run the following commands:
Now complete the installation and reboot.
Once rebooted we need to configure networking bonding and the second iSCSI network and multipathing. I'll cover those in part two if people are interested.
Hope this helps
Mark
I've spent far too much time getting PVE installed in our Cisco UCS blade environment. So in the hope that it helps others, I will share my installation instructions. Thanks to those who provided little nuggets of information to help along the way. Feel free to take and improve; I'm not an expert here; just determined
Our environment consists of the aforementioned Cisco UCS system, NetApp storage and Cisco switching. Each blade has four interfaces and the design intent is to use two of these for iSCSI A and B networks and bond the other two then overlay VLANs to separate traffic.
This guide just covers the installation of Debian 12 but once installed you can follow Proxmox's guide for installing PVE on Debian.
Broadly follow https://forums.debian.net/viewtopic.php?p=754711#p754711 but drop out to a shell after the network interfaces have been detected but before configuring the network interfaces. As we have VLANs on our interfaces you need to create the VLAN interface on the command line as follows:
Proceed with the installation until you reach the network configuration screen:
At this point, select Go Back, then select Execute a shell:
In the shell we need to setup the VLAN that will configure so that the installer can talk to the outside world:
Bash:
vconfig add eno5 312
ip link set eno5.312 mtu 1500
ip link set eno5 up
exitYou should now see an additional interface as shown below. Select this and configure accordingly.
Configure this network interface but don't worry if the network configuration reports an error, just continue until the disk detection fails, e.g.
At this point, select Go Back and Execute a shell. Then enter these commands:
Bash:
ip link set eno5 up
ip route add default via 10.31.2.1 dev eno5.312
# You should also probe for the iscsi boot information from the iSCSI Boot Firmware Table
modprobe iscsi_ibft
iscsistart -N # bring up the network as specified by iBFT
iscsistart -b # create a session to the target using iBFT
modprobe multipath
exitLogin to both interfaces on the storage system via the iscsi-a network (or whatever network was configured at boot time), for our storage system this is 10.31.5.101 and 10.31.5.102 for site-b or 10.31.5.103 and 10.31.5.104 for site-a hosts.
Then proceed with the installation.
If necessary, specify a proxy to access the outside world.
Before rebooting, Go Back and once more Execute a shell and run the following commands:
Bash:
mount --bind /sys /target/sys
mount --bind /proc /target/proc
mount --bind /dev /target/dev
chroot /target
echo 'iscsi_ibft' >> /etc/initramfs-tools/modules
echo 'ISCSI_AUTO=true' > /etc/iscsi/iscsi.initramfs
apt-get update
apt-get install multipath-tools
multipath -a /dev/sdd
apt-get install multipath-tools-boot
update-initramfs -u
exit
exitNow complete the installation and reboot.
Once rebooted we need to configure networking bonding and the second iSCSI network and multipathing. I'll cover those in part two if people are interested.
Hope this helps
Mark
