Automating Network Configurations to be Permanent

Zubin Singh Parihar

Well-Known Member
Nov 16, 2017
55
1
48
42
Hey Proxmox team.

I've installed Proxmox VE 8 and and then changed over to the 'no-subscription' repository and have done an 'apt update/upgrade' as of September 29, 2023

I'm running the following Version of Proxmox VE:


~# dpkg -l | grep -i "pve-" | awk '{print $2 "\t\t" $3;}'
libpve-access-control 8.0.5
libpve-apiclient-perl 3.3.1
libpve-cluster-api-perl 8.0.3
libpve-cluster-perl 8.0.3
libpve-common-perl 8.0.8
libpve-guest-common-perl 5.0.4
libpve-http-server-perl 5.0.4
libpve-rs-perl 0.8.5
libpve-storage-perl 8.0.2
libpve-u2f-server-perl 1.2.0
pve-cluster 8.0.3
pve-container 5.0.4
pve-docs 8.0.4
pve-edk2-firmware 3.20230228-4
pve-firewall 5.0.3
pve-firmware 3.8-2
pve-ha-manager 4.0.2
pve-i18n 3.0.7
pve-kernel-6.2 8.0.5
pve-kernel-6.2.16-1-pve 6.2.16-1
pve-kernel-6.2.16-2-pve 6.2.16-2
pve-kernel-6.2.16-3-pve 6.2.16-3
pve-lxc-syscalld 1.3.0
pve-manager 8.0.4
pve-qemu-kvm 8.0.2-6
pve-xtermjs 4.16.0-3


I'm all about automation, and so I like to configure my cluster on the command line.

For one of my clusters, I'm configuring the /etc/network/interfaces file and then I run an 'ifreload -a' to get the network configured the way I like.


SCENARIO

Upon initial install of the cluster a I configure a few interfaces and openvswitch vlan's from the Web UI. I then proceeded to write directly to my /etc/network/interfaces and than ran 'ifreload -a' and then all my Network configurations appeared as the should.

However, I noticed a spelling error in the comment area of one of my network devices when observing from the Web UI, and so directly from the WEB UI itself, I corrected the spelling error, and then pressed 'OK'.

But then I noticed that all my changes that I had previously done directly to the /etc/network/interfaces had been set for removal, and if i press "Apply Configuration" it removes.

So it would seem that the WEB UI is writing the changes to some sort of separate DB... and when I make changes directly on the filesystem and run ifreload, it does not update this DB.

I then found teh following line in /etc/network/interfaces:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

For automation purposes, is there a way on the command line that we can somehow apply these network changes to be pemanent?

Sysadmins lives and jobs are on the line, LOL

Thanks.



PS: below is an example of my /etc/network/interfaces that I manually update, and would like to avoid hundreds of clicks on each Hypervisor in my Cluster (especially if I decide to run a 32 Node Cluster.


# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface ens1f0 inet manual
#1GBs - Rosewell Connection

auto eno1
iface eno1 inet manual
ovs_type OVSPort
ovs_bridge vmbr2
#10GB - Dell 8024 Switch Connection

auto eno2
iface eno2 inet manual
ovs_type OVSPort
ovs_bridge vmbr3
#10GB - Dell 8024 Switch Connection

auto ens1f1
iface ens1f1 inet manual
ovs_type OVSPort
ovs_bridge vmbr1
#1GBs - Dell 6248 Switch Connection

iface ens1f2 inet manual
#1GBs

iface ens1f3 inet manual
#1GBs

auto corosync_ring_1
iface corosync_ring_1 inet static
address 192.168.6.42/24
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_options tag=6
#1GBs - Corosync (6) Network Ring 1

auto lan
iface lan inet static
address 192.168.3.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=3
#10GBs - LAN

auto ceph_public
iface ceph_public inet static
address 192.168.10.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=10
#10GBs - Ceph Public Network

auto ceph_sync
iface ceph_sync inet static
address 192.168.11.42/24
ovs_type OVSIntPort
ovs_bridge vmbr3
ovs_options tag=11
#10GBs - Ceph Storage Sync Network

auto proxmoxbackup
iface proxmoxbackup inet static
address 192.168.7.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=7
#10GBs - Proxmox VM Backup

auto nfs_storage
iface nfs_storage inet static
address 192.168.8.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=8
#10GBs - NFS Storage Network

auto iscsi_storage
iface iscsi_storage inet static
address 192.168.9.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=9
#10GBs - iSCSI Storage Network

auto vm_migration
iface vm_migration inet static
address 192.168.99.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=99
#10GBs - Management Network

auto management
iface management inet static
address 192.168.100.42/24
ovs_type OVSIntPort
ovs_bridge vmbr1
ovs_options tag=100
#1GBs - Management Network

auto vlan110
iface vlan110 inet static
address 192.168.110.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=110
#10GBs - VLAN110 (pfsense)

auto vlan120
iface vlan120 inet static
address 192.168.120.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=120
#10GBs - VLAN120 (pfsense)

auto vlan130
iface vlan130 inet static
address 192.168.130.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=130
#10GBs - VLAN130 (pfsense)

auto vlan140
iface vlan140 inet static
address 192.168.140.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=140
#10GBs - VLAN140 (pfsense)

auto vlan150
iface vlan150 inet static
address 192.168.150.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=150
#10GBs - VLAN150 (pfsense)

auto vlan160
iface vlan160 inet static
address 192.168.160.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=160
#10GBs - VLAN160 (pfsense)

auto vlan170
iface vlan170 inet static
address 192.168.170.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=170
#10GBs - VLAN170 (pfsense)

auto vlan180
iface vlan180 inet static
address 192.168.180.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=180
#10GBs - VLAN180 (pfsense)

auto vlan190
iface vlan190 inet static
address 192.168.190.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=190
#10GBs - VLAN190 (pfsense)

auto vlan200
iface vlan200 inet static
address 192.168.200.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=200
#10GBs - VLAN200 (pfsense)

auto vlan210
iface vlan210 inet static
address 192.168.210.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=210
#10GBs - VLAN210 (pfsense)

auto vlan220
iface vlan220 inet static
address 192.168.220.42/24
ovs_type OVSIntPort
ovs_bridge vmbr2
ovs_options tag=220
#10GBs - VLAN220 (pfsense)

auto vmbr0
iface vmbr0 inet static
address 192.168.2.42/24
gateway 192.168.2.254
bridge-ports ens1f0
bridge-stp off
bridge-fd 0
#1GBs - WAN Bridge

auto vmbr1
iface vmbr1 inet manual
ovs_type OVSBridge
ovs_ports ens1f1 corosync_ring_1 management
#1GBs - Bridge

auto vmbr2
iface vmbr2 inet manual
ovs_type OVSBridge
ovs_ports eno1 lan ceph_public proxmoxbackup
#10GBs - Bridge

auto vmbr3
iface vmbr3 inet manual
ovs_type OVSBridge
ovs_ports eno2 ceph_sync
#10GBs - Bridge
 
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.

Thank me later.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!