Change IPs on nodes

[joseantonio.trujillo]

Hello mates,

Due to changes in our network (we're going to change VLANs), we need to change the IP on nodes/cluster Proxmox (cluster 2 nodes)

Is there some procedure to change the IPs?

Regards, Jose

 

[shanreich]

You need to edit:

• /etc/network/interfaces
• /etc/hosts

Additionally, depending on how your cluster is set up you might also need to edit /etc/pve/corosync.conf in order for Corosync to pick up on your new IP

 

[joseantonio.trujillo]

Hello Stefan,

OK, Only this? From GUI, I should change anything else?
Then I changed this files, I should restart it? (node)

I have the 6.1-7 version.

Thanks

 

[shanreich]

Should not need to change anything in the UI after this, you will need to reload your network configuration afterwards by running systemctl reload networking

 

[joseantonio.trujillo]

Should not need to change anything in the UI after this, you will need to reload your network configuration afterwards by running systemctl reload networking

Ok, I'm going to do this in two week time. I will inform you and tell you the procedure applied for this.
Thanks

 

[joseantonio.trujillo]

Hello,

Sorry by my doubts.

I read another ticket, https://forum.proxmox.com/threads/change-ip-of-cluster-node.106676/
They changed IP too, but they had some issue.

I will change IP in two week time. So I need to verify my procedure.
Could you help me, please?

Summary:

• I have two nodes

• How I can know if they are a cluster or active-active?
• Finally I need to update the /etc/pve/corosync.conf file?

• Current status:

• Previous notes, only change these files:

/etc/network/interfaces
/etc/hosts

• Do I need to change or stop any thing else?
  • Restart services
  • Recomendation: restart server ( I can do it)
  • Any another file update?

Thanks for your help
Regards, Jose

 

[shanreich]

If you intend to reboot the servers anyway, then it would make sense to change the IP configuration in the 3 files (/etc/network/interfaces, /etc/hosts, /etc/pve/corosync.conf) and then reboot the servers. After the reboot of both nodes the cluster should work as before, if the IPs are all set correctly. So I think your procedure should work.

As an additional note: Judging from your corosync.conf you have two nodes in the cluster. The problem with two nodes is, that the loss of one node means loss of quorum which leads to the whole cluster not functioning anymore. You should consider at least adding a QDevice [1] to your cluster that has a vote, so your whole cluster does not stand still if one node malfunctions.

[1] https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

 

[joseantonio.trujillo]

If you intend to reboot the servers anyway, then it would make sense to change the IP configuration in the 3 files (/etc/network/interfaces, /etc/hosts, /etc/pve/corosync.conf) and then reboot the servers. After the reboot of both nodes the cluster should work as before, if the IPs are all set correctly. So I think your procedure should work.

As an additional note: Judging from your corosync.conf you have two nodes in the cluster. The problem with two nodes is, that the loss of one node means loss of quorum which leads to the whole cluster not functioning anymore. You should consider at least adding a QDevice [1] to your cluster that has a vote, so your whole cluster does not stand still if one node malfunctions.

[1] https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_external_vote_support

Thanks for your fast response.

Really, I can't restart both nodes to the same time.
My idea will be to move all VM to first node. Stop the second node and move it to another place. There, this second node will start it again with same IP.
I mean, the cluster during several hours will be only with a node running.

Do I have to do this anyway? Create un external vote support?

Once I will verify that both nodes are running successfully and check to move VM between nodes in places different, then I will change IP.
So, I will have to update these files (three files)?

That's to say, one day I will move a node. I will leave it stable and then I will move another node next day.
I will make it stable again and then I will change IPs

Regards, Jose

 

[joseantonio.trujillo]

Hello,
Another data else...
I have don't HA configured

# ha-manager status
quorum OK
#

Another question regarding previous question.
I will lost quorum but the single node can continue working, right? VMs will run succesffully?

Regards, Jose

 

[ooptimum]

My idea will be to move all VM to first node. Stop the second node and move it to another place. There, this second node will start it again with same IP.
I mean, the cluster during several hours will be only with a node running.

If you can't stop both nodes simultaneously, there will be some period when they will have IPs from different networks (if I understand you correctly). How do you plan to migrate VMs then? More generally, does Corosync support this scenario? The latter is a question to devs, I suppose.

 

[joseantonio.trujillo]

If you can't stop both nodes simultaneously, there will be some period when they will have IPs from different networks (if I understand you correctly). How do you plan to migrate VMs then? More generally, does Corosync support this scenario? The latter is a question to devs, I suppose.

No, when I move the node to another place both nodes will have same IPs. My network team will extend the network.
So, I only want to know:
- if I can do it without external vote
- When both server will be new site, I will change IP, there... Do I will need to update three files?
Thanks

 

[ooptimum]

Again, if I understand you correctly, you can't tolerate downtime of virtual machines, right? Therefore, at least one node must be running at all times. Is this assumption correct?

 

[joseantonio.trujillo]

Again, if I understand you correctly, you can't tolerate downtime of virtual machines, right? Therefore, at least one node must be running at all times. Is this assumption correct?

Hello, Yes. At least one node must be running at all times with all VM allocated there.
Thanks

 

[shanreich]

Hello, Yes. At least one node must be running at all times with all VM allocated there.
Thanks

This will be hard to achieve since if you change the IP of one node, but not the other, they will not be able to communicate and be out of quorum. This means migrating between those two nodes will be impossible.

Are the new IPs in the same subnet or are they in different subnets? Do you have multiple network ports? With multiple network ports it might be possible to do this in a smooth fashion.

 

[joseantonio.trujillo]

Apologies, I did not explain myself well.
My task has two parts:

My current situtation and plan (part 1)

• I have two nodes in place1 - same network (192.168.116.x)
• I will switch off one node2 in place1 to move to place2. This place2 will has the same network (192.168.116.x)
  • Between place1 and place2 have same network. Both nodes will have connection between them
  • Before, I will migrate all VMS from Node2 to Node1. When I will switch off node2 all VM will be on node1.
    • First question: I need a quorum for this?
    • Second question: Can I always have an UP node with all machines correctly?
• Once Both nodes are UP in differents places (node1 in place1 and node2 in place2) with same network ( 192.168.116.x) They will have connection between them.
  • Now, I will migrated all VMS from node1 to node2.
  • Switch off node1 and move node to place2
  • Both nodes in place2 with same network

Both nodes in new place, place2. Here, I want to change IPs. (part 2)

• Here, I will have a plan to change IPs to the same time. I will assume a shutdown of the virtual machines for a few minutes
• Procedure to change IP:
  • Edit /etc/network/interfaces
  • Edit /etc/hosts
• Questions: I need to update /etc/pve/corosync.conf file? What I need else?

Thanks for your help
Regards, Jose

 

[shanreich]

Here, I will have a plan to change IPs to the same time. I will assume a shutdown of the virtual machines for a few minutes

Ah okay, I thought you wanted to have uptime between both nodes during the IP change as well. In that case it should just be sufficient to edit the 3 files you listed and reboot the nodes. You should shutdown the VMs before changing any configuration, since you will not be able to do so after you lose quorum and might run into issues when rebooting. Start them again only after your IP changes are succesful.

Again, it should be sufficient to just edit these 3 files on both nodes and then reboot. I tried this on my test cluster and it worked fine:

• /etc/network/interfaces
• /etc/hosts
• /etc/pve/corosync.conf

Please be careful when updating /etc/pve/corosync.conf - you need to increase the config_version of the file, otherwise it will not work. You can look at our wiki for more details: https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_configuration

First question: I need a quorum for this?

While only one node is running, this node will not have quorum without the second node. This means you cannot perform any actions that need quorum (so, almost all actions such as Start/Stop/..). You can set the expected votes to 1 temporarily with pvecm expected 1, then you will have a quorum on the single node as long as it is alone. Make sure to set the expected to 2 everytime before you start a second node again. This will lead to a short loss of quorum while it is set to 2 and the other node is still booting, but should not be a problem for a short amount of time.

As long as you don't have HA enabled having no quorum should not be that big of a problem as long as it is temporarily. If you have HA enabled, make sure to disable it during this whole process - otherwise the node that is alone will reboot.

Please also make sure that the network between place1 and place2 is stable and has low latency (at most 5 ms constantly). If you have high latency on this connection, clustering will break down and migrations will not work.

 

[joseantonio.trujillo]

That's great! Thanks for your explications.
So, I have clear:
- I can move node without configure a third node as quorum. However I need put 'pvecm expected 1' How do I do it? This command I shoud execute any proxmox node: # pvecm expected 1 --> Before switch off and move node2, right?
- I need to increase the config_version of the file. How? Backup file and execute: # journalctl -b -u corosync
Thanks. Sorry my question, I am new here!
Regards, Jose

 

[shanreich]

- I can move node without configure a third node as quorum. However I need put 'pvecm expected 1' How do I do it? This command I shoud execute any proxmox node: # pvecm expected 1 --> Before switch off and move node2, right?

It should be fine to do it after switching off the other node, if you do not have HA enabled. You will lose quorum very shortly then, but it is safer this way.

I need to increase the config_version of the file. How?

As outlined in [1], in the corosync.conf file you will find the line:

totem {
    [...]
    config_version: 4        # here
    [...]
}

just increment that number by one, so in this case change 4 to 5.


[1] https://pve.proxmox.com/wiki/Cluster_Manager#_corosync_configuration

 

[joseantonio.trujillo]

Hi,

OK, I have config_version 2. I should put to 5. Why? Is to understood?




I can change version and IPs to the same time, right? Then, check and restart servers:


Regarding HA, exactly I have don't HA activated (none VM)
I will execute the command: pvecm expected 1 on node1 (UP status).

Thanks

 

[shanreich]

OK, I have config_version 2. I should put to 5. Why? Is to understood?

If you have config_version 2, then you should change it to 3 . increment = add 1 to the number

I can change version and IPs to the same time, right? Then, check and restart servers:

yes, you have to change both at the same time. corosync.conf should be synced between both nodes, so you would only need to change it once. but check the other server as well - just in case