ProxMox ">_ Console" doesn't work across the cluster, in some cases

TasMot

New Member
Oct 29, 2018
13
0
1
61
I have a 3 node ProxMox cluster. All updated to:
Kernel Version : Linux 4.15.18-26-pve #1 SMP PVE 4.15.18-54 (Sat, 15 Feb 2020 15:34:24 +0100)

PVE Manager Version : pve-manager/5.4-13/aee6f0ec

I use the current version of Chrome running on Windows 10 to access and manage the ProxMox cluster.

When I am logged on to ProxMox1 or ProxMox3 (I know very original names), I cannot access the console of the VMs on ProxMox2. However; when I am logged on to ProxMox2 I can access the console for all 3 members of the cluster. I know this can be fixed because I had the problem before (believe it or not when I only had ProxMox1 and ProxMox3). BUT, I forgot to take a note of how to solve the problem and I can't find that article again. There is no fancy configuration, all SSL is still at the default.

All ProxMox hosts have a static IP for IPv4 and IPv6.

Thanks in advance for any help you can provide,

Tom
P.S. I will keep a note this time because I should have #4 coming on line soon (as soon as I can have email down for a couple hours while I convert from XenServer to ProxMox).
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,535
276
88
hi,

When I am logged on to ProxMox1 or ProxMox3 (I know very original names), I cannot access the console of the VMs on ProxMox2. However; when I am logged on to ProxMox2 I can access the console for all 3 members of the cluster

what does it tell you when you try to access console of VMs on PMX2 from PMX1-3 ? can you access the node shell of PMX2 from the other nodes?
if you post the output of pvecm status from all nodes it could help.

i also encourage you to upgrade your cluster to the latest PVE 6.1 whenever you can.
 

TasMot

New Member
Oct 29, 2018
13
0
1
61
hi,



what does it tell you when you try to access console of VMs on PMX2 from PMX1-3 ? can you access the node shell of PMX2 from the other nodes?
if you post the output of pvecm status from all nodes it could help.

i also encourage you to upgrade your cluster to the latest PVE 6.1 whenever you can.

root@ProxMox1:~# pvecm status
Quorum information
------------------
Date: Fri Mar 20 12:11:34 2020
Quorum provider: corosync_votequorum
Nodes: 3
Node ID: 0x00000003
Ring ID: 3/26248
Quorate: Yes

Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 3
Quorum: 2
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000003 1 2001:470:1f07:da3::221 (local)
0x00000002 1 2001:470:1f07:da3::223
0x00000001 1 2001:470:1f07:da3::242
root@ProxMox1:~#

root@ProxMox2:~# pvecm status
Quorum information
------------------
Date: Fri Mar 20 12:14:25 2020
Quorum provider: corosync_votequorum
Nodes: 3
Node ID: 0x00000001
Ring ID: 3/26248
Quorate: Yes

Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 3
Quorum: 2
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000003 1 2001:470:1f07:da3::221
0x00000002 1 2001:470:1f07:da3::223
0x00000001 1 2001:470:1f07:da3::242 (local)
root@ProxMox2:~#

root@ProxMox3:~# pvecm status
Quorum information
------------------
Date: Fri Mar 20 12:16:51 2020
Quorum provider: corosync_votequorum
Nodes: 3
Node ID: 0x00000002
Ring ID: 3/26248
Quorate: Yes

Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 3
Quorum: 2
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000003 1 2001:470:1f07:da3::221
0x00000002 1 2001:470:1f07:da3::223 (local)
0x00000001 1 2001:470:1f07:da3::242
root@ProxMox3:~#


This is the console when I try to connect to 2 from 1 or 3:

1584721256905.png
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,535
276
88
-> can you connect to ProxMox2 shell? (click ProxMox2 on the left menu and choose 'Shell')

-> what do you see in the tasklog (bottom) after connection fails? can you post the output of that task?
 

TasMot

New Member
Oct 29, 2018
13
0
1
61
From 1 and 3 when I click on the ProxMox2 Shell, I have to enter the root password, but it connects.
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,535
276
88
can you ssh from all nodes to all the others? (ex. pmx1 -> pmx3/pmx2 and vice versa for all of them)
it should work without any user interaction.
if it doesn't, then check the /etc/hosts of all nodes. make sure hostname --ip-address returns the correct IP for all nodes.
 

TasMot

New Member
Oct 29, 2018
13
0
1
61
can you ssh from all nodes to all the others? (ex. pmx1 -> pmx3/pmx2 and vice versa for all of them)
it should work without any user interaction.
if it doesn't, then check the /etc/hosts of all nodes. make sure hostname --ip-address returns the correct IP for all nodes.

No, I can't accessing ProxMox2 always asks for a password.
However; I think I know what is going on now.

A while back I noticed that in a moment of plain dumb, I assigned ProxMox2 the same IP as my main switch (an older 3COM 48 port switch). Since I couldn't get into the switch to change it's IP, I changed the IP for ProxMox2.

I have since changed the IP for the switch.

So, now the question is, "will it fix everything if I change the IP for ProxMox2 back, or am I totally hosed?" If that is not the solution, how do I proceed?

Thanks,
Tom
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,535
276
88
No, I can't accessing ProxMox2 always asks for a password.
it should not ask a password. please check the /etc/hosts and hostname like i told you
 

TasMot

New Member
Oct 29, 2018
13
0
1
61
it should not ask a password. please check the /etc/hosts and hostname like i told you

I do have a fully functional Active Directory integrated DNS system, so I only have minimal info in the hosts file. Here are the 3 hosts files:

root@ProxMox1:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
2001:470:1f07:da3::221 ProxMox1.corp.TasMot.com ProxMox1 pvelocalhost

# The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
root@ProxMox1:~#

root@ProxMox2:~/.ssh# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
2001:470:1f07:da3::242 ProxMox2.corp.TasMot.com ProxMox2 pvelocalhost

# The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
root@ProxMox2:~/.ssh#

root@ProxMox3:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
2001:470:1f07:da3::223 ProxMox3.corp.TasMot.com ProxMox3 pvelocalhost

# The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
root@ProxMox3:~#


Thanks,

Tom
 

TasMot

New Member
Oct 29, 2018
13
0
1
61
can you ssh from all nodes to all the others? (ex. pmx1 -> pmx3/pmx2 and vice versa for all of them)
it should work without any user interaction.
if it doesn't, then check the /etc/hosts of all nodes. make sure hostname --ip-address returns the correct IP for all nodes.
Yes all hostname commands return the correct IP using the DNS system.
 

TasMot

New Member
Oct 29, 2018
13
0
1
61
After months of working on this on my own and then back in March finally asking for help, I have finally found the solution to this problem. The answer was because of a slightly different problem that caused a different line of searching for an answer. In any case, the problem was bad permissions on the /etc directory. Here is the explanation of the way I found the solution: https://forum.proxmox.com/threads/vm-migration-failed.72544/#post-324309.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!