pveproxy in cluster partially working

[tuxis]

Hi,

I have a cluster of three nodes, one of which rebooted last week. node1 is the node I use for management.

Since the reboot last week, I can use node1 to do e.g. `pvesh get /nodes/node2/netstat` but not for node3. I do get a '200 OK', and after that stuff timesout. When I use the webinterface on node1, I do see the vms on node3, but when I try to open one, there are no details or notes available (nor can I manage it).

So, obviously, something is wrong. `clustat` and `pvecm status` all look good.. Anyone with ideas where to look further?

 

[dietmar]

Any hint in /var/log/syslog?

 

[tuxis]

No, nothing. pvesh sais:
pve:/nodes/proxmox1-3> get netstat
200 OK
Timeout, server 10.10.0.3 not responding.
proxy handler failed
pve:/nodes/proxmox1-3>

 

[dietmar]

Can you do a ssh connect from one node to other without specifying a password?

 

[tuxis]

Ah! That's a usefull hint!

I can SSH
I can run pvesh locally on node3
I cannot run `ssh -o BatchMode=yes root@10.10.0.3 pvesh --noproxy get nodes/proxmox1-3/netstat`
I can run `ssh -t -o BatchMode=yes root@10.10.0.3 pvesh --noproxy get nodes/proxmox1-3/netstat'

So the issue seems to be in pvesh on node3..

 

[tuxis]

Strace on another terminal shows that it reads /proc/net/dev , and thats about it..

 

[dietmar]

What is the output if you run this on proxmox1-3 (ip 10.10.0.3):

pvesh --noproxy get nodes/proxmox1-3/netstat'

Or what address is 'node3' Is it named 'node3' or 'proxmox1-3'?

 

[tuxis]

What is the output if you run this on proxmox1-3 (ip 10.10.0.3):

pvesh --noproxy get nodes/proxmox1-3/netstat'

I get normal, full output.

(node3 is proxmox1-3, sorry for the confusion)

 

[tuxis]

It seems to be a network issue, rather than a linux/proxmox issue. Sorry.