Errors with pveproxy

G

geormanth

Renowned Member
Sep 9, 2013
16
0
66
Hi All,

I just install a 3 node cluster connected with tinc vpn.
node1 is in different data center from node2 and node3 that are in the same.

I have some problems:
1. When I am login to web node 1 and try to see anything from other nodes proxmox loged me out with the following error
"error connection error 596: Broken pipe" the same from other nodes
2. When I am try to create new CT I cant see nodes or loged me out.

These are syslog errors from the nodes:

Node1 cloud01- IP 10.10.200.10
When I am trying to see syslog it loged me out with the following error
Connection error 401: permission denied - invalid ticket

Node2 gam01- IP 10.10.200.11
Sep 10 00:36:20 gam01 pveproxy[3671]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:36:20 gam01 pveproxy[3671]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:36:21 gam01 corosync[3426]: [TOTEM ] Retransmit List: 99c 99d
Sep 10 00:36:21 gam01 pveproxy[3670]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:36:21 gam01 pveproxy[3670]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:36:23 gam01 pveproxy[3671]: problem with client 10.10.200.12; rsa_eay_public_decrypt: padding check failed
Sep 10 00:36:23 gam01 pveproxy[3671]: WARNING: proxy detected vanished client connection
Sep 10 00:36:24 gam01 pveproxy[3670]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:36:24 gam01 pveproxy[3670]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:36:25 gam01 pveproxy[3669]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:36:25 gam01 pveproxy[3669]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:36:26 gam01 pveproxy[3671]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:36:26 gam01 pveproxy[3671]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:36:26 gam01 pveproxy[3669]: problem with client 10.10.200.12; rsa_eay_public_decrypt: padding check failed
Sep 10 00:36:26 gam01 pveproxy[3669]: WARNING: Can't call method "timeout_reset" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 170.

Node3 gam02- IP 10.10.200.12

Sep 10 00:39:36 gam02 pveproxy[6873]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:39:36 gam02 pveproxy[6873]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:39:39 gam02 pveproxy[6873]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:39:39 gam02 pveproxy[6873]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.
Sep 10 00:39:40 gam02 pveproxy[7245]: problem with client 10.10.200.10; rsa_padding_check_pkcs1_type_1: block type is not 01
Sep 10 00:39:40 gam02 pveproxy[7245]: WARNING: Can't call method "on_drain" on an undefined value at /usr/share/perl5/PVE/HTTPServer.pm line 160.

The pvecm nodes is different from each node so

from node1
root@cloud01 ~ # pvecm nodes
Node Sts Inc Joined Name
1 M 2312 2013-09-09 22:47:32 cloud01
2 M 2316 2013-09-09 22:48:16 gam02
3 M 2320 2013-09-09 23:21:38 gam01

from node2
root@gam01~# pvecm nodes
Node Sts Inc Joined Name
1 M 2320 2013-09-10 00:21:39 cloud01
2 M 2320 2013-09-10 00:21:39 gam02
3 M 4 2013-09-10 00:21:30 gam01

from node3
root@gam02 ~# pvecm nodes
Node Sts Inc Joined Name
1 M 2316 2013-09-09 23:48:16 cloud01
2 M 2308 2013-09-09 23:48:13 gam02
3 M 2320 2013-09-10 00:21:38 gam01

The version is the same for all nodes

pveversion -v
proxmox-ve-2.6.32: 3.1-109 (running kernel: 2.6.32-23-pve)
pve-manager: 3.1-3 (running version: 3.1-3/dc0e9b0e)
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-7
qemu-server: 3.1-1
pve-firmware: 1.0-23
libpve-common-perl: 3.0-6
libpve-access-control: 3.0-6
libpve-storage-perl: 3.0-10
pve-libspice-server1: 0.12.4-1
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: not correctly installed
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: not correctly installed
glusterfs-client: 3.4.0-2

Please help.

Thanks in advance
George
 
Please try to clear your browser cache, then stop the browser and restart it again (or test with another browser).
 
tom said:
> apt-get install proxmox-ve-2.6.32 ksm-control-daemon vzprocps
Click to expand...

Thanks for your quick response.

After the installation it seems that working fine.
But I see a lot corosync TOTEM logs

Here are some of them

Sep 10 15:02:08 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6b8 6b9 6bd 6bf
Sep 10 15:02:08 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6b8 6b9 6bd 6bf
Sep 10 15:02:08 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6b8 6b9 6bd 6bf
Sep 10 15:02:08 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6b8 6b9 6bd 6bf
Sep 10 15:02:11 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6ce 6cf 6d0 6d1 6d4 6d8
Sep 10 15:02:13 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6ce 6cf 6d0 6d1 6d8
Sep 10 15:02:16 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6ce 6cf 6d8
Sep 10 15:02:21 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6ce
Sep 10 15:02:21 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6ce
Sep 10 15:02:27 gam01 corosync[3426]: [TOTEM ] Retransmit List: 6e2


Regards
George
 
tom said:
cluster communication is not working, due to your network design.

check http://pve.proxmox.com/wiki/Multicast_notes
Click to expand...

Before create cluster I was checked multicast and is working fine.
Here are the results from omping 10.10.200.10 10.10.200.11 10.10.200.12

10.10.200.11 : multicast, seq=5, size=69 bytes, dist=0, time=142.178ms
10.10.200.12 : unicast, seq=1, size=69 bytes, dist=0, time=129.531ms
10.10.200.12 : multicast, seq=1, size=69 bytes, dist=0, time=130.940ms
10.10.200.12 : unicast, seq=2, size=69 bytes, dist=0, time=136.172ms
10.10.200.12 : multicast, seq=2, size=69 bytes, dist=0, time=137.885ms
10.10.200.11 : unicast, seq=6, size=69 bytes, dist=0, time=139.649ms


10.10.200.12 : multicast, seq=1, size=69 bytes, dist=0, time=277.151ms
10.10.200.10 : unicast, seq=4, size=69 bytes, dist=0, time=113.396ms
10.10.200.10 : multicast, seq=4, size=69 bytes, dist=0, time=113.511ms
10.10.200.12 : unicast, seq=2, size=69 bytes, dist=0, time=225.602ms
10.10.200.12 : multicast, seq=2, size=69 bytes, dist=0, time=227.515ms
10.10.200.10 : unicast, seq=5, size=69 bytes, dist=0, time=128.708ms

10.10.200.10 : multicast, seq=18, size=69 bytes, dist=0, time=122.435ms
10.10.200.11 : unicast, seq=18, size=69 bytes, dist=0, time=291.292ms
10.10.200.11 : multicast, seq=18, size=69 bytes, dist=0, time=291.459ms
10.10.200.11 : unicast, seq=19, size=69 bytes, dist=0, time=240.302ms
10.10.200.11 : multicast, seq=19, size=69 bytes, dist=0, time=241.990ms
10.10.200.10 : unicast, seq=20, size=69 bytes, dist=0, time=132.519ms
10.10.200.10 : multicast, seq=20, size=69 bytes, dist=0, time=132.547ms


Also checked with ssmpingd between two nodes: node1 with node2, node2 with node3, node1 with node3 and working fine.
 
geormanth said:
I read here: http://www.hastexo.com/resources/hints-and-kinks/whats-totem-retransmit-list-all-about-corosync
to reduce window_size to 50 in corosync.conf, but I cant find where is corosync.conf.
What is your opinion about that and do you know where is the file?
Click to expand...

Your need to add a 'totem' section to /etc/pve/cluster.conf (increase version before you save the file).
Code: 
[COLOR=#009900][COLOR=#000000][B]<totem[/B][/COLOR] [COLOR=#000066]window_size[/COLOR]=[COLOR=#ff0000]"50"[/COLOR][COLOR=#000000][B]/>[/B][/COLOR][/COLOR]
 
dietmar said:
Your need to add a 'totem' section to /etc/pve/cluster.conf (increase version before you save the file).
Code: 
[COLOR=#009900][COLOR=#000000][B]<totem[/B][/COLOR] [COLOR=#000066]window_size[/COLOR]=[COLOR=#ff0000]"50"[/COLOR][COLOR=#000000][B]/>[/B][/COLOR][/COLOR]
Click to expand...

Hi,

It works fine and the retransmitting stop.

I would like to ask what type of fence device to use, I don't know what is behind the server to choose the correct one.
As you mention in wiki we have APC, Intel, Dell, IPMI .

Thanks again for useful help
George
 
You must log in or register to reply here.
Top Bottom