Is it possible to configure cluster using other than Port22 to do syncing?

N

Nikole

Well-Known Member
Jun 3, 2013
41
0
46
As the title says I need to use another port (not 22) for this purpose.
Can I simply declare the new port somewhere in the PMG config?
Note that that same port will be also configured in the SSHD deamon of Debian.
Any help is much appreciated!

Thank you.
Nicole
 
Yes, it uses plain ssh to sync. So you could change both, the SSHD (serve) and the SSH (client) default port.

For SSHD just open /etc/ssh/sshd_config then uncomment the "Port" key and change the value to the new desired port,

For SSH you can go two routes either also /etc/ssh/ssh_config (note the missing "d" in the file name) but then you need to explicitly pass the port for all other (manual) ssh operation you do from the PMG host to the outside. Another way would be to add entires to "/root/.ssh/config" alá:
Code: 
host pmgnode1
    Port 12345

host pmgnode2
    Port 12345

# ... and so on

for all PMG nodes and copy that file to all nodes after that.

Finally restart the sshd service
Code: 
systemctl restart ssh.service
and the change should be in effect.

I recommend testing that before doing it, or at least keeping an open SSH session during that so that it can be rolled back if anything fails.
 
Hi Thomas,

Thank for the information.
I created the file /root/.ssh/config (as it was not present) with the correct details.
I then tested that I can ssh from one system to the other without using password and it does work.
However the syncing process is failing.

I am getting the error below on both systems (each refers to a database named after the remote system):


Code: 
Aug 28 06:35:07 member1 pmgmirror[917]: database sync 'masternode' failed - command 'rsync '--rsh=ssh -l root -o BatchMode=yes -o HostKeyAlias=masternode' -q -aq --timeout 10 '[x.x.x.x]:/var/spool/pmg/cluster/2/' /var/spool/pmg/cluster/2 --include spam/ --include 'spam/*' --include 'spam/*/*' --include virus/ --include 'virus/*' --include 'virus/*/*' --exclude '*'' failed: exit code 23

Aug 28 06:35:07 member1 pmgmirror[917]: cluster syncronization finished  (1 errors, 1.81 seconds (files 0.00, database 1.17, config 0.65))

...where x.x.x.x is the public IP of the masternode

I did try a few things to troubleshoot it but no luck.
Any ideas?

Note that both systems are in two different networks with NAT port forwards. Within the cluster.conf file(s) I have edited and substituted private IPs with the public ones. Several server restarts.... Nothing :(
 
could you please post the complete journal around the time you get the error?
What's the output of `pmgcm status`?

Thanks!
 
The status is:

Code: 
root@masternode:~# pmgcm status
NAME(CID)--------------IPADDRESS----ROLE-STATE---------UPTIME---LOAD----MEM---DISK
masternode(1)          x.x.x.x    master S           10:02   0.49    79%    19%
member1(3)         y.y.y.y   node   S           10:02   0.34    85%    34%
root@masternode:~#

...where x.x.x.x and y.y.y.y are the public IPs of the the masternode and member1 respectively.

I am also attaching some of the syslog which shows that is a constant error.
 

Attachments

Code: 
Aug 28 09:40:55 masternode pmgtunnel[762]: restarting crashed tunnel 7705 y.y.y.y

seems pmgtunnel is not able to keep an ssh-connection open...
could you please:
* stop the pmgtunnel service: `systemctl stop pmgtunnel`
* start it in debug mode (keeps it in the foreground and might shed some more light on the issue): `pmgtunnel start --debug`

Paste any output you gather

Afterwards kill it (Ctrl-C should work) and start the 'pmgtunnel' service again

Thanks!
 
I think I found out why.
See this:

root@spamtrap:~# pmgtunnel start --debug
ssh: connect to host y.y.y.y port 22: Connection timed out

where y.y.y.y = the public IP of member1

Why is trying to connect to 22 however?
I have already edited the file /root/.ssh/config with the following:

host member1
Port 12345

To test it I simply run ssh root@member1 and it connects fine (no pass needed as well).
So why the ssh command works with the custom port but not the syncing job?

(Note that I also have an entry in my etc/hosts file "y.y.y.y member1.mydomain.com member1" and hence the ssh command resolves fine)
Any ideas?
 
hmm - the ssh_config does not do a resolution afaik - in order to test this:
does: `ssh y.y.y.y` work without password?
if not - please make the line look like:
Code: 
Host member1 member1.mydomain.com y.y.y.y

and try again
 
OK there is some progress but...
1) ssh y.y.y.y could not connect at all (let alone without a password)
2) I made the change you suggested and then it did work fine (no pass needed as well)
HOWEVER, same problems.
Running the pmgtunnel with debug on now indicates : "Host key verification failed"

Is the error related to the ssh key? If yes so why it works when I manually run the ssh command and connects?
...or it relates to the "Fingerprint" of the cluster??
:rolleyes:
 
One step closer :)

Please try: `/usr/bin/ssh -o BatchMode=yes -o HostKeyAlias=member1 y.y.y.y`

(That's for the greatest part what pmgtunnel runs)
 
Nikole said:
Host key verification failed
Click to expand...
well - this needs to be addressed - ssh -vvv -o BatchMode=yes -o HostKeyAlias=member1 y.y.y.y should tell you where the offending comes from
- fix that and try again
 
take a look at the ssh -vvv output (or paste it here) - somewhere before the program exits with error it should tell you where it got the not-matching ssh host-key from

I hope this helps!
 
I did run that command however the output below is beyond me...
I am guessing the problem is in here? >>
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts2"

But what should I do is that I don't know.
My nerves are like noodles now... :eek:



---
Code: 
root@masternode:~# ssh -vvv -o BatchMode=yes -o HostKeyAlias=member1 y.y.y.y
OpenSSH_7.4p1 Debian-10+deb9u6, OpenSSL 1.0.2s  28 May 2019
debug1: Reading configuration data /root/.ssh/config
debug1: /root/.ssh/config line 1: Applying options for y.y.y.y
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "y.y.y.y" port 1234
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to y.y.y.y [y.y.y.y] port 1234.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u6
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u6 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to y.y.y.y:1234 as 'root'
debug1: using hostkeyalias: member1
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts2"
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:g8jyn0VbX1zsv5tTanqKRDZ5X2JoXEDbFggbptNKZhcvRM
debug3: put_host_port: [y.y.y.y]:1234
debug1: using hostkeyalias: member1
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts2"
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from [y.y.y.y]:1234
debug3: hostkeys_foreach: reading file "/etc/ssh/ssh_known_hosts2"
Host key verification failed.
root@masternode:~#

---
 
Nikole said:
/root/.ssh/known_hosts:2
Click to expand...
seems like the offending key is in /root/.ssh/known_hosts - line 2 - copy the file away (make a backup) , edit it and remove line 2 - then try again
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom