New Node 9 trouble migrating to other nodes

[Chris Rivera]

I just added a new node and am having weird issues.

I can migrate to node 9.... but cannot migrate off node 9 to other nodes in the cluster except node 1

Do i need to run a command to update this?


Sep 10 09:47:14 # /usr/bin/ssh -c blowfish -o 'BatchMode=yes' root@63.217.249.153 /bin/true
Sep 10 09:47:14 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Sep 10 09:47:14 @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
Sep 10 09:47:14 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Sep 10 09:47:14 IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Sep 10 09:47:14 Someone could be eavesdropping on you right now (man-in-the-middle attack)!
Sep 10 09:47:14 It is also possible that the RSA host key has just been changed.
Sep 10 09:47:14 The fingerprint for the RSA key sent by the remote host is
Sep 10 09:47:14 92:9c:ed:3d:2d:d5:2b:23:e6:bd:b9:0f:89:28:53:87.
Sep 10 09:47:14 Please contact your system administrator.
Sep 10 09:47:14 Add correct host key in /root/.ssh/known_hosts to get rid of this message.
Sep 10 09:47:14 Offending key in /root/.ssh/known_hosts:3
Sep 10 09:47:14 RSA host key for 63.217.249.153 has changed and you have requested strict checking.
Sep 10 09:47:14 Host key verification failed.
Sep 10 09:47:14 ERROR: migration aborted (duration 00:00:00): Can't connect to destination address using public key
TASK ERROR: migration aborted

 

[tom]

you should be careful here. who changed the keys?

 

[Chris Rivera]

1. I am the only person who manages the cloud and i do not know how to change the keys as of right now. This is not the first time this has happened. I have has this issue previously with nodes and migrations... After trying to migrate over and over i was able to get it to finally migrate.


2. Console on node 9 does NOT work at all. I have to provision on other nodes than migrate to node 9 since i cannot console in.


3. Brute force attack on Proxmox nodes. Do you guys offer a tool or additional software to detect brute force attacks and try to eliminate them? I do see in a log of one of the nodes that i am having brute force attacks being sent. I have changed the password time and time again to ensure that this password does not get cracked, but would like to have a software handle this.

Is there an apt-get install pve-bruteforce app i can use or is there something already built in that i can configure?

 

[Chris Rivera]

I can see that one of the vms on proxmox is the vm that is bruteforce attacking the proxmox node.


is it possible to get the ip address of successful authentications?

Sep 13 12:15:30 proxmox1 pvedaemon[1842]: <root@pam> successful auth for user 'root@pam'


You only provide the ip address for fail authentications... this would help troubleshoot the issue. I need to know that the successful authentication is from internal ips and not a successful hack attempt

Sep 13 12:28:18 proxmox1 pvedaemon[2564]: authentication failure; rhost=63.217.116.51 user=root@pam msg=Authentication failure

 

[dietmar]

I would accept a patch for that.

 

[snowman66]

Hi

if you have problems with brute forcing...i wrote decoder for OSSEC that will automatically block IP for 10 minutes if user fails to login 4 times within 30 seconds.

decoder:

<!--pve-->
<decoder name="pvedaemon">
  <program_name>pvedaemon</program_name>
</decoder>

<decoder name="pve-login-failure">
<parent>pvedaemon</parent>
  <regex offset="after_parent">authentication failure; rhost=(\d+.\d+.\d+.\d+) user=(\w+) msg=(\.+)</regex>
  <order>srcip,user,data</order>
</decoder>
<!--pve-->

rule:

<!-- pve -->
<group name="pve">
<rule id="800020" level="5">
        <if_sid>2501</if_sid>
        <decoded_as>pvedaemon</decoded_as>
        <description>pve login attempt failed</description>
         <options>alert_by_email</options>
</rule>

<rule id="800021" level="12" frequency="2" timeframe="30">
        <if_matched_sid>800020</if_matched_sid>
        <description>pve multiple login failures</description>
 </rule>
</group>
<!-- pve -->

Sample e-mail warning:

OSSEC HIDS Notification.
2012 Sep 14 12:27:53

Received From: (PVE-1) 192.168.1.2->/var/log/syslog
Rule: 800020 fired (level 5) -> "pve login attempt failed"
Portion of the log(s): Sep 14 12:27:51 proxmox-virt01 pvedaemon[258841]: authentication failure; rhost=192.168.41.200 user=root@pam msg=Authentication failure
--END OF NOTIFICATION

log test:

**Phase 1: Completed pre-decoding.
       full event: 'Sep 14 12:27:51 proxmox-virt01 pvedaemon[258841]: authentication failure; rhost=192.168.41.200 user=root@pam msg=Authentication failure'
       hostname: 'proxmox-virt01'
       program_name: 'pvedaemon'
       log: 'authentication failure; rhost=192.168.41.200 user=root@pam msg=Authentication failure'

**Phase 2: Completed decoding.
       decoder: 'pvedaemon'
       srcip: '192.168.41.200'
       dstuser: 'root@pam'
       extra_data: 'Authentication failure'

**Phase 3: Completed filtering (rules).
       Rule id: '800020'
       Level: '5'
       Description: 'pve login attempt failed'
**Alert to be generated.