GUI Manager disconnecting every minute and extremely weird process on loop

[silca80]

Hello, I will appreciate any guidance on this problem

Proxmox PVE 7.2 fresh installed and patched full install (Several VMs and PFsense) working good and responsive


Once the server went to the data center, some changes need to be made (IPs DNS, PFsense GTW, and DNSs)
Hours later everything start to get complicated. GUI starts to show disconnections every minute and returns very laggy.
NoVNC same issue. At this point VMs working but impossible to use since timeout and console restarting were the result.

Looking at the traffic I manage to detect a lot of packets dropped by PFsense and UFW (Proxmox Firewall was deactivated)
I proceed to shut down all VMs, deactivate UFW, shut off PFsense Activate Proxmox firewall allowing ports 22 and 8006 only to gain access
The result was the same, losing connectivity every 60 seconds and dropping packets despite no firewall rules active
Ping (enable ICPM) was intermittent and this makes me notice some weird process running in batch from nowhere.
Literally, those processes are like command line things (ps, find, whoami, ls, etc..etc) that saturate the CPU and the Net out traffic.
Using Htop I manage to kill the main one and after that Proxmox started to respond but just 60 seconds when another command in batch saturates the CPUs
and Netout traffic.


My question is any one seen this before?
Any clues on how these commands are trigered?

My network interfaces look like this

auto lo
iface lo inet loopback

iface eno1 inet manual
#Slave Port Connected

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

iface ens1f0 inet manual

iface ens1f1 inet manual

auto vmbr0
iface vmbr0 inet static
address 66.187.71.186/29
gateway 66.187.71.185
bridge-ports eno1
bridge-stp off
bridge-fd 0
#OPtlin.Fix IP

auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr2
iface vmbr2 inet static
address 10.99.99.200/24
bridge-ports none
bridge-stp off
bridge-fd 0

Please let me know if any additional information should be provided.
Thanks in advance

Silvio

 

[spirit]

proxmox code don't do any whoami, find ,...

are you sure you don't got a ransomware, crypto miner or something like this ?

can you send the output of "ps -aux" ? to see all process running

 

[silca80]

Hello!!! thanks for your time to answer!!! Tourns out like yes was a crypto attack, since a bunch of files appears in cron tab jobs and also in /usr/bin with some unknown names all of the executables and when you delete them they are getting back

Now after closing all the ports and deleting all users - only root with an extreme password is in place and looks like is a little better.

Still, my hunch is the server is fully compromised and I think a full re-install should be mandatory.

Now I'm more concerned with how to perform a remote install and save the VMS and config living in a couple of ZRaid1 arrays inside.

ps -aux output is attached since was pretty messy to post and somehow big will this post be difficult to read

I will appreciate any clue or toughs in a possible approach to get the best solution to this issue.

Thanks again..!!

Silvio