[SOLVED] Pbs storage - protection from Encrypting ransomware

[Fra]

We (too) need to protect backup storage from encrypting ransomware (aka Cryptolockers), and feel we are not doing enough: we are glad if you can give some advice or even just web links that may help us.

We have an ordinary proxmox pve/pbs setup:
* 2 Proxmox pve with nightly backup on a Pbs (ACL permission role: DataStoreBackup)
* the Pbs is (baremetal) in another region, reachable with a public IP:8007
* one remote pbs that rsync everything

(then we have a secondary backup, limited to some directories of VM and CT, under borg/borgmatic/rsync.net)

Pbs storage is on ZFS, the backup storage is not encrypted: should it be?

Ordinary firewall rules: nothing special.


We also backup some directory of the proxmox hosts, as well, with a proxmox-backup-client basic script: here we need an advice on how to protect the password.

Since we know that a backup client, with ACL permission role: DataStoreBackup can backup itself, but can also read his own backups, we are afraid that an encrypting ransomware could easily reach backups, too (removing them, for example).

 

[dietmar]

Backup your data to a tape, then moves your tapes to an offline vault.

 

[oguz]

hi,

We (too) need to protect backup storage from encrypting ransomware (aka Cryptolockers), and feel we are not doing enough: we are glad if you can give some advice or even just web links that may help us.

1. set up a strong root password for your PVE/PBS and enable 2FA (since your PBS is reachable by public IP)
2. install and configure fail2ban [0]

Pbs storage is on ZFS, the backup storage is not encrypted: should it be?

if you're making encrypted backups then it should be alright

We also backup some directory of the proxmox hosts, as well, with a proxmox-backup-client basic script: here we need an advice on how to protect the password

you can use an API token for that, see our wiki [1]


[0]: https://pve.proxmox.com/wiki/Fail2ban
[1]: https://pbs.proxmox.com/docs/user-management.html#api-tokens

 

[Fra]

> Backup your data to a tape, then moves your tapes to an offline vault.

yes, dietmar, that would be solution, and we will probably have to set it up

> 1. set up a strong root password for your PVE/PBS and enable 2FA (since your PBS is reachable by public IP)

thanks oguz

we could have the PBS to join the VPN where all pve hosts (and CT and VM) are connected, to move the PBS out of the internet: I guess this would help

> 2. install and configure fail2ban [0]

the list of backup client's IP is on the pbs firewall, so I was thinking that fail2ban would not help, but you are right: if a backup client is compromised (or if somebody is spoofing IP), it may be used to brute attack the pbs server to access other backups, so yes, this will help.

> you can use an API token for that, see our wiki [1]

oh, yes, thanks oguz but still I think that, if a backup client is able to send data for its backup it is also able to destroy them: the best would be to have an ACL that allows backup client to only send backup, but not to have access to its own old backups: does this make sense?

 

[Fra]

On thing appears to be 100% sure:

If a VM or CT is compromised, can a ransomware from inside that VM/CT reach the backup in pbs? no, since the backup client is the proxmox host, and not the VM.

am I right?

 

[oguz]

we could have the PBS to join the VPN where all pve hosts (and CT and VM) are connected, to move the PBS out of the internet: I guess this would help

yes, that should also help

oh, yes, thanks @oguz but still I think that, if a backup client is able to send data for its backup it is also able to destroy them: the best would be to have an ACL that allows backup client to only send backup, but not to have access to its own old backups: does this make sense?

if you give your user or API token only the Datastore.Backup permission it's not able to delete the backups (even when it's the owner)

see the corresponding section of the docs [0]

If a VM or CT is compromised, can a ransomware from inside that VM/CT reach the backup in pbs? no, since the backup client is the proxmox host, and not the VM.

am I right?

yes if you're running the client on the PVE side then malicious container/VM won't be able to read the backups (they need to install the backup client and have the credentials, and also have the correct permissions)

[0]: https://pbs.proxmox.com/docs/user-management.html#access-control

 

[Fra]

if you give your user or API token only the Datastore.Backup permission it's not able to delete the backups (even when it's the owner)

see the corresponding section of the docs [0]


yes if you're running the client on the PVE side then malicious container/VM won't be able to read the backups (they need to install the backup client and have the credentials, and also have the correct permissions)

[0]: https://pbs.proxmox.com/docs/user-management.html#access-control

ah, great, I did not get this before: this is super useful, thanks!

 

[Fra]

In case anybody need fail2ban configuration for proxmox-pbs (v. 2.0-7/bullseye): just as in https://pve.proxmox.com/wiki/Fail2ban with these obvious changes:

# /etc/fail2ban/jail.local
< port = https,http,8006
> port = 8007

# /etc/fail2ban/filter.d/proxmox.conf
< failregex = pvedaemon\[.*authentication failure; rhost=<HOST> user=.* msg=.*
> failregex = proxmox-backup-api\[.*authentication failure; rhost=<HOST> user=.* msg=.*

(if anybody has to suggest changes, please do)