Should an official Proxmox "Hardening" wiki page be created?

justinclift

Active Member
Apr 1, 2024
421
102
43
As a general thought, I'm wondering if an official Proxmox Hardening wiki page would be useful?

Maybe placed here or similar?

https://pve.proxmox.com/wiki/Hardening

Asking because hardening a server (or cluster thereof) isn't rocket science, but seems to be under documented apart from various random articles on peoples blogs.

So my thought is having some official guidelines (preferably Best Practices) and/or process would probably be a useful thing.

Thoughts? :)
 
Yes, even some overview pointing to guides would be useful. I guess there are tons of documentations for Debian security hardening out there that should also apply for PVE/PBS. Like making use of public private keys instead of passwords for SSH, fail2ban, proper monitoring, log collectors, SIEM, installing microcode and keeping BIOS + BMC firmware up to date, unattended-upgrades or at least a note to keep PVE properly upgraded (especially a hint that PVE won't find upgrades once it is EoL so people don'T think all is up to date while it isn't), making use of DMZs, dedicated management interface, air gapping using the proxmox offline mirror, how to best keep informed about important CVEs hitting PVE, ... this would be a massive article.
 
  • Like
Reactions: justinclift
first hardening : do not open host to public, built-in firewall to allow only whitelisted ip or vpn.
 
As a general thought, I'm wondering if an official Proxmox Hardening wiki page would be useful?

1. This would be impossible to do, would turn into a farce. PVE is architected with the idea that it runs on a separate VLAN.


2. If you read some of the bugreports (never rotated CAs, keys, CSFR approach, etc.), how PVE team approaches security risks (CVEs) or how beta the "non-subscription" repo is, publishing something like that would just open the company to liability for implying anything other than (1) above.

Asking because hardening a server (or cluster thereof) isn't rocket science, but seems to be under documented apart from various random articles on peoples blogs.

3. You should not have the nodes open to any kind of public access, with external firewall as you never know what an update could do due to the (2) above. There's no iptables, no prohibit-password, everything runs under root, no at-rest-encryption, etc. even out of the box. Following some kind of Debian howto will just give you sense of false security thinking you do not have to do (1) above.

So my thought is having some official guidelines (preferably Best Practices) and/or process would probably be a useful thing.

4. In fact, do not even open anything virtio to the public except via external reverse proxies.

5. Even just sharing the cluster resources amongst 3rd parties is also asking for trouble.

Thoughts? :)

Sorry.
 
Last edited:
  • Like
Reactions: IsThisThingOn
Isn't the purpose of the non-subscription repo to pretty much be a shakedown test of built packages?

Just consider this, when one visits the Downloads page [1], they only get the latest of e.g. v8.2 as of today. If you wanted e.g. v8.1, you would need to go to not that well search-indexed location [2]. I would infer that this and the very fact something is not called e.g. unstable but no-subscription' is a conscious choice.

Uh oh. Sounds like I've missed that in the documentation.

Not that I know of.

Is the mention of that somewhere obvious you could point me at? :)

These security-kind-of-questions often reappear and then the OP is sort of made into a scapegoat for even asking [3] - it's not that the OP's point is always relevant , but they do not ask to get mocked. They would not be asking if it had been documented. Might that be a conscious decision too?

[1] https://www.proxmox.com/en/downloads
[2] https://enterprise.proxmox.com/iso/
[3] https://forum.proxmox.com/threads/security-hardening.134055/
 
I would infer that this and the very fact something is not called e.g. unstable but no-subscription' is a conscious choice.
That seems like a super weird take to me.

For another perspective "Fedora Linux" is widely regarded as the shake down / end user testing for what ends up in RHEL.

The current Proxmox approach of "no-subscription" and "enterprise" repo seems to broadly do the same kind of thing.

For a Hardening guide, it'd make sense to have a pretty prominent item about "Get a Proxmox subscription and use the Enterprise repositories".

That should help reduce Weird Shit in the less stable no-subscription repo from mucking things up. In theory. :)



Just read through the entire thread you're using as an example of someone being mocked and scapegoated for even asking.

Um... is that the right link? I'm not seeing anyone being mocked or scapegoated at all in that one. :confused:
 
:) @LnxBil Ahhh. My reading of that is "not as experimental as the test repo, but still not as real world tested as the packages that get to the enterprise repo".
 
That seems like a super weird take to me.

See below.*

For another perspective "Fedora Linux" is widely regarded as the shake down / end user testing for what ends up in RHEL.

I do not think it's a good analogy, Fedora is basically exploring around (consider the flavours like Silverblue even) and overall appears to me rather independent (they stick to BTRFS last time I remember even as RHEL ditched it altogether for the reputational risk).

The current Proxmox approach of "no-subscription" and "enterprise" repo seems to broadly do the same kind of thing.

* So that was a hypothesis, but how else would one explain it's not called something that gives away the fact that it's beta. I understand there's testing one officially (should be called unstable), but when new features are implemented on the mailing lists, lots of times the alpha is like "has anyone tried this - oh yeah it worked ok in the default config, let's ship to no-subscription" - I do not think repo name was chosen as a coincidence. The price you pay is being (arguably) involuntary beta tester. It's one thing to have a no support or community flavour of something, it's another to end up reporting regressions that even basic automated tests would have revealed if it was led more rigorously.

For a Hardening guide, it'd make sense to have a pretty prominent item about "Get a Proxmox subscription and use the Enterprise repositories".

That's the thing, the enterprise label is there to imply it's really necessary if PVE is meant as business critical, i.e. should not be risking running no-subscription - no one should get fired for CVE fallout if they paid anything "enterprise", right? But it really does not offer anything more than best-effort SLA. Again, hardening in this sense would serve what purpose? To have it open to the internet? Something it was never designed for ...

Just read through the entire thread you're using as an example of someone being mocked and scapegoated for even asking.

Um... is that the right link? I'm not seeing anyone being mocked or scapegoated at all in that one. :confused:

It is the right link from a quick search, no @LnxBil was not mocking anyone, he is one of the decent ones on the forum. One could find other posts which are worse. You find a design flaw and you are using it wrong, you report a security issue and it is not meant to be exposed (i.e. your fault). Then some promises are given (we will not fix this code, we will not refactor it, we will rewrite it from scratch to abandon the original concept), then it takes forever, then it gets forgotten.

But to sum it up, I do not see PVE being focused on security, it ships more "insecure" than plain Debian out of the box even. You can open a topic about sudo, apparmor, etc, but in the end, it is concluded it adds complexity for no additional (other than perceived) benefit. I think the team is too small to focus on all those things. I do not think there's anyone really security-first-focused. One person had higher than average awareness, but the focus is on getting new features in. The core features. Proxmox is not in the infosec business.
 
  • Like
Reactions: IsThisThingOn
What is your real goal? You can install a local package mirror and edit /etc/apt/sources.list. Then internet access for the Server is not requiered. (sorry for typing, I'm native german)
 
The goal is having secure systems in a local data centre which host live, internet facing VMs. Thus the hosts themselves are internet facing too, or should at least be considered as such.

Proper hardening isn't rocket science, but it sounds like the Proxmox testing and packaging process has enough gaps that extra care needs to be taken. Need to make sure those gaps don't fuck things up for people over time who use Proxmox servers on the wider internet. ;)
 
Proper hardening isn't rocket science, but it sounds like the Proxmox testing and packaging process has enough gaps that extra care needs to be taken. Need to make sure those gaps don't fuck things up for people over time who use Proxmox servers on the wider internet. ;)
As already said: Don't face them to the internet (directly)... there are plenty of technologies available ranging from VPN to TLS client certificates, that need to be deployed around your infrastructure.

VLAN you stuff properly (at least this 3-layer solution):
  • hardware mangagement interfaces (iLO/idrac/irmc/IPMI, switches, UPSes, etc.)
  • hypervisor management and backup thereof
  • all other (maybe additional VLANs to seperate by concerns
Each layer should be independent from each other and NOT accessible at all.
 
  • Like
Reactions: IsThisThingOn
Each layer should be independent from each other and NOT accessible at all.

I would just add: take care not to have any sort of VLAN tagging to be possible done by the VMs themselves.

I think the OP's concern is relevant in the sense that there really is no FAQ section how to run PVE "professionally" whatsover. At least I do not know of any assumptions page at the least. Then people make lots of assumptions of their own, every time new forum post...
 
  • Like
Reactions: justinclift
Interesting. Thus far, I've been avoiding VLANs in an attempt to keep things simple.

In my head (so far) VLANs are a management thing (separation of concerns style), as they don't seem to offer any additional security.

As far as I'm aware, putting something in a vlan doesn't make its traffic magically "invisible" to sniffing over the wire. aka it's not IPsec

The traffic is still as unencrypted as it was before, unless the traffic is independently encrypted in some form (https/ssh/ssl/etc).

Is that the wrong way to look at it?
 
Last edited:
there really is no FAQ section how to run PVE "professionally" whatsover.
Yeah, that's pretty much why I'm thinking of this topic.

There's a bunch of random stuff on the net (example).

But there's nothing definitive, let alone any kind of Best Practices that have been developed over time. :(
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!