I'm creating a system for work to automatically deploy servers with proxmox to then be provisioned and then later decommissioned. The entire point of this system is that I don't pay for server time I don't use, so a lot of my work is creating code to automatically deploy servers on demand. I now have gotten to the point that servers come online with Proxmox running, however, to add them to the cluster I'm required to use my root password? That's insane if that's true, is there no way for me to give that permission to anything other than the key to everything? Can I create a one time ticket to invite a node to join? Use another user's password?
Join a cluster programmatically without using the root password
- Thread starter gido5731
- Start date
Some things are hardcoded to root@pam for security reasons. But I heard they are working on patches to allow tokens and PVE realm users to use some kind of superuser privileges (link is here but its in german).
But according to this you might be able to use the API:
https://forum.proxmox.com/threads/s...it-be-done-the-regular-way.112209/post-484118
But according to this you might be able to use the API:
https://forum.proxmox.com/threads/s...it-be-done-the-regular-way.112209/post-484118
Last edited:
I saw that post but I was wondering if maybe that question was too specific and might've not answered what I had in mind.
I'll say from a scalability and security standpoint, the goal should be to NEVER use the root password, and this just seems flat out stupid. Very much not enjoying my first experiences with proxmox in that regard.
With the API, though, what authentication is done then? I can't find any resources of joining a cluster via the CLI without SSH...
I'll say from a scalability and security standpoint, the goal should be to NEVER use the root password, and this just seems flat out stupid. Very much not enjoying my first experiences with proxmox in that regard.
With the API, though, what authentication is done then? I can't find any resources of joining a cluster via the CLI without SSH...
According to the API documentation (click on the post button at: https://pve.proxmox.com/pve-docs/api-viewer/#/cluster/config/join) it looks like the API also requires root to do a cluster join:
Ugh, that's beyond stupid. The worst part is it seems like there is a setup for this internally with the addnode option in pvecm, however that's "for internal use only" and there isn't any documentation on what it even does.
I hate to sound like "that guy," but anyone who writes something with the idea of scalability without allowing you to provision those permissions to some non-root user should not be making software at scale.
I hate to sound like "that guy," but anyone who writes something with the idea of scalability without allowing you to provision those permissions to some non-root user should not be making software at scale.
@gido5731 You have a few options, none is mutually exclusive of others:
- open a feature request https://bugzilla.proxmox.com/
- submit a patch https://pve.proxmox.com/wiki/Developer_Documentation
- contact Proxmox https://www.proxmox.com/en/about/contact and inform them how many license subscriptions are on the line if something you want is not implemented
- open a feature request https://bugzilla.proxmox.com/
- submit a patch https://pve.proxmox.com/wiki/Developer_Documentation
- contact Proxmox https://www.proxmox.com/en/about/contact and inform them how many license subscriptions are on the line if something you want is not implemented
yes, joining requires the root password since it does quite far-reaching modification on both ends. my comment in the other thread was referring to password-based SSH authentication 
that being said, of course you can do the whole join thing on your own as well (it's not "supported", but in the end, all PVE does on cluster joining is (re)generate/distribute keys and certificates, rewrite some configs and start corosync on the joining node).
code that runs on the joining node (you can see where it makes the call to the internal API endpoint on the existing cluster to add itself there)
code that runs on the existing cluster
as you can see there are quite a few asserts and safeguard in place there to ensure the join operation doesn't break the existing cluster. you also have to be careful w.r.t. timing - corosync really doesn't like it if nodes share a key, but have an inconsistent config.
basically what you'd want to do is:
- prepare joining node/assert everything is okay config wise
- addnode on the existing cluster
- finish_join on the joining node
- check that quorum including newly added node has been established
BUT, and here comes the big caveat, PVE is really not designed for a constantly scaled up and down cluster scenario. the assumption is that nodes rarely get added (when the existing cluster becomes too small, or nodes are replaced with better hardware) or decommissioned (EOL of the whole cluster, replacing with better hardware, rotating out broken hardware).
).
that being said, of course you can do the whole join thing on your own as well (it's not "supported", but in the end, all PVE does on cluster joining is (re)generate/distribute keys and certificates, rewrite some configs and start corosync on the joining node).
code that runs on the joining node (you can see where it makes the call to the internal API endpoint on the existing cluster to add itself there)
code that runs on the existing cluster
as you can see there are quite a few asserts and safeguard in place there to ensure the join operation doesn't break the existing cluster. you also have to be careful w.r.t. timing - corosync really doesn't like it if nodes share a key, but have an inconsistent config.
basically what you'd want to do is:
- prepare joining node/assert everything is okay config wise
- addnode on the existing cluster
- finish_join on the joining node
- check that quorum including newly added node has been established
BUT, and here comes the big caveat, PVE is really not designed for a constantly scaled up and down cluster scenario. the assumption is that nodes rarely get added (when the existing cluster becomes too small, or nodes are replaced with better hardware) or decommissioned (EOL of the whole cluster, replacing with better hardware, rotating out broken hardware).
sounds like you are running PVE on some sort of VPS/cloud environment/.. ? while this does/can work, it's also not what PVE is usually used for (bare metal installations, providing this kind of abstraction to guests, not itself being such a guest
).Imo it doesn't matter how far reaching it is, everything should be able to be delegated to more useful user levelsyes, joining requires the root password since it does quite far-reaching modification on both ends. my comment in the other thread was referring to password-based SSH authentication
I actually considered this but, due to timing, I didn't have the time to go dig though another unfamiliar codebase
That does fit with what I've seen so far, but I don't really see why this is (or why it really matters, but it does appear to
This is a misconception, I am renting full bare metal servers by the hour. No VPSs involved. That's why I'm using Equinix's platform (which I thought I mentioned earlier, oops) to deploy on bare metal, not AWS or something more traditionalsounds like you are running PVE on some sort of VPS/cloud environment/.. ? while this does/can work, it's also not what PVE is usually used for (bare metal installations, providing this kind of abstraction to guests, not itself being such a guest
That being said, thanks for the input, but honestly my past day or so with Proxmox is making it seem really unworkable, and I am not planning to continue using it, instead pivoting most likely to raw libvirt, which has the features I need more properly fleshed out and that I am more familiar with.