Join a cluster programmatically without using the root password

gido5731

New Member
Jul 17, 2022
5
0
1
I'm creating a system for work to automatically deploy servers with proxmox to then be provisioned and then later decommissioned. The entire point of this system is that I don't pay for server time I don't use, so a lot of my work is creating code to automatically deploy servers on demand. I now have gotten to the point that servers come online with Proxmox running, however, to add them to the cluster I'm required to use my root password? That's insane if that's true, is there no way for me to give that permission to anything other than the key to everything? Can I create a one time ticket to invite a node to join? Use another user's password?
 
Some things are hardcoded to root@pam for security reasons. But I heard they are working on patches to allow tokens and PVE realm users to use some kind of superuser privileges (link is here but its in german).
But according to this you might be able to use the API:
https://forum.proxmox.com/threads/s...it-be-done-the-regular-way.112209/post-484118
yeah, that works. the only thing in PVE that uses password based auth is joining a node to an existing cluster using SSH (which is the non default mode, the default one goes over the API).
 
Last edited:
I saw that post but I was wondering if maybe that question was too specific and might've not answered what I had in mind.

I'll say from a scalability and security standpoint, the goal should be to NEVER use the root password, and this just seems flat out stupid. Very much not enjoying my first experiences with proxmox in that regard.
With the API, though, what authentication is done then? I can't find any resources of joining a cluster via the CLI without SSH...
 
Ugh, that's beyond stupid. The worst part is it seems like there is a setup for this internally with the addnode option in pvecm, however that's "for internal use only" and there isn't any documentation on what it even does.
I hate to sound like "that guy," but anyone who writes something with the idea of scalability without allowing you to provision those permissions to some non-root user should not be making software at scale.
 
I'll hopefully get to the first one today, and normally I would look into making a patch if I didn't have a deadline so soon.
And, unfortunately, my org is not a license holder as of now, but maybe eventually they'll earn that too.
 
yes, joining requires the root password since it does quite far-reaching modification on both ends. my comment in the other thread was referring to password-based SSH authentication ;)

that being said, of course you can do the whole join thing on your own as well (it's not "supported", but in the end, all PVE does on cluster joining is (re)generate/distribute keys and certificates, rewrite some configs and start corosync on the joining node).

code that runs on the joining node (you can see where it makes the call to the internal API endpoint on the existing cluster to add itself there)
code that runs on the existing cluster

as you can see there are quite a few asserts and safeguard in place there to ensure the join operation doesn't break the existing cluster. you also have to be careful w.r.t. timing - corosync really doesn't like it if nodes share a key, but have an inconsistent config.

basically what you'd want to do is:
- prepare joining node/assert everything is okay config wise
- addnode on the existing cluster
- finish_join on the joining node
- check that quorum including newly added node has been established

BUT, and here comes the big caveat, PVE is really not designed for a constantly scaled up and down cluster scenario. the assumption is that nodes rarely get added (when the existing cluster becomes too small, or nodes are replaced with better hardware) or decommissioned (EOL of the whole cluster, replacing with better hardware, rotating out broken hardware).

I don't pay for server time I don't use
sounds like you are running PVE on some sort of VPS/cloud environment/.. ? while this does/can work, it's also not what PVE is usually used for (bare metal installations, providing this kind of abstraction to guests, not itself being such a guest ;)).
 
yes, joining requires the root password since it does quite far-reaching modification on both ends. my comment in the other thread was referring to password-based SSH authentication ;)
Imo it doesn't matter how far reaching it is, everything should be able to be delegated to more useful user levels

that being said, of course you can do the whole join thing on your own as well (it's not "supported", but in the end, all PVE does on cluster joining is (re)generate/distribute keys and certificates, rewrite some configs and start corosync on the joining node).
I actually considered this but, due to timing, I didn't have the time to go dig though another unfamiliar codebase
code that runs on the joining node (you can see where it makes the call to the internal API endpoint on the existing cluster to add itself there)
code that runs on the existing cluster

as you can see there are quite a few asserts and safeguard in place there to ensure the join operation doesn't break the existing cluster. you also have to be careful w.r.t. timing - corosync really doesn't like it if nodes share a key, but have an inconsistent config.

basically what you'd want to do is:
- prepare joining node/assert everything is okay config wise
- addnode on the existing cluster
- finish_join on the joining node
- check that quorum including newly added node has been established

BUT, and here comes the big caveat, PVE is really not designed for a constantly scaled up and down cluster scenario. the assumption is that nodes rarely get added (when the existing cluster becomes too small, or nodes are replaced with better hardware) or decommissioned (EOL of the whole cluster, replacing with better hardware, rotating out broken hardware).
That does fit with what I've seen so far, but I don't really see why this is (or why it really matters, but it does appear to
sounds like you are running PVE on some sort of VPS/cloud environment/.. ? while this does/can work, it's also not what PVE is usually used for (bare metal installations, providing this kind of abstraction to guests, not itself being such a guest ;)).
This is a misconception, I am renting full bare metal servers by the hour. No VPSs involved. That's why I'm using Equinix's platform (which I thought I mentioned earlier, oops) to deploy on bare metal, not AWS or something more traditional



That being said, thanks for the input, but honestly my past day or so with Proxmox is making it seem really unworkable, and I am not planning to continue using it, instead pivoting most likely to raw libvirt, which has the features I need more properly fleshed out and that I am more familiar with.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!