Why is QDevice setup by PVE backwards?

[esi_y]

The pvecm qdevice setup requires an ssh connection to the QD, which is not there for "casting votes". As QDs are meant to be run externally, why is this not the other (natural) way around, i.e. generating script for one to execute on the QD, possibly provide the feature (as a perk) by calling PVE's API to pull it to the QD?

 

[fabian]

because giving the PVE system root access to the qdevice host is less problematic for fairly obvious reasons than the other way round..

 

[esi_y]

because giving the PVE system root access to the qdevice host is less problematic for fairly obvious reasons than the other way round..

I think i should have phrased myself better, why would there need to be ssh connection to execute it when it's one-off? It's far safer to be connected to the QD by whatever means outside of the cluster knowledge for such operation. I choose and execute the payload that the cluster node gave me and that's it.

Before I am told I can just delete the key after the deed is done, well I would like to e.g. have that in cloud-init for QD then. Basically no ssh conn requirement when not needed. Unless it's needed, but I do not see why.

 

[fabian]

the QD would need to register itself with PVE. that part requires elevated privileges, it entails modifying core settings of the cluster after all. instead of giving that kind of access to the (external, lesser trusted) system running the QD, we execute this part directly on the PVE node in the current scheme, and have the control flow in the other direction.

 

[esi_y]

the QD would need to register itself with PVE. that part requires elevated privileges, it entails modifying core settings of the cluster after all. instead of giving that kind of access to the (external, lesser trusted) system running the QD, we execute this part directly on the PVE node in the current scheme, and have the control flow in the other direction.

Do you mind I open this as separate (wider) Bugzilla issue # with reference (if it stays like it is, should be relevant to currently being implemented SSH changes) and take the discussion further there?

I think currently it only goes for the roundtrip to get the cert signed, which given the fact CA itself is stored at every node of the cluster and the whole exchange requires prior ssh established, even as it goes on to ssh-copy-id (potentially) additional key, should not be the most straightforward way.

Basically I would suggest this is maybe the one case where SSH should not be used at all, also in the light of logic like pvecm {add|addnode}. A more streamlined approach would be:

1. either (for the brave)

qd# curl api | bash

2. or manually

any_node# pvecm addqd $addr > payload.sh
qd# ./payload.sh

3. both options could be provided, but apparently API is done first

I would expect that each command also takes care of installing e.g. qdevice package across nodes and the payload qnetd on the known distros or throws a warning.

 

[fabian]

option 1. is nacked for sure - like I said, we don't want to give the external system access to the PVE cluster in any fashion other than the qnetd<->qdevice connection post-setup.
option 2. has no real benefit - how do you get the payload to the external system? in 99% of the cases via SSH.. qdevice is already a niche feature, we don't want to cater to niche variants of a niche feature in our tooling - our time is better spent on the 99% of common use cases and features/improvements there.

 

[esi_y]

option 1. is nacked for sure - like I said, we don't want to give the external system access to the PVE cluster in any fashion other than the qnetd<->qdevice connection post-setup.

I thought the API is mostly meant to be exposed (subject to one's own firewall security groups) and make any authorized caller happy, also as it is a |, those two steps could be done separately, i.e. from my third machine where I have access to both the cluster and the qd.

option 2. has no real benefit - how do you get the payload to the external system?

Copy & paste? Cloud-init? Anything that is out of scope for the PVE. I mean it's a one-off... e.g. I run QD external and the port 22 will never be open on that machine.

in 99% of the cases via SSH..

I think this is speculation, most cloud providers even give one pty without ssh need.

qdevice is already a niche feature

Judging from support tickets, forum posts or ... I have not seen PVE report home on how many systems e.g. corosync-qdevice unless it's done in upgrade scripts. It might simply not be a source for troubleshooting because it works so well once set up.

, we don't want to cater to niche variants of a niche feature in our tooling - our time is better spent on the 99% of common use cases and features/improvements there.

How likely (%) would you accept an external contributor's patch that allows for this (including other than ffsplit setups which is currently hardcoded)?

 

[fabian]

yes, judging from experience across support channels. it depends on what exactly the changes are - I doubt we'd want to implement lms (as default or recommended option) for example. a well-designed setup API endpoint might be okay, or improvements to the existing CLI command (for example, optionally skipping SSH key enrollment, or just printing the commands needed to run instead of doing them over SSH) might be on the table.

note that you can always customize your setup in the fashion you want - just be aware that you are the one that needs to fix it then if it breaks, and please include a disclaimer/all the relevant info when asking for help in such cases.