At a loss added ssh key via API maybe some bug ?

[Craig St George]

I have a lot of things working via the API create clone config an others all works good.
I then tried to add and SSH key via the API for a QUEM machine and get invalid urlencoded string

So I tried via pvesh set /nodes/promox1/qemu/317/config -sshkeys ....... and found as long as I used RFC 3986 encoding it works fine eg % signs
I copied the exact encoded key that worked on the command line and sent via the API and it still errors out.

So digging deeper I found JSONSchema.pm and changed the message in there for a test.
Via the API I still get invalid urlencoded string via the command line I get the modified message.

Restarted pveproxy still get the original message weird so I m not sure if there a different test for that via the API or if I need to restart some other service.

I only have one node as its my DEV system so I know I m connecting to the right one.

So right now I have 2 thoughts
1) there is a different test for the urlencoded string when using the API ?
2) The amount of data is too long and the HTTP server is some how truncating that and hence the error

Any ideas at all ?

thanks

 

[dcsapak]

what exactly to you send to the api? and what is the exact error message?

 

[Craig St George]

Sorry for all the data

if I use curl
curl --silent --insecure --cookie "$(<cookie)" --header "$(<csrftoken)" -X POST\
--data-urlencode sshkeys="ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAghS7G6XN8JnsLRUFZA2PRh84oo031Xz5tnbc8Q0nzgSOfOhoc8Az89xFwxnOyEJyhEKKjo0G00jz5GJtRfCA4q4TwzTtTe+wlLl7POSxwTm70n57WKPHoLvhMlCvspBr7xwYL+PnF/ngAkjchBZ0cWNPDEaKk0dQMS36En5aWFIvVQ3gIrpyM1uH95W4C/F20LnLd3ej/nXb1F6786Wo8/vsHKs+KX89wlQsy0o2pcwuR/fVCKDn2B6zGKvFpvFpy/o0KaqVGVef75A1n6VMS27vCBmOKpGxqR4fRo38dgFTTYCYviMvhx2NoDAWTmdrquikJfJ0KIOUL6EOYHRu2w== testkey@test.com" https://$APINODE:8006/api2/json/nodes/$TARGETNODE/qemu/317/config

I get
{"errors":{"sshkeys":"invalid format - invalid urlencoded string: ssh-rsa AAAAB etc

If I try via pvesh with the encoded key it works
pvesh set /nodes/promox1/qemu/317/config -sshkeys ssh-rsa%20AAAAB3NzaC1yc2EAAAABJQAAAQEAghS7G6XN8JnsLRUFZA2PRh84oo031Xz5tnbc8Q0nzgSOfOhoc8Az89xFwxnOyEJyhEKKjo0G00jz5GJtRfCA4q4TwzTtTe%2BwlLl7POSxwTm70n57WKPHoLvhMlCvspBr7xwYL%2BPnF%2FngAkjchBZ0cWNPDEaKk0dQMS36En5aWFIvVQ3gIrpyM1uH95W4C%2FF20LnLd3ej%2FnXb1F6786Wo8%2FvsHKs%2BKX89wlQsy0o2pcwuR%2FfVCKDn2B6zGKvFpvFpy%2Fo0KaqVGVef75A1n6VMS27vCBmOKpGxqR4fRo38dgFTTYCYviMvhx2NoDAWTmdrquikJfJ0KIOUL6EOYHRu2w%3D%3D%20testkey%40test.com
update VM 317: -sshkeys ssh-rsa%20AAAAB3NzaC1yc2EAAAABJQAAAQEAghS7G6XN8JnsLRUFZA2PRh84oo031Xz5tnbc8Q0nzgSOfOhoc8Az89xFwxnOyEJyhEKKjo0G00jz5GJtRfCA4q4TwzTtTe%2BwlLl7POSxwTm70n57WKPHoLvhMlCvspBr7xwYL%2BPnF%2FngAkjchBZ0cWNPDEaKk0dQMS36En5aWFIvVQ3gIrpyM1uH95W4C%2FF20LnLd3ej%2FnXb1F6786Wo8%2FvsHKs%2BKX89wlQsy0o2pcwuR%2FfVCKDn2B6zGKvFpvFpy%2Fo0KaqVGVef75A1n6VMS27vCBmOKpGxqR4fRo38dgFTTYCYviMvhx2NoDAWTmdrquikJfJ0KIOUL6EOYHRu2w%3D%3D%20testkey%40test.com

If I try via the API from my provisioning system it also fails

$returnCode = 400
$SendData = sshkeys=ssh-rsa%20AAAAB3NzaC1yc2EAAAABJQAAAQEAghS7G6XN8JnsLRUFZA2PRh84oo031Xz5tnbc8Q0nzgSOfOhoc8Az89xFwxnOyEJyhEKKjo0G00jz5GJtRfCA4q4TwzTtTe%2BwlLl7POSxwTm70n57WKPHoLvhMlCvspBr7xwYL%2BPnF%2FngAkjchBZ0cWNPDEaKk0dQMS36En5aWFIvVQ3gIrpyM1uH95W4C%2FF20LnLd3ej%2FnXb1F6786Wo8%2FvsHKs%2BKX89wlQsy0o2pcwuR%2FfVCKDn2B6zGKvFpvFpy%2Fo0KaqVGVef75A1n6VMS27vCBmOKpGxqR4fRo38dgFTTYCYviMvhx2NoDAWTmdrquikJfJ0KIOUL6EOYHRu2w%3D%3D%20testkey%40test.com
$action = /nodes/promox1/qemu/317/config
$this->apiurl = https://promox1.iwv.works:8006/api2/json

$apiResponse = {"errors":{"sshkeys":"invalid format - invalid urlencoded string: ssh-rsa

I found with the pvesh I had to use % encoding so I made sure my Provisioning system was doing that. as that is what
-data-urlencode in curl is doing I did also try the other way RFC 1738
All other config commands like cipassword etc all work fine

I hope you can replicate the error and it seems only the API as like I said I edited the error message in that
/usr/share/perl5/PVE/JSONSchema.pm and when I use pvesh I see the edited message but when I use API i see the original one so weird

 

[dcsapak]

can you try to send the urlencoded string with --data-urlencode ? (so double url encoded)

 

[Craig St George]

Yes that works now

 

[Dragoshi]

Hello,

I'm having the same problem but i'm newer to Python. Can you help me out?

What i have so far:

import urllib

cluster = "<cluster_name>"
node = "prox3"
vmID = 104
sshKey = "ssh-rsa AAAAB3<blah blah blah> user@domain.com"

auth = wrdConnProxmox(cluster)

data = {}
data["sshkeys"] = sshKey
data = urllib.parse.urlencode(data)
print(data)
result = auth.wPost("nodes/" + str(node) + "/qemu/" + str(vmID) + "/config", data)
print(result)

And the result:

sshkeys=ssh-rsa+AAAAB3NzaC1yc2EAAAADAQABAAACAQDP3MmOrvy0mz7SBpxfTvQoO%2Focy35dz32%2BIbGPaAHp6NGaZYXUTjVXV8Z1N1gmfN3qFupYBs%2Fn3NNXvBZ4sadrfcfwSTPVGb1jR60CKv7nb5klDh8M%2B40O6TJXI4%2B791feCwU0OdUSPoNMZg1pU4zHfZ9m2nbDj9qzHslZ46KcoYczfOhYfkA15b2WRpZQVtFWb%2F5CmFwWOLw7o54f17mJqiXTEVVWvFwOOkuUGm9Lp3Cg9ovJd8LRVXfVJ4pCbm6AdX9gb7nI8FU2SdSOrPcj84m3%2FL01ViYHDvoJbfW5qNx7tstjWXafQMfmgC3o6zDSN8LJGJWr9X%2F%2BOj3yYKiWRfA6EgnMWBJhPE98B12x8DpRCIf%2BBXQ04%2FWgn3odO4XFPGRyqYnUM8EGn9S2FL%2Bquk%2BkXhrFlD335MPXWTRP1l9ndHsFn9XjJRJk%2BbnebK%2FCiqaznUxM401y%2FoFzngYIKCDE1LagscbIOoxSSfEoRPziagWlosd3cYLgYhB1pretuN%2F45o%2B%2FNhOhhJtySSaxfinF%2BKHD2BvCLXOuA3BmZq%2FBJ8rzFVdVHxw%2FRquQBwDOg0dh2fc3Ysl1Waao1D9EVoDjgajNi4iMOlUi%2Bz1pf97y%2Ffx7wrfv1lCl4zvoCHwB1BRcTRSBpXAdC8pDITTTfYVCBOnCE%2Bn7CiN%2FqvhoWcQ%3D%3D+user%40domain.com

{'data': None, 'errors': {'sshkeys': 'invalid format - invalid urlencoded string: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDP3MmOrvy0mz7SBpxfTvQoO/ocy35dz32+IbGPaAHp6NGaZYXUTjVXV8Z1N1gmfN3qFupYBs/n3NNXvBZ4sadrfcfwSTPVGb1jR60CKv7nb5klDh8M+40O6TJXI4+791feCwU0OdUSPoNMZg1pU4zHfZ9m2nbDj9qzHslZ46KcoYczfOhYfkA15b2WRpZQVtFWb/5CmFwWOLw7o54f17mJqiXTEVVWvFwOOkuUGm9Lp3Cg9ovJd8LRVXfVJ4pCbm6AdX9gb7nI8FU2SdSOrPcj84m3/L01ViYHDvoJbfW5qNx7tstjWXafQMfmgC3o6zDSN8LJGJWr9X/+Oj3yYKiWRfA6EgnMWBJhPE98B12x8DpRCIf+BXQ04/Wgn3odO4XFPGRyqYnUM8EGn9S2FL+quk+kXhrFlD335MPXWTRP1l9ndHsFn9XjJRJk+bnebK/CiqaznUxM401y/oFzngYIKCDE1LagscbIOoxSSfEoRPziagWlosd3cYLgYhB1pretuN/45o+/NhOhhJtySSaxfinF+KHD2BvCLXOuA3BmZq/BJ8rzFVdVHxw/RquQBwDOg0dh2fc3Ysl1Waao1D9EVoDjgajNi4iMOlUi+z1pf97y/fx7wrfv1lCl4zvoCHwB1BRcTRSBpXAdC8pDITTTfYVCBOnCE+n7CiN/qvhoWcQ== user@domain.com\n'}}

I don't know what you guys talk about double urlencode.

 

[Dragoshi]

Yes that works now

Can you have a look at my below post, please? Help would be much apreciated

 

[Craig St George]

you need to encode the key I guess in Python
data["sshkeys"] =urllib.parse.urlencode(sshKey)

then you encode data when you send to the API hence the key is double encoded

 

[Dragoshi]

you need to encode the key I guess in Python
data["sshkeys"] =urllib.parse.urlencode(sshKey)

then you encode data when you send to the API hence the key is double encoded

If i do this, it raises TypeError:

raise TypeError
TypeError: not a valid non-string sequence or mapping object

That's why i was encoding the whole "data" variable

 

[Craig St George]

Remove the second one that encodes the $data as I assume what u post to the APi maybe is also encoding it. Eg just try the one that encodes the key.

And confirm. Everything use curl as a test then make u code do the same as curl is doing

 

[Dragoshi]

I'm gonna let this here for everyone might have the same problem. The trick is:

from urllib.parse import quote

sshKey = "ssh-rsa <blah blah blah key> user@domain.com"
sshKey = quote(sshKey, safe='')
data = {}
data["sshkeys"] = sshKey

auth.wPost("nodes/"+str(node)+"/qemu/"+str(vmID)+"/config",data)

The magic happens in that

safe=''

argument

 

[LexyJexon]

I have the same problem, but I work with proxmox api using proxmox-api-go package. Help me please with encoding ssh-key into urlencode in golang