API Token Problems

[jhollowe]

I'm one of the maintainers of the proxmoxer python library and I'm trying to get it working with PBS. I have several issues when trying to get the API tokens working. They work perfectly with PVE, but there seem to be issues/differences with PBS.

1. Why is the separator for the token id from the value different between PVE and PBS? PVEAPIToken=<user>@<realm>!<token_name>=<token_value> vs PBSAPIToken=<user>@<realm>!<token_name>:<token_value>. pbs code showing colon, pbs docs, and pve docs.
2. With my testing using PBS 1.0-1 (ISO install), I can't get any API Token to work. The contents of the "Authorization" header is PBSAPIToken=root@pam!test1<separator>e4bed930-8720-451b-b040-a342042f00b8 . Using a "=" separator gives me the error 401 authentication failed - invalid token name, using a ":" separator gives me the error 400 invalid realm in user id, using "::" as the separator gives me the error 401 authentication failed - invalid credentials, and passing only PBSAPIToken=root@pam!test1 gives the error 401 authentication failed - failed to split API token header.
3. I saw that the api viewer data has not been exported yet. Is that something that I (or others in the community) could help with?

 

[fabian]

I'm one of the maintainers of the proxmoxer python library and I'm trying to get it working with PBS. I have several issues when trying to get the API tokens working. They work perfectly with PVE, but there seem to be issues/differences with PBS.

1. Why is the separator for the token id from the value different between PVE and PBS? PVEAPIToken=<user>@<realm>!<token_name>=<token_value> vs PBSAPIToken=<user>@<realm>!<token_name>:<token_value>. pbs code showing colon, pbs docs, and pve docs.

IIRC this was an oversight, both could easily be made to accept either, but since the prefix/authentication realm is (and should be!) different anyhow a client needs to build the full string anyway based on a product switch..

1. With my testing using PBS 1.0-1 (ISO install), I can't get any API Token to work. The contents of the "Authorization" header is PBSAPIToken=root@pam!test1<separator>e4bed930-8720-451b-b040-a342042f00b8 . Using a "=" separator gives me the error 401 authentication failed - invalid token name, using a ":" separator gives me the error 400 invalid realm in user id, using "::" as the separator gives me the error 401 authentication failed - invalid credentials, and passing only PBSAPIToken=root@pam!test1 gives the error 401 authentication failed - failed to split API token header.

there was a bug earlier that only accepted ' ' as separator between PBSAPIToken and the rest, but recent versions accept both '=' and ' ' there.

which API path are you trying? some API paths will only work with users, not with tokens (e.g. /ticket, the user-related API, ..), and the message invalid realm in user id with status 400 sounds like an API call that first tried to extract a user from the logged in entity and failed (to parse the tokenid as userid). if you give me more details, I can try to look into making the error message more explicit.

1. I saw that the api viewer data has not been exported yet. Is that something that I (or others in the community) could help with?

I think @t.lamprecht already started working on something there, so hopefully we'll see some progress (/a working apidump + viewer) in January. it's just a bit of work to get all the machinery in place, and it's only relevant for a smaller subset of users so it got put on the back-burner.

do you need the dump as input to autogenerate bindings, or just to get an overview over what is where? if the latter, feel free to ask questions here or on pbs-devel if something is not obvious. the API endpoints are all in https://git.proxmox.com/?p=proxmox-backup.git;a=tree;f=src/api2 , and the #[api( macro defines the schema with parameters and return types, needed permissions, etc. sometimes you'll just see type: Foo, then that type is defined somewhere and has its own api macro invocation defining the schema, and sometimes you'll see schema: SOME_CONSTANT, then that constant is defined somewhere and defines the schema. most of those are in src/api2/types.rs

 

[jhollowe]

IIRC this was an oversight, both could easily be made to accept either, but since the prefix/authentication realm is (and should be!) different anyhow a client needs to build the full string anyway based on a product switch.

It would be nice to allow it to use "=" as a separator for the token value. While the client does have to rebuild the string, it would make interacting with the APIs nicer if they all follow the same rules for formatting.

which API path are you trying?

I got other paths working correctly. I was using /access/users and it was giving me invalid realm in user id even when it should have access to that path:

It looks like the ACL for this path might be broken. For example, this should work, but does not.

curl --location --insecure --request GET 'https://<ip>:8007/api2/json/access/users' --header 'Authorization: PBSAPIToken=root@pam!test1:e4bed930-8720-451b-b040-a342042f00b8'
# returns "invalid realm in user id"

The screenshot also shows "Group" as an identifier, but (at least in the webUI) there is no way to create groups. Is this a copy/paste error or a feature not yet fully implemented?

It would also be quite helpful if the error message for the incorrect HTTP method on a correct path would return an error different from a completely non-existent path. I know that might be difficult with the way the API code is designed though. Maybe even just returning the method in the error would help. I thought /access/password was broken until I looked at the code and saw it was a PUT, not a GET.

if you give me more details, I can try to look into making the error message more explicit.

/access/password (and the broken /access/users) returns invalid realm in user id. These should probably say in some way the API token cannot be used for this. It would be good to also have a different message if the user id is valid, but doesn't have permission for that path.

do you need the dump as input to autogenerate bindings, or just to get an overview over what is where?

Just to test it. Proxmoxer just makes calls to the path the user specifies, so I don't need to make any mappings ahead of time.

 

[fabian]

It would be nice to allow it to use "=" as a separator for the token value. While the client does have to rebuild the string, it would make interacting with the APIs nicer if they all follow the same rules for formatting.

yeah, maybe we'll do that..

I got other paths working correctly. I was using /access/users and it was giving me invalid realm in user id even when it should have access to that path:
View attachment 22330
It looks like the ACL for this path might be broken. For example, this should work, but does not.

curl --location --insecure --request GET 'https://<ip>:8007/api2/json/access/users' --header 'Authorization: PBSAPIToken=root@pam!test1:e4bed930-8720-451b-b040-a342042f00b8'
# returns "invalid realm in user id"

yeah, this is intentional although the message is not clear. user management is restricted to proper users (for the time being).

The screenshot also shows "Group" as an identifier, but (at least in the webUI) there is no way to create groups. Is this a copy/paste error or a feature not yet fully implemented?

Groups are not yet implemented.

It would also be quite helpful if the error message for the incorrect HTTP method on a correct path would return an error different from a completely non-existent path. I know that might be difficult with the way the API code is designed though. Maybe even just returning the method in the error would help. I thought /access/password was broken until I looked at the code and saw it was a PUT, not a GET.


/access/password (and the broken /access/users) returns invalid realm in user id. These should probably say in some way the API token cannot be used for this. It would be good to also have a different message if the user id is valid, but doesn't have permission for that path.

yeah, some more explanatory error message would go a long way here - maybe something like "API endpoint restricted to proper users - 'AUTHID' failed to parse as Userid: 'ERROR MESSAGE FROM PARSING'" or even "API endpoint not allowed for API tokens", since we know at this point that 'something' is logged in, and it's not a Userid so it must be a an API token