Creating snippets using PVE API

[Rafael Fernández López]

Hello,

Thank you for Proxmox! I'm running 5.4-3 in a test lab right now.

I am trying to use the `customci` configuration parameter for VM's, so I can provide arbitrary `user-data` in the cloud-init configuration. For that, I see that the new snippet feature is the way to go if I understood correctly.

However, I noticed that it's not possible to create snippets using the PVE API (at least through the /api2/json/nodes/{node}/storage/{storage}/upload endpoint). The error I'm currently getting is:

400 Parameter verification failed.
content: upload content type 'snippets' not allowed

I've checked the source code and I see that only the `iso` and `vztmpl` content types are considered for this endpoint: https://git.proxmox.com/?p=pve-stor...9e6fe495501eaed874eee94367d49cb3;hb=HEAD#l410

I think it would make sense to allow snippet managing using the PVE API, is this possible in some way I missed? If it's not, is this planned for the near future? Would it be a desired feature that could be contributed?


Thank you!

 

[Rafael Fernández López]

As a first approach I wrote a patch that allows the snippet creation on this very endpoint:

diff --git a/PVE/API2/Storage/Status.pm b/PVE/API2/Storage/Status.pm
index 9a5a952..fd7ac5f 100644
--- a/PVE/API2/Storage/Status.pm
+++ b/PVE/API2/Storage/Status.pm
@@ -417,8 +417,10 @@ __PACKAGE__->register_method ({
                raise_param_exc({ filename => "missing '.tar.gz' or '.tar.xz' extension" });
            }
            $path = PVE::Storage::get_vztmpl_dir($cfg, $param->{storage});
-       } else {
-           raise_param_exc({ content => "upload content type '$content' not allowed" });
+       } elsif ($content eq 'snippets') {
+           $path = PVE::Storage::get_snippet_dir($cfg, $param->{storage});
+  } else {
+      raise_param_exc({ content => "upload content type '$content' not allowed" });
        }
 
        die "storage '$param->{storage}' does not support '$content' content\n"
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index eb5e86f..bc0981b 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -340,6 +340,15 @@ sub get_iso_dir {
     return $plugin->get_subdir($scfg, 'iso');
 }
 
+sub get_snippet_dir {
+    my ($cfg, $storeid) = @_;
+
+    my $scfg = storage_config($cfg, $storeid);
+    my $plugin = PVE::Storage::Plugin->lookup($scfg->{type});
+
+    return $plugin->get_subdir($scfg, 'snippets');
+}
+
 sub get_vztmpl_dir {
     my ($cfg, $storeid) = @_;

However this endpoint does not seem the right place for handling snippets (at least creation and deletion), since in REST terms the deletion of the snippet would look awkward? In any case, I'll wait for your feedback

 

[dcsapak]

thats right, currently you cannot create/delete snippets via the api, you have to create them on the target storage (e.g. via a commandline tool, file explorer, etc.)

edit:
if you want to be able to upload some, please open an enhancement request here:

https://bugzilla.proxmox.com

 

[morph027]

Added another +1 to the feature request

I'm working on some kind of proxmox driver for misc. workloads to bootstrap cloud init vm's without a hassle (e.g. CI Pipelines, ...) in https://gitlab.com/morph027/pve-cloud-init-creator and currently i'm using a workaround using S3 server to upload snippets.

 

[ninjitsu]

Another +1 to the feature request.

 

[guerby]

thats right, currently you cannot create/delete snippets via the api, you have to create them on the target storage (e.g. via a commandline tool, file explorer, etc.)

edit:
if you want to be able to upload some, please open an enhancement request here:

https://bugzilla.proxmox.com

For reference it has been opened: https://bugzilla.proxmox.com/show_bug.cgi?id=2208

Bumped multiple times

 

[guerby]

Patch but not working yet, help welcomed!

https://lists.proxmox.com/pipermail/pve-devel/2022-April/052548.html

 

[lethargos]

Any intention of adding this feature? Using terraform/packer with proxmox becomes really difficult because of this limitation.

 

[guerby]

One blocking issue has been fixed recently about the upload of small files via the API:

https://bugzilla.proxmox.com/show_bug.cgi?id=3990

Upload of small iso fails with error 501 upload failed

So nothing blocks the snippet yaml patch technically now.

 

[hradaideh]

poking it again in 2023 ..

we need a way to push snippets for cloud init using the API, or a better way to customize cloud init configuration since the current are not sufficient

 

[lethargos]

I have to say this is one of the more important reasons why we have to use vsphere instead of proxmox in production. I would have liked to understand the thinking behind it/the general direction. There are a lot of good things about proxmox (like the clearly superior GUI to that of vSphere) and this just intriguing.

 

[morph027]

Chime in, would love this feature too

My current workaround: https://gitlab.com/morph027/pve-cloud-init-creator/#snippet-storage