Ansible 'Operation not permitted' copying ssl keys directly

rml · Apr 28, 2021

I've got as far as working out that /etc/pve is a fuse mount and that I should use the gui to replace the certs (although it would be nice to automate this)

So, more for my leaning, is there a better way to copy these into place through ansible that I'm missing please?
Or what am I missing here about why it's not copying please?

Some interesting behaviour...

1) it doesn't seem to be an immutable switch but interesting that I'm not getting back information on the files.

Code:

lsattr: Function not implemented While reading flags on ./pve-ssl.pem
lsattr: Function not implemented While reading flags on ./pve-ssl.key

2) trying to delete- Ansible looks like it's deleting the previous files ok, should it?
I only ask as I thought it report they were deleted but it wasn't doing it until I manuall deleted them- but I may be misremembering

Code:

- name:
    file:
      path: "{{item}}"
      state: absent
    loop:
      - /etc/pve/nodes/{{node}}/pveproxy-ssl.pem
      - /etc/pve/nodes/{{node}}/pveproxy-ssl.key

Code:

TASK [file] ******************************************************************************
changed: [rmlprox] => (item=/etc/pve/nodes/rmlprox/pveproxy-ssl.pem)
changed: [proxrml] => (item=/etc/pve/nodes/pve/pveproxy-ssl.pem)
changed: [rmlprox] => (item=/etc/pve/nodes/rmlprox/pveproxy-ssl.key)
changed: [proxrml] => (item=/etc/pve/nodes/pve/pveproxy-ssl.key)

3) Copying them directly with ansible.builtin.copy with doesn't work, either directly, or by copying them locally and them using
copy:
remote_src: yes

Code:

- name: move the files
    ansible.builtin.copy:
      src: "{{item}}"
      dest: "/etc/pve/nodes/{{node}}/"
      remote_src: yes
      owner: root
      group: www-data
      mode: 0640
    become: yes
    loop:
      - /root/certs/pveproxy-ssl.pem
      - /root/certs/pveproxy-ssl.key

Code:

failed: [rmlprox] (item=/root/certs/pveproxy-ssl.pem) => {"ansible_loop_var": "item", "changed": false, "item": "/root/certs/pveproxy-ssl.pem", "msg": "failed to copy: /root/certs/pveproxy-ssl.pem to /etc/pve/nodes/rmlprox/pveproxy-ssl.pem", "traceback": "Traceback (most recent call last):\n  File \"/tmp/ansible_ansible.legacy.copy_payload_j8q7eurp/ansible_ansible.legacy.copy_payload.zip/ansible/modules/copy.py\", line 684, in main\n  File \"/tmp/ansible_ansible.legacy.copy_payload_j8q7eurp/ansible_ansible.legacy.copy_payload.zip/ansible/module_utils/basic.py\", line 2468, in atomic_move\n    os.chmod(b_dest, DEFAULT_PERM & ~umask)\nPermissionError: [Errno 1] Operation not permitted: b'/etc/pve/nodes/rmlprox/pveproxy-ssl.pem'\n"}

4) but you can run a command


  - name: sudo move files
    ansible.builtin.command:
      cmd: "cp {{item}} /etc/pve/nodes/{{node}}/"
    loop: 
     - /root/certs/pveproxy-ssl.pem
     - /root/certs/pveproxy-ssl.key
    become: yes

fabian · Apr 28, 2021

can you try leaving out the mode, owner, and group from your move command? pmxcfs handles that itself, and the error indicates that it chokes on setting the permissions...

rml · Apr 28, 2021

Thanks but doesn't work for me I'm getting two different errors (some info obsfucated)
Don't want to waste anyone's time as it works the shell way works - but interested to understand this

In both cases I'm running these as sudo ansible-playbook
(not to self: is there a better way to get privilege escalation for the control machine to access the files?
Perhaps import_playbook with delegate_to: localhost?)

Copying straight in place (r

YAML:

- name: move the files as directly (sudo ansible-playbook)
    ansible.builtin.copy:
      src: "{{item}}"
      dest: "/etc/pve/nodes/{{node}}/"
      # remote_src: yes
      # owner: root
      # group: www-data
      # mode: 0640
    become: yes
    become_user: root
    ansible_ssh_user: root
    loop:
      - "/etc/letsencrypt/live/{{inventory_hostname}}.{{web_domain}}.{{web_suffix}}/fullchain.pem"
      - "/etc/letsencrypt/live/{{inventory_hostname}}.{{web_domain}}.{{web_suffix}}/privkey.pem"
    ignore_errors: yes

YAML:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: PermissionError: [Errno 1] Operation not permitted: b'/etc/pve/nodes/rmlprox/.ansible_tmpn50wguabfullchain.pem'
failed: [rmlprox] (item=/etc/letsencrypt/live/rmlprox.X.X.X/fullchain.pem) => {"ansible_loop_var": "item", "changed": false, "checksum": "f750aa0d1bedefae4d5e35e54f48f63c3b73dd1d", "item": "/etc/letsencrypt/live/rmlprox.X.X.X/fullchain.pem", "msg": "Failed to replace file: b'/tmp/.ansible-root/tmp/ansible-tmp-1619611155.5270648-115854-61259630597801/source' to /etc/pve/nodes/rmlprox/fullchain.pem: [Errno 1] Operation not permitted: b'/etc/pve/nodes/rmlprox/.ansible_tmpn50wguabfullchain.pem'"}

YAML:

- name: move the files with remote_src: yes
    ansible.builtin.copy:
      src: "{{item}}"
      dest: "/etc/pve/nodes/{{node}}/"
      remote_src: yes
      # owner: root
      # group: www-data
      # mode: 0640
    become: yes
    become_user: root
    ansible_ssh_user: root
    loop:
      - /root/certs/pveproxy-ssl.pem
      - /root/certs/pveproxy-ssl.key

YAML:

WARNING]: File '/etc/pve/nodes/rmlprox/pveproxy-ssl.pem' created with default permissions '666'. The previous default was '666'. Specify 'mode' to avoid this warning.
failed: [rmlprox] (item=/root/certs/pveproxy-ssl.pem) => {"ansible_loop_var": "item", "changed": false, "item": "/root/certs/pveproxy-ssl.pem", "msg": "failed to copy: /root/certs/pveproxy-ssl.pem to /etc/pve/nodes/rmlprox/pveproxy-ssl.pem", "traceback": "Traceback (most recent call last):\n  File \"/tmp/ansible_ansible.legacy.copy_payload_hhfr78kt/ansible_ansible.legacy.copy_payload.zip/ansible/modules/copy.py\", line 684, in main\n  File \"/tmp/ansible_ansible.legacy.copy_payload_hhfr78kt/ansible_ansible.legacy.copy_payload.zip/ansible/module_utils/basic.py\", line 2468, in atomic_move\n    os.chmod(b_dest, DEFAULT_PERM & ~umask)\nPermissionError: [Errno 1] Operation not permitted: b'/etc/pve/nodes/rmlprox/pveproxy-ssl.pem'\n"}

Though I understand a bit better what's going on now....

For the next guy
* /etc/pve is actually a fuse mounted sqlite database
** "you can’t change file permissions (permissions are based on path)"
** see https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)

fabian · Apr 28, 2021

yeah, not sure what ansible is doing under the hood there

simply copying the file should work, guess they do something more "magic".

rml · Apr 28, 2021

Cool, don't let it bother you, thanks for spending the time looking.

6uellerbpanda · Apr 28, 2021

first copy them to /tmp folder and then copy it from there to /etc/pve/nodes/.
not very elegant but working, at least for me. ansible 2.7 and pve 6.2.

rml · Apr 28, 2021

Thanks, that's what I did in the end but, just to check, how did you so that copy from tmp to nodes? With ansible.builtin.copy? or with a shell sudo command?

6uellerbpanda · Apr 28, 2021

rml said:
Thanks, that's what I did in the end but, just to check, how did you so that copy from tmp to nodes? With ansible.builtin.copy? or with a shell sudo command?

with shell command

rml · Apr 28, 2021

Cool, thanks, that was in the original question but appreciate it was long. Thanks

6uellerbpanda · Apr 28, 2021

rml said:
Cool, thanks, that was in the original question but appreciate it was long. Thanks

Yes you're right...i missed that it seems.

srinath · Jan 10, 2022

Touch the files before copying, if the file already exists ansible won't try to set mode or owner.

Code:

- name: "Create empty cert files"
  shell: "touch /etc/pve/local/pveproxy-ssl.pem /etc/pve/local/pveproxy-ssl.key"
  creates: "/etc/pve/local/pveproxy-ssl.pem"

- name: "Copy certificate files"
  copy:
    dest: "/etc/pve/local/pveproxy-ssl.pem"
    content: |
             {{ lookup('file', 'resources/cert.pem') }}
             {{ lookup('file', 'resources/cert_chain.pem') }}

- name: "Copy keys files"
  copy:
    dest: "/etc/pve/local/pveproxy-ssl.key"
    content: "{{ vault_key_file }}"

Trust_ITS · Mar 7, 2023

Using command solved my problems!
Thanks

cfgmgr · Nov 20, 2023

This thread was very helpful in figuring out I just needed to touch the files to solve for it. Thusly dont need to worry about perms when performing the template

Code:

- name: Touch the files to set permissions
  ansible.builtin.file:
    path: "{{ files }}"
    state: touch
  loop:
    - /etc/pve/local/pveproxy-ssl.pem
    - /etc/pve/local/pveproxy-ssl.key
  loop_control:
    loop_var: files

- name: Template LE certificate to Proxmox host
  ansible.builtin.template:
    src: "{{ cert.src }}"
    dest: "{{ cert.dest }}"
    follow: true
  loop:
    - { src: 'fullchain.pem.j2', dest: '/etc/pve/local/pveproxy-ssl.pem' }
    - { src: 'privkey.pem.j2', dest: '/etc/pve/local/pveproxy-ssl.key' }
  loop_control:
    loop_var: cert
  notify:
    - Restart pveproxy

Yoslan · Aug 31, 2024

Please share the complete playbook

Search

Search

Ansible 'Operation not permitted' copying ssl keys directly

rml

Member

fabian

Proxmox Staff Member

rml

Member

fabian

Proxmox Staff Member

rml

Member

6uellerbpanda

Well-Known Member

rml

Member

6uellerbpanda

Well-Known Member

rml

Member

6uellerbpanda

Well-Known Member

srinath

Member

Trust_ITS

New Member

cfgmgr

Member

Yoslan

Renowned Member