Encoding issues with accented characters in Proxmox VM notes set via Ansible playbook

[maxiReglisse]

Hello all,

It seems to me that sending notes to Proxmox containing accented characters using an Ansible playbook is no more possible with Ansible 2.19 and PVE 9.

At the command line on a PVE, this can be done with the qm set command:

<span>root@pve3380:~ </span><span data-darkreader-inline-color=""># qm set 175 --description "Test avec un é à l'intérieur"</span><span><br></span><span>update VM </span><span data-darkreader-inline-color="">175</span><span>: -description Test avec un � � l'int�rieur</span>

The command output isn't very pretty... but we can verify that the encoding is correct:

[CODE]<span>root@pve3380:~ </span><span data-darkreader-inline-color=""># qm config 175 | grep description</span><span><br></span>description: Test avec un %C3%A9 %C3%A0 l'int%C3%A9rieur

[/CODE]
So the command does its job!



Now let's try with a playbook. I invite you to copy/paste this on your Ansible deployment machine if you have one:

---
- name: Test accented characters in Proxmox
  hosts: localhost
  gather_facts: no
  vars:
    pve_fqdn: "pve3380.inframshe.univ-fcomte.fr"
    vm_id: 175
    description: "Test avec un é à l'intérieur"  # Test with an accented character inside


  tasks:
    - name: Update VM description with an accented character
      ansible.builtin.shell: |
        qm set {{ vm_id }} --description "{{ description }}"
      delegate_to: "{{ pve_fqdn }}"
      register: result
      ignore_errors: yes


    - name: Display the command result
      debug:
        var: result


    - name: Check the saved description
      ansible.builtin.shell: |
        qm config {{ vm_id }} | grep description
      delegate_to: "{{ pve_fqdn }}"
      register: description_result


    - name: Display the saved description
      debug:
        var: description_result.stdout_lines

You should observe something like this:
root@pbs14200:/usr/local/ansible-admin # ansible-playbook playbooks/test_accent.yml
PLAY [Test accented characters in Proxmox] ******************************************************************************************************************************************
TASK [Update VM description with an accented character] **********************************************************************************************************************
[WARNING]: Host 'localhost' is using the discovered Python interpreter at '/usr/bin/python3.13', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/devel/reference_appendices/interpreter_discovery.html for more information.
[ERROR]: Task failed: Refusing to deserialize an invalid UTF8 string value: 'utf-8' codec can't encode character '\udce9' in position 41: surrogates not allowed
Task failed.
Origin: /usr/local/ansible-admin/playbooks/test_accent.yml:11:7
9
10 tasks:
11 - name: Update VM description with an accented character
^ column 7
<<< caused by >>>
Refusing to deserialize an invalid UTF8 string value: 'utf-8' codec can't encode character '\udce9' in position 41: surrogates not allowed This check can be disabled with the `MODULE_STRICT_UTF8_RESPONSE` setting.
fatal: [localhost -> pve3380.inframshe.univ-fcomte.fr]: FAILED! => {"changed": false, "msg": "Task failed: Refusing to deserialize an invalid UTF8 string value: 'utf-8' codec can't encode character '\\udce9' in position 41: surrogates not allowed"}
...ignoring
TASK [Display the command result] ***************************************************************************************************************************************************
ok: [localhost] => {
"result": {
"changed": false,
"exception": "(traceback unavailable)",
"failed": true,
"msg": "Task failed: Refusing to deserialize an invalid UTF8 string value: 'utf-8' codec can't encode character '\\udce9' in position 41: surrogates not allowed",
"warnings": [
"Host 'localhost' is using the discovered Python interpreter at '/usr/bin/python3.13', but future installation of another Python interpreter could cause a different interpreter to be discovered."
]
}
}
TASK [Check the saved description] ***************************************************************************************************************************************************
changed: [localhost -> pve3380.inframshe.univ-fcomte.fr]
TASK [Display the saved description] ***************************************************************************************************************************************************
ok: [localhost] => {
"description_result.stdout_lines": [
"description: Test avec un %C3%A9 %C3%A0 l'int%C3%A9rieur"
]
}
PLAY RECAP ***********************************************************************************************************************************************************************************
localhost : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=1



This issue is new. This playbook used to run without any problems on PVE version 8, but since we upgraded to version 9 (with Debian 13?), we've been encountering this issue.


My current PVE version: Manager Version pve-manager/9.0.6


To work around the problem, I've had to resort to replacing accented characters with their ASCII equivalents. But this is a functional regression.


I forgot to mention the Ansible version on my deployment machine:

ansible [core 2.19.0b6]
  python version = 3.13.5 (main, Jun 25 2025, 18:55:22) [GCC 14.2.0] (/usr/bin/python3)
  jinja version = 3.1.6
  pyyaml version = 6.0.2 (with libyaml v0.2.5)

Thank you in advance for your feedback.

Ernest.

Best regards,Ernest.

 

[dcsapak]

[ERROR]: Task failed: Refusing to deserialize an invalid UTF8 string value: 'utf-8' codec can't encode character '\udce9' in position 41: surrogates not allowed

it seems this error comes from ansible itself, so has nothing to do with pve really, not sure why this only comes up on pve9 installs

it looks like there is somewhere a \uDCE9 character in the text (https://www.compart.com/en/unicode/U+DCE9) maybe that happened with copy and paste ?

 

[maxiReglisse]

You're right, I tried with an older version of Ansible, 2.17 instead on 2.19, and the playbook successfully sent the data containing accented characters. Thank you for your message.
Best regards.
Ernest.