Roadmap for integration with Ansible

[Inperpetuammemoriam]

As far as I can see, Proxmox VE is currently only partially configurable through community provided Ansible modules. (See below)

Inventory Plugins:

• proxmox

Modules:

• proxmox
• proxmox_domain_info
• proxmox_group_info
• proxmox_kvm
• proxmox_nic
• proxmox_snap
• proxmox_storage_info
• proxmox_tasks_info
• proxmox_template
• proxmox_user_info

Are there any plans to create an official collection in order to make Proxmox VE fully configurable through Ansible?

VMware already provides such a collection. It would be awesome, if Proxmox provided one too.

I already have started to implement a few modules making use of the Proxmox VE API. I would gladly contribute my stub to an official collection.

 

[Inperpetuammemoriam]

bump

 

[dcsapak]

bump

please don't bump threads without more information..

did you start already? if yes, where can someone find it?

AFAIK there is no plan to provide such modules from our side, and note that the ones for vmware are also contributed by the community:
https://github.com/ansible-collections/community.vmware
at least i did not see any offical vmware contributers in the repo .. (mostly redhat/ansible people)

 

[Inperpetuammemoriam]

Yes, but I haven't published it yet. I will try to do so at the end of next week.

Up to now, I have implemented modules for the configuration of domains, groups, roles and users. The modules for TFA, tokens and ACLs are WIP. Thorough testing is probably required for all currently implemented modules.

You might be right with the contributors. Official or not, I still think that it may be an asset for PVE to provide such an interface, especially for larger deployments.

 

[Glowsome]

in regards of Ansible - and running it actively on a 4-node cluster for management/keeping settings in check i do not really see an issue.
this is excluding the absence of the needs-reboot -detection (as currently not implemented/present in proxomox - specified in reboot required )

As you are mixing your post together with settings and Provisioning its hard to get a real bearing as to what the goal exactly is.

In my case i strictly put a difference in regards of system management/keeping in check config on the system level and provisioning users.
- for System i use Ansible
- for user provisioning i am after a federative solution( openid) , but current implementation is running in to issues as to the way some data is transmitted.

- Glowsome

 

[Inperpetuammemoriam]

in regards of Ansible - and running it actively on a 4-node cluster for management/keeping settings in check i do not really see an issue.
this is excluding the absence of the needs-reboot -detection (as currently not implemented/present in proxomox - specified in reboot required )

As you are mixing your post together with settings and Provisioning its hard to get a real bearing as to what the goal exactly is.

In my case i strictly put a difference in regards of system management/keeping in check config on the system level and provisioning users.
- for System i use Ansible
- for user provisioning i am after a federative solution( openid) , but current implementation is running in to issues as to the way some data is transmitted.

- Glowsome

I think there might be a misunderstanding. The goal is not just to provision users but to be able to define an entire PVE node's system state.

I just started with the implementation of the modules that work on the different aspects of PVE user management. The domain module for example should give you the possibility to configure PVE to use your OpenID realm.

Technically speaking, the modules will mainly wrap around the provided API as this probably is the easiest interface for Proxmox to ensure stability on future releases.

 

[jsabater]

Any updates on your work? I am starting to gather information regarding Ansible and Proxmox to create some playbooks to maintain my cluster and would be interested in this.

Also, any good tutorial to get started with what we already have available, nicely listed by you in your original post.

 

[Inperpetuammemoriam]

Any updates on your work? I am starting to gather information regarding Ansible and Proxmox to create some playbooks to maintain my cluster and would be interested in this.

Also, any good tutorial to get started with what we already have available, nicely listed by you in your original post.

As I have been quite busy, I haven't found the time to clean up and test the modules I already have written.

I will try to find some time for it on the week-end.

The modules will be documented in their DOCUMENTATION blocks. Examples will also be provided through the EXAMPLES blocks.

 

[Inperpetuammemoriam]

After having a look at the current state:

I have just created a repository.

As I don't want to unleash low-quality code on unsuspecting users, I will not push all the modules at once. Instead, I intend to go through all of them and do some quality assurance. (The lack of unit and integration tests currently is the biggest show-stopper. I will have to read Ansible's documentation on that matter first.)

Some modules also seem to have become incomplete as the API seemingly has been extended in the meantime. That will also need an extra effort.

 

[ednxzu]

Hello,

After, it's been a few months now and nothing really seem to have come up.
I'm also very interested by this ansible collection of modules for proxmox. Is there any way people can help build this ?

Best,

 

[Glowsome]

@ednxzu
I know its not as TS is after but i do have a set of playbooks to set my ProxMox env to how i want/need it.
Remember, i'm sort of running an exotic env with specific demands, and its absolutely not optimised in regards of tasks, but it works.
Happy to share it tho.

- Glowsome

 

[ednxzu]

@ednxzu
I know its not as TS is after but i do have a set of playbooks to set my ProxMox env to how i want/need it.
Remember, i'm sort of running an exotic env with specific demands, and its absolutely not optimised in regards of tasks, but it works.
Happy to share it tho.

- Glowsome

Hey ! I'd happily take a look at it if you don't mind sharing. However, when I said help build it, I was rather asking if people would be interested to come together and write these modules. It shouldn't be too complicated since there are already some that are available, and the logic stays pretty much the same. I know I'll probbly not do it by myself, but I'd be happy to contribute if other people decide to team up on that one.

 

[jsabater]

I am in the process of learning how to automate deployment of LXC in an existing Proxmox Virtual Environment cluster, so I would be interested as well. Even contribute, hopefully.

 

[Glowsome]

I am in the process of learning how to automate deployment of LXC in an existing Proxmox Virtual Environment cluster, so I would be interested as well. Even contribute, hopefully.

Hey ! I'd happily take a look at it if you don't mind sharing. However, when I said help build it, I was rather asking if people would be interested to come together and write these modules. It shouldn't be too complicated since there are already some that are available, and the logic stays pretty much the same. I know I'll probbly not do it by myself, but I'd be happy to contribute if other people decide to team up on that one.

 

[Glowsome]

For reference, my setup of the cluster was documented in another Forum post : [TUTORIAL] PVE 7.x Cluster Setup of shared LVM/LV with MSA2040 SAS [partial howto]

So do keep in mind that what i a doing in my playbook(s) is aimed at keeping my (system-)setup coherant.

Next to that, i have one standalone box in a datacenter which i also manage with the same playbook - explaining the conditional execution.

---
# ./roles/proxmox/tasks/main.yml

- name: 01 Remove ProxMox Enterprise repository from sources list using specified filename (Debian 10)
  ansible.builtin.apt_repository:
    repo: deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise
    state: absent
    filename: pve-enterprise
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "10"

- name: 02 Remove ProxMox Enterprise repository from sources list using specified filename (Debian 11)
  ansible.builtin.apt_repository:
    repo: deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise
    state: absent
    filename: pve-enterprise
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "11"

- name: 03 remove HP repository mcp repository to reset content
  ansible.builtin.file:
    path: /etc/apt/sources.list.d/mcp.list
    state: absent

- name: 04 Add HP repository into sources list using specified filename (Debian 10)
  ansible.builtin.apt_repository:
    repo: deb http://downloads.linux.hpe.com/SDR/repo/mcp buster/current non-free
    state: present
    filename: mcp
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "10"

- name: 05 Add HP repository into sources list using specified filename (Debian 11)
  ansible.builtin.apt_repository:
    repo: deb http://downloads.linux.hpe.com/SDR/repo/mcp bullseye/current non-free
    state: present
    filename: mcp
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "11"

- name: 06 Add ProxMox free repository into sources list using specified filename (Debian 10)
  ansible.builtin.apt_repository:
    repo: deb http://download.proxmox.com/debian buster pve-no-subscription
    state: present
    filename: pve-install-repo
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "10"

- name: 07 Add ProxMox free repository into sources list using specified filename (Debian 11)
  ansible.builtin.apt_repository:
    repo: deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
    state: present
    filename: pve-install-repo
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "11"

- name: 08 Remove ProxMox Enterprise repository from sources list using specified filename (Debian 11)
  ansible.builtin.apt_repository:
    repo: deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise
    state: absent
    filename: pve-enterprise
  when:
    - ansible_facts['distribution'] == "Debian"
    - ansible_facts['distribution_major_version'] == "11"

- name: 09 Register hostname to determine if its part of a cluster
  ansible.builtin.command: 'hostname --fqdn'
  register: nodename

- name: 10 Install additional packages needed for ProxMox Cluster environment
  ansible.builtin.apt:
    name:
      - lvm2-lockd
      - dlm-controld
      - gfs2-utils
      - lsscsi
      - multipath-tools
    state: present
  when: nodename.stdout is regex("^node0?\.*.")

- name: 11 Update apt-get repo and cache
  ansible.builtin.apt:
    update_cache: yes
    force_apt_get: yes
    cache_valid_time: 3600

- name: 12 Upgrade all apt packages
  ansible.builtin.apt:
    upgrade: dist
    force_apt_get: yes

# ProxMox does not have /var/run/reboot-required, but for future purposes this task is setup
- name: 13 Check if a reboot is needed for ProxMox boxes
  ansible.builtin.stat:
    path: /var/run/reboot-required
  register: check_reboot

- name: 14 Print information about reboot
  ansible.builtin.debug:
    var: check_reboot

- name: 15 Ensure directory /etc/dlm is present
  ansible.builtin.file:
    path: /etc/dlm
    state: directory
    mode: '0755'
  when: nodename.stdout is regex("^node0?\.*.")

- name: 16 Ensure customised dlm.conf is present
  ansible.builtin.template:
    src: 'dlm.conf.j2'
    dest: '/etc/dlm/dlm.conf'
    mode: 0600
  when: nodename.stdout is regex("^node0?\.*.")

- name: 17 Ensure lvm.conf contains lvmlockd = 1
  ansible.builtin.template:
    src: 'lvm.conf.j2'
    dest: '/etc/lvm/lvm.conf'
    mode: 0600
  when: nodename.stdout is regex("^node0?\.*.")

- name: 18 Ensure shared volumes and mountpoint definition file is present
  ansible.builtin.template:
    src: 'lvmshared.conf.j2'
    dest: '/etc/lvm/lvmshared.conf'
    mode: 0600
  when: nodename.stdout is regex("^node0?\.*.")

- name: 19 Ensure the mountscript for shared volume is available
  ansible.builtin.template:
    src: lvmmount.sh.j2
    dest: '/usr/local/share/lvmmount.sh'
    mode: 0700
  when: nodename.stdout is regex("^node0?\.*.")

- name: 20 Ensure Systemd service for shared volumes is present
  ansible.builtin.template:
    src: 'lvshared.service.j2'
    dest: '/usr/lib/systemd/system/lvshared.service'
    mode: 0644
  when: nodename.stdout is regex("^node0?\.*.")

- name: 21 Ensure multipath configuration file is present
  ansible.builtin.template:
    src: 'multipath.conf.j2'
    dest: '/etc/multipath.conf'
    mode: 0644
  when: nodename.stdout is regex("^node0?\.*.")

- name: 22 Remove possible wrong location of After=lvshared.service
  ansible.builtin.lineinfile:
    path: /lib/systemd/system/pve-guests.service
    regexp: '^After=lvshared.service'
    state: absent
  when: nodename.stdout is regex("^node0?\.*.")

- name: 23 Ensure Systemd service pve-guests has an After=lvshared.service entry
  ansible.builtin.lineinfile:
    path: /lib/systemd/system/pve-guests.service
    regexp: '^After=lvshared.service'
    insertafter: '^After=pve-ha-crm.service.*'
    line: After=lvshared.service
    mode: 0644
  when: nodename.stdout is regex("^node0?\.*.")

- name: 24 Force systemd to reread configs (2.4 and above)
  ansible.builtin.systemd:
    daemon_reload: yes

- name: 25 Ensure Systemd service for shared volumes is enabled
  ansible.builtin.systemd:
    name: lvshared
    enabled: yes
    masked: no
  when: nodename.stdout is regex("^node0?\.*.")

- name: 26 Check /root/.ssh/authorised_keys
  ansible.builtin.stat:
    path: /root/.ssh/authorized_keys
    get_checksum: no
  register: ssh_root_authorized_keys_stat

- name: 27 Print information about root-authorized_keys
  ansible.builtin.debug:
    var: ssh_root_authorized_keys_stat

- name: 28 Delete /root/.ssh/authorised_keys when not a symlink
  ansible.builtin.file:
    path: /root/.ssh/authorised_keys
    state: absent
  when:
    - ssh_root_authorized_keys_stat.stat.islnk | bool

- name: 29 Symlink /root/.ssh/authorized_keys to /etc/pve/priv/authorized_keys
  ansible.builtin.file:
    src: /etc/pve/priv/authorized_keys
    dest: /root/.ssh/authorized_keys
    owner: root
    state: link
  when:
    - ssh_root_authorized_keys_stat.stat.islnk | bool or ssh_root_authorized_keys_stat.stat.lnk_target != "/etc/pve/priv/authorized_keys"

- name: 30 Check /root/.ssh/known_hosts
  ansible.builtin.stat:
    path: /root/.ssh/known_hosts
    get_checksum: no
  register: ssh_root_known_hosts_stat

- name: 31 Print information about root-known_hosts
  ansible.builtin.debug:
    var: ssh_root_known_hosts_stat

- name: 32 Delete /root/.ssh/known_hosts for correct symlinking if it exists.
  ansible.builtin.file:
    path: /root/.ssh/known_hosts
    state: absent
  when:
    - ssh_root_known_hosts_stat.stat.exists

- name: 33 Symlink /root/.ssh/known_hosts to /etc/pve/priv/known_hosts
  ansible.builtin.file:
    src: /etc/pve/priv/known_hosts
    dest: /root/.ssh/known_hosts
    owner: root
    state: link
  when:
    - not ssh_root_known_hosts_stat.stat.exists

- name: 34 Check /etc/ssh/ssh_known_hosts
  ansible.builtin.stat:
    path: /etc/ssh/ssh_known_hosts
    get_checksum: no
  register: ssh_etc_known_hosts_stat

- name: 35 Print information about etc-known_hosts
  ansible.builtin.debug:
    var: ssh_etc_known_hosts_stat

- name: 36 Delete /etc/ssh/ssh_known_hosts when not a symlink or not linked correctly
  ansible.builtin.file:
    path: /etc/ssh/ssh_known_hosts
    state: absent
  when:
    - ssh_etc_known_hosts_stat.stat.islnk | bool

- name: 37 Symlink /etc/ssh/ssh_known_hosts to /etc/pve/priv/known_hosts
  ansible.builtin.file:
    src: /etc/pve/priv/known_hosts
    dest: /etc/ssh/ssh_known_hosts
    owner: root
    state: link
  when:
    - ssh_etc_known_hosts_stat.stat.islnk | bool

- name: 38 Add nodes to known_hosts
  ansible.builtin.known_hosts:
    path: /etc/pve/priv/known_hosts
    name: '{{ item.name }}'
    key: '{{ item.name }} {{ item.key }}'
  loop: '{{ my_node_keys }}'
  no_log: true
  when: nodename.stdout is regex("^node0?\.*.")

- name: 39 Check if /root/.ssh/ssh_known_hosts
  ansible.builtin.stat:
    path: /root/.ssh/known_hosts
    get_checksum: no
  register: root_known_hosts_stat

- name: 40 Print information about root_known_hosts
  ansible.builtin.debug:
    var: root_known_hosts_stat

- name: 41 Delete /root/.ssh/known_hosts when not a symlink
  ansible.builtin.file:
    path: /root/.ssh/known_hosts
    state: absent
  when:
    - root_known_hosts_stat.stat.islnk | bool

- name: 42 Symlink /root/.ssh/known_hosts to /etc/pve/priv/known_hosts
  ansible.builtin.file:
    src: /etc/pve/priv/known_hosts
    dest: /root/.ssh/known_hosts
    owner: root
    state: link
  when:
    - not root_known_hosts_stat.stat.exists

- name: 43 Set up Node authorized keys
  ansible.posix.authorized_key:
    manage_dir: no
    path: /etc/pve/priv/authorized_keys
    user: root
    state: present
    key: '{{ item.key }}'
  loop: '{{ my_node_keys }}'
  no_log: true
  when: nodename.stdout is regex("^node0?\.*.")

- name: 44 Add keys to ssh_known_hosts
  ansible.builtin.known_hosts:
    path: /etc/pve/priv/known_hosts
    name: '{{ item.name }}'
    key: '{{ item.name }} {{ item.key }}'
  loop: '{{ my_host_keys }}'
  no_log: true
  when: nodename.stdout is regex("^node0?\.*.")

 

[kraut.hosting]

IMHO The most valuable missing plugin for Ansible is a connection one like those community ones:

This would allow us to manage VMs without SSH & network like for VMware, libvirt, LXD, Qubes & co.

 

[Glowsome]

IMHO The most valuable missing plugin for Ansible is a connection one like those community ones:
View attachment 56021
This would allow us to manage VMs without SSH & network like for VMware, libvirt, LXD, Qubes & co.

For your info, i have not yet gone into the deep regarding actual guest-management.
I just require (for now) managing the nodes of my cluster.

- Glowsome

 

[kraut.hosting]

@Glowsome Get you and so do we, but that's mostly managing Debian with bunch on services on top.
Common Ansible modules can be used for that. Integration starts is once you need command & shell:

- name: create cluster on first node
  command: "pvecm create {{ pve_cluster_name }}"
  args:
    creates: /etc/pve/corosync.conf
  register: pve_cluster_create
  run_once: true

But even that can be solved in some hacky but working ways. Real integration is new Ansible plugins.
Like an API module for cluster management to replace `command` around pvesh or `uri` against API:
https://github.com/robinelfrink/ansible-proxmox-api
This could be a starting point, but haven't evaluated it yet. Maybe someone has experience with it?