Creating VMs automatically - ansible, vagrant, API .. ?

locusofself

Member
Mar 29, 2016
39
3
8
37
Is anyone on here provisioning and configuring VMs on Proxmox, in a programmatic way?

I'm hoping to get something going with vagrant, ansible, etc, and/or perhaps the Proxmox API. I have several proxmox nodes in a cluster.

Specifying a KVM template, hardware specifics (disk size, VLAN id etc), and then being able to create, boot and execute a script or ansible playbook to configure that VM or group of VMs is the goal.

Right now I'm manually cloning a KVM template, doing a git check out of shell script(s), and running them to configure the VM for a specific role, along with setting IP info and hostname, and setting appropriate VLAN .. at a minimum.

Whenever I bring up a new "stack" there are several VMs involved, so the whole process is quite tedious (LDAP, web apps, mail, database, storage stuff, VoIP etc). Like I said I have some shell scripts, but id love to be able to abra-cadabra from my macbook, specify variable information in .yaml file or similar .. and have a new stack sitting on my proxmox cluster, with the correct packages, apache confs, and networking info etc.

I've been doing some of this with Docker on KVM, but theres just some stuff that doesn't lend well to that (dockerizing VoIP for example, or Postfix, or legacy apps with complex upgrade paths.. I'm probably preaching to the choir here)


In short, I want to automate bringing up a new "customer stack" with several VMs together, even just to demo for a customer and then blow it away without wasting tons of time.

I'm just overwhelmed by the amount of automation and virtualization/containerization stuff out there.. its really insane! ansible, docker, vagrant, openstack, the list goes on and on and on.

I do like proxmox and already have it running and have a teeny experience with ansible and vagrant now (and lots of docker actually)..


Thanks for any info.
 

morph027

Well-Known Member
Mar 22, 2013
446
60
48
Leipzig
morph027.gitlab.io
CloudInit is on the Roadmap, when it's working, you can use the API to deploy VMs. Right now, i'm just using Ansible to deploy VMS using existing methods: https://morph027.gitlab.io/post/pve-kickseed/ where my Kickstart/Preseed files are dynamic scripts (i still prefer PHP, use whatever you like) which are being called like

Code:
inst.ks=https://deploy/target.php?hostname=example.com&ip_address=10.10.10.10&netmask=255.255.255.0&whatever_you_want=somehing
 
  • Like
Reactions: mjw and locusofself

locusofself

Member
Mar 29, 2016
39
3
8
37
@morph027 Thanks for this. Looks like a reasonable solution. I didn't realize that using 'qm' commands directly was supported. Looks like you can even do qm create/clone, then 'qm set' parmaters such as vlan.
 

morph027

Well-Known Member
Mar 22, 2013
446
60
48
Leipzig
morph027.gitlab.io
Jip...creating a template with custom args and all the settings is a possibilty. But then i need to change the template all the time ;) Creating the VM directly through qm (API) just needs changes in my scripts.
 

locusofself

Member
Mar 29, 2016
39
3
8
37
Jip...creating a template with custom args and all the settings is a possibilty. But then i need to change the template all the time ;) Creating the VM directly through qm (API) just needs changes in my scripts.

Makes sense.. although your template could have a dynamic script in it also ;)

Mostly Im just thinking about apt-get and yum slowness in provisioning a new VM vs a template with packages already installed. But that could be mitigated by a mirror/repo on the LAN of course.
 

dbayer

Active Member
Apr 15, 2016
53
6
28
51
Hello Morph,

Thank you for creating the proxmox_kvm module. I've tried using it, but got an error message. I posted the details on your pull request listed above.

Thanks,
Daniel
 

Denny

Active Member
Jul 28, 2016
86
17
28
56
Personally I am beginning to get frustrated, I have tried vagrant and had troubles. The create from ISO works but getting an ISO prepped to work with vagrant is proving a bit of a challenge. Create by cloning doesn't work because the current api refuses a linked clone (issue reported but there has been no activity on that project since spring of 2016).

Saltstack is straightforward to work with so I tried it next as it provides the tools for Proxmox. It errors out when I try to create or clone. The api gives out a cryptic 400 error with no details what parameter it doesn't like.

Chef doesn't look promising either. The Knife-proxmox tool hasn't seen love since August of 2014.

Anyone have a success story to share?
 

morph027

Well-Known Member
Mar 22, 2013
446
60
48
Leipzig
morph027.gitlab.io
Personally I am beginning to get frustrated, I have tried vagrant and had troubles. The create from ISO works but getting an ISO prepped to work with vagrant is proving a bit of a challenge. Create by cloning doesn't work because the current api refuses a linked clone (issue reported but there has been no activity on that project since spring of 2016).

Saltstack is straightforward to work with so I tried it next as it provides the tools for Proxmox. It errors out when I try to create or clone. The api gives out a cryptic 400 error with no details what parameter it doesn't like.

Chef doesn't look promising either. The Knife-proxmox tool hasn't seen love since August of 2014.

Anyone have a success story to share?

I'm using the ansible module to create vms from scratch. Tried this one https://github.com/LordGaav/proxmox-deploy but encountered errors (https://github.com/LordGaav/proxmox-deploy/issues/9).
 

Denny

Active Member
Jul 28, 2016
86
17
28
56
I'm using the ansible module to create vms from scratch. Tried this one https://github.com/LordGaav/proxmox-deploy but encountered errors (https://github.com/LordGaav/proxmox-deploy/issues/9).
I have actually got proxmox-deploy working. I struggled for awhile trying to figure out why networking didn't come up after provisioning was complete. I eventually mounted the vm's system disk manually and added a user password so I could get into the console. This was an Ubuntu 16 image and the default ethernet device is ens18 instead of eth0 (the default for proxmox-deploy) with that fixed it just worked.
 

Denny

Active Member
Jul 28, 2016
86
17
28
56
Still not working. Tried on multiple browsers. I also tried searching for your username and can see two snippets. I can access the pve-virt-viewer item but get a 500 error when accessing proxmox-kickseed.yml.
 

morph027

Well-Known Member
Mar 22, 2013
446
60
48
Leipzig
morph027.gitlab.io
Code:
# Ansible version of https://morph027.gitlab.io/post/pve-kickseed/
# Example args file: https://gitlab.com/snippets/1548095
---

- hosts: localhost
  gather_facts: false
 
  vars_prompt:
  - name: "vmname"
    prompt: "VM name?: "
    private: no
  - name: "args_template"
    prompt: "args template?: "
    default: "files/args-ubuntu-docker.yml"
    private: no
  - name: "kernel"
    prompt: "kernel?: "
    default: "http://deploy.example.com/preseed/linux"
    private: no
  - name: "initrd"
    prompt: "initrd?: "
    default: "http://deploy.example.com/preseed/initrd.gz"
    private: no
  - name: "api_host"
    prompt: "PVE API host?: "
    default: "my-proxmox"
    private: no
  - name: "node"
    prompt: "PVE target node?: "
    default: "another-proxmox-node"
    private: no
  - name: "api_user"
    prompt: "PVE API user?: "
    default: "root@pam"
    private: no
  - name: "api_password"
    prompt: "PVE API password?: "
    private: yes

  tasks:
  - name: Random port number
    set_fact:
      port_number: "{{ 65000 |random(start=32000) }}"
     
  - name: Create KVM
    proxmox_kvm:
      api_user: "{{ api_user }}"
      api_password: "{{ api_password }}"
      api_host: "{{ api_host }}"
      node: "{{ node }}"
      name: "{{ vmname }}"
      net: '{"net0":"virtio,bridge=vmbr0"}'
      virtio: '{"virtio0":"rpool-vm:64"}'
      cores: 2
      memory: 2048
      balloon: 512
      vga: qxl
      ostype: l26
      args: "{{ lookup('template', '{{ args_template }}') | replace('\n', ' ')}}"
      state: present

  - name: Let it settle
    pause:
      seconds: 5

  - name: Fetch kernel
    get_url:
      url:  "{{ kernel }}"
      dest: "/tmp/kernel"
    delegate_to: "{{ node }}"

  - name: Fetch initrd
    get_url:
      url:  "{{ initrd }}"
      dest: "/tmp/initrd"
    delegate_to: "{{ node }}"

  - name: Wait for kernel
    wait_for:
      path: /tmp/kernel
    delegate_to: "{{ node }}"

  - name: Wait for initrd
    wait_for:
      path: /tmp/initrd
    delegate_to: "{{ node }}"

  - name: Deploy VM
    proxmox_kvm:
      api_user: "{{ api_user }}"
      api_password: "{{ api_password }}"
      api_host: "{{ api_host }}"
      node: "{{ node }}"
      name: "{{ vmname }}"
      state: started

  - name: Get VMID
    proxmox_kvm:
      api_user: "{{ api_user }}"
      api_password: "{{ api_password }}"
      api_host: "{{ api_host }}"
      node: "{{ node }}"
      name: "{{ vmname }}"
      state: current
    register: result

  - name: Set VMID
    set_fact:
      vmid: "{{ result.msg | regex_replace('^.* = (\\d+).*?$', '\\1') }}"

  - name: Wait for deployment to be finished
    wait_for:
      host: "{{ node }}"
      port: "{{ port_number }}"
      delay: 5
      state: stopped
      timeout: 1200

  - name: Remove deploy args from VM
    shell: "/usr/sbin/qm set {{ vmid }} -delete args"
    delegate_to: "{{ node }}"
   
#  https://github.com/ansible/ansible/issues/21056
#  - name: Remove deploy args from VM
#    proxmox_kvm:
#      api_user: "{{ api_user }}"
#      api_password: "{{ api_password }}"
#      api_host: "{{ api_host }}"
#      node: "{{ node }}"
#      name: "{{ vmname }}"
#      delete: args

  - name: Start VM (again)
    proxmox_kvm:
      api_user: "{{ api_user }}"
      api_password: "{{ api_password }}"
      api_host: "{{ api_host }}"
      node: "{{ node }}"
      name: "{{ vmname }}"
      state: started
 

Denny

Active Member
Jul 28, 2016
86
17
28
56
Thank you sir. I appreciate you going to the extra effort. I am at a loss to explain what I am seeing. It is a 500 error generated on gitlab (custom 500 error page with logo).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!