Proxmox with Terraform and Ansible

[Faraday]

Hi. Im trying to use Terraform to build VMs on proxmox and config them with Ansible. Does someone have a sample code or playbooks for ansible. I have some stuggels getting everything together. Hopefully someone can help me.

 

[wmorrison]

I do, but it's not online (yet.) I'll find some examples and post later. How far have you got?

I'm mostly using Proxmox containers, usually Alpine Linux, and my latest status is:

- Have a Terraform script that spins up two Alpine lxc's, sequential ids starting from a specified number
- Terraform script also uses the telmate/proxmox and ansible/ansible providers, the ansible one creates inventory info
- Ansible script uses cloud.terraform.terraform_provider to get inventory info, also uses ansible-lxc-ssh to connect to containers
- Ansible script checks for and installs Python3 with raw, then other ansible commands work

I'm still learning so the code may be messy/uncommented, wasn't planning to share it yet.

 

[wmorrison]

OK, the code in the attached zip is messy, things that should be parameterized are hardcoded, sensitive values changed to stuff like "your password here," and so on. No root folder in the zip, so unzip it inside a new folder.

Terraform and Ansible scripts run separately, but Terraform creates the inventory for Ansible. Not sure yet the best way to run both as a group. (Trigger Ansible from Terraform, or vice-versa?)

Feel free to ask questions due to the lack of comments... If I ever put this on github or something, it will be commented better.

If you are doing VMs, you can take out the lxc-ssh stuff, but it will probably need more editing, and see the part in main.tf about your public key which you would probably need then. Or try the alpine container template first just to see it work. Also, I think lxc-ssh requires that you put your public key in proxmox /root/.ssh/authorized_keys, it doesn't work with password authorization.

 

[Faraday]

Omg your a genius. Thanks. I try it today. I have already the terraform main.tf to create VMs, but i had some issues connecting everything with ansible. They talk always of a server that has linux on it to start with ansible. But in our case, do we ned something like that? We have our provider in terraform implemented.

What do you mean with this sentence? => Ansible script uses cloud.terraform.terraform_provider to get inventory info.

I work also with telemat/proxmox and i have my statefile in the terraform cloud.

 

[wmorrison]

What do you mean with this sentence? => Ansible script uses cloud.terraform.terraform_provider to get inventory info.

After you run terraform plan, if you look inside terraform.tfstate, there's an instances array with info built from the "ansible_host" resources in the plan.

If you look in inventory.yml, it only has

plugin: cloud.terraform.terraform_provider

and that makes ansible read the inventory (machines for ansible to control and variables) from terraform.tstate.

 

[Faraday]

Okey, now i got it. Your great. Im testing now and try to built it.

 

[wmorrison]

So what part isn't working?

I've only used containers (lxc) so far. Which are ready-to-use images. But a VM from an iso means, when you get a console in Proxmox, you get to the installer, not a ready to use Linux system. I have not yet looked into how to automate that. Are you? Or are you setting it up manually, before using Ansible?

This is one reason I started with containers instead of VMs, to skip the install step. But I do want to learn this.

From what I understand, this is often done by creating a VM, running the install (manually?) then creating a template from that, and using the template to create your new VMs. But I haven't done that, and could be wrong.

 

[Faraday]

Okey, i started testing yesterday. The problem is, that setting up the basic installation of a vm with Terraform and Cloud-Init works good, but i have some struggles with ansible. And yes i tried that. I can send you my sample code.

 

[jsterr]

I wrote a tutorial for absolute beginners how to get started with Terraform and Proxmox VE:

https://www.thomas-krenn.com/de/wiki/Terraform_mit_Proxmox_VE_Tutorial

Feedback welcome, translation incoming. Disclaimer: Im also a absolute beginner regarding Terraform but people might find this useful when searching via google etc. and get to this post.

 

[Faraday]

Thats great. Thanks

 

[jsterr]

Thats great. Thanks

Thanks! Still updating as I am still learning. Terraform always wants to change qemu_os, although Im not using it in the config:


# proxmox_vm_qemu.srv_demo_4 will be updated in-place
~ resource "proxmox_vm_qemu" "srv_demo_4" {
id = "PMX4/qemu/103"
name = "srv-demo-4"
- qemu_os = "l26" -> null


You know how to properly use qemu_os?

Edit: Oh damn its L 26 with a small l not 126. Works fine now!

 

[jsterr]

@eset here is my config (referring to a post from pve 8.1 release post)

Edit: Oh I see, there was a new version 17 days ago, had no issues with the previous one. Will test it soon and reply here again.

@eset can you provide me an example .tf file, they changed nearly all of the syntax that was working in the prev version :-/

 

[eset]

@eset here is my config (referring to a post from pve 8.1 release post)

Edit: Oh I see, there was a new version 17 days ago, had no issues with the previous one. Will test it soon and reply here again.

@eset can you provide me an example .tf file, they changed nearly all of the syntax that was working in the prev version :-/

Yeah of course. I have already prepared whole setup for two scenarios.
For use pve with terraform indeed we need template so I also have created ansible scripts that creates templates instead of manually putting `qm` commands in CLI. It also used some conditional to avoid issues with lack of indempotency when there is no ansible module for specific process and there isn't of course when it comes to qm. There is one for template but I wasn't using it.

Here is the terraform code with cloud-init used in cicustom in cloudinit https://github.com/sonic-networks/terraform/tree/master/proxmox
and the ansible for creating templates that can be easily extended with many other images by just adding additional dict in vars
https://github.com/sonic-networks/ansible/tree/master/proxmox

 

[eset]

Thanks! Still updating as I am still learning. Terraform always wants to change qemu_os, although Im not using it in the config:


# proxmox_vm_qemu.srv_demo_4 will be updated in-place
~ resource "proxmox_vm_qemu" "srv_demo_4" {
id = "PMX4/qemu/103"
name = "srv-demo-4"
- qemu_os = "l26" -> null


You know how to properly use qemu_os?

Edit: Oh damn its L 26 with a small l not 126. Works fine now!

btw I got this to. After I struggle with not working tf code on PVE8 that removes cloudinit with cicustom I break the process and run again and then I get cloudinit back again plus it does this thing above and it does it over and over constantly every time I launch plan. What was the solution ?

 

[eset]

@eset here is my config (referring to a post from pve 8.1 release post)

Edit: Oh I see, there was a new version 17 days ago, had no issues with the previous one. Will test it soon and reply here again.

@eset can you provide me an example .tf file, they changed nearly all of the syntax that was working in the prev version :-/

@jsterr And did you done something according to the issue?