In my testing, I've been having great success using Cloud-Init with Centos 7 VMs. I have essentially followed the instructions here https://pve.proxmox.com/wiki/Cloud-Init_Support but used used the "Generic Cloud" Centos 7 image from here http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.raw.tar.gz
This image has all the required packages pre-installed - cloud-init itself, qemu guest agent, auto storage resize etc etc - and seems well set up for my needs. In other words it just works, and it works well.
Unfortunately I have encountered a problem with Console access after Migrating such a VM from one Node to another Node in a two Node cluster, and I can't understand why.
Can anybody suggest what might be wrong please?
I do not use ceph/shared storage. All storage is local. I use the default noVNC for Console access.
Due to the bug/limitation/whatever that has been discussed elsewhere on the forum, it is not possible to Migrate (live or otherwise) a VM from one Node to another when it has a Cloud-Init CD-ROM attached.
BUT you can easily Migrate if you simply disconnect the Cloud-Init CD-ROM first.
Then, after the Migration is complete, you can create and attach a new Cloud-Init CD-ROM to the migrated VM.
And what I have just described is what I have done, and almost everything seems to work as expected - except Console access.
part of the initial config in the wiki referenced above to to set up a serial console:
qm set <VMID> --serial0 socket --vga serial0
When you do this you will see in the GUI:
Display: "Serial Terminal (serial0)"
Serial Port (serial0): "Socket"
This works perfectly before Migration and you can see 100% of the boot process as you would with a screen connected to a real system, including all the elements of Cloud-Init when they trigger.
But after the Migration (shut down VM, remove Cloud-Init CD-ROM, Migrate, Add Cloud-Init CD-ROM, Regenerate Image), Console doesn't work at all. All I get is Connected (encrypted) at the top of the window then a flash of black screen then back to Connected (encrypted) over and over. In the meantime, the VM is booting correctly.
To regain some Console access, I have to set Display to VGA. This does work but the only information shown during boot is what you'd see in dmesg as opposed to the usual boot output like [ OK ] blah blah and so on. I'm sorry, I do not know what this is officially called.
I have tried various things: Changing Display to VGA and then back to Serial and re-issuing the "qm set <VMID> --serial0 socket --vga serial0" command again on the Node the VM is now migrated to. The VM is shutdown between each change.
I also tried adding a second serial port and using that for the Console:
qm set <VMID> --serial1 socket --vga serial1
But that didn't work either.
Needless to say, I have also clicked on the Regenerate Image in the Cloud-Init section in the GUI and rebooted the VM numerous times.
Unfortunately I do not know enough about the way that the OS deals with serial ports to look into this any further.
** Please be aware: My current testing is being done in Proxmox running under Virtualbox, and "KVM hardware virtualization" has to be disabled in the Options section for the VM in Proxmox to make it work at all. I realise that this might have some influence on my problem, but I did test the same situation on physical servers previously and had similar issues with Console access as explained here.
This image has all the required packages pre-installed - cloud-init itself, qemu guest agent, auto storage resize etc etc - and seems well set up for my needs. In other words it just works, and it works well.
Unfortunately I have encountered a problem with Console access after Migrating such a VM from one Node to another Node in a two Node cluster, and I can't understand why.
Can anybody suggest what might be wrong please?
I do not use ceph/shared storage. All storage is local. I use the default noVNC for Console access.
Due to the bug/limitation/whatever that has been discussed elsewhere on the forum, it is not possible to Migrate (live or otherwise) a VM from one Node to another when it has a Cloud-Init CD-ROM attached.
BUT you can easily Migrate if you simply disconnect the Cloud-Init CD-ROM first.
Then, after the Migration is complete, you can create and attach a new Cloud-Init CD-ROM to the migrated VM.
And what I have just described is what I have done, and almost everything seems to work as expected - except Console access.
part of the initial config in the wiki referenced above to to set up a serial console:
qm set <VMID> --serial0 socket --vga serial0
When you do this you will see in the GUI:
Display: "Serial Terminal (serial0)"
Serial Port (serial0): "Socket"
This works perfectly before Migration and you can see 100% of the boot process as you would with a screen connected to a real system, including all the elements of Cloud-Init when they trigger.
But after the Migration (shut down VM, remove Cloud-Init CD-ROM, Migrate, Add Cloud-Init CD-ROM, Regenerate Image), Console doesn't work at all. All I get is Connected (encrypted) at the top of the window then a flash of black screen then back to Connected (encrypted) over and over. In the meantime, the VM is booting correctly.
To regain some Console access, I have to set Display to VGA. This does work but the only information shown during boot is what you'd see in dmesg as opposed to the usual boot output like [ OK ] blah blah and so on. I'm sorry, I do not know what this is officially called.
I have tried various things: Changing Display to VGA and then back to Serial and re-issuing the "qm set <VMID> --serial0 socket --vga serial0" command again on the Node the VM is now migrated to. The VM is shutdown between each change.
I also tried adding a second serial port and using that for the Console:
qm set <VMID> --serial1 socket --vga serial1
But that didn't work either.
Needless to say, I have also clicked on the Regenerate Image in the Cloud-Init section in the GUI and rebooted the VM numerous times.
Unfortunately I do not know enough about the way that the OS deals with serial ports to look into this any further.
** Please be aware: My current testing is being done in Proxmox running under Virtualbox, and "KVM hardware virtualization" has to be disabled in the Options section for the VM in Proxmox to make it work at all. I realise that this might have some influence on my problem, but I did test the same situation on physical servers previously and had similar issues with Console access as explained here.
