[SOLVED] Does TPM disk affect snapshot support? Cannot take snapshot

[FGD-Garuda]

Hello there!

I recently found about Proxmox and completely migrated my VirtualBox VMs into it. Really cool to work with Proxmox!

I often perform snapshots to have the ability to revert back very quickly, because I use some of my VMs for testing purposes and like everyone sometimes I either make mistakes loll or install/remove stuff that turned out not to be the way to go.

I have 2 types of VMs (no containers) right now as we speak. One type runs Windows 11 and the other runs Linux-based VMs.
On the Linux VMs, I can take snapshots no problem.
On the Windows VMs, I cannot. The famous "The current guest configuration does not support taking new snapshots" msg shows up in the Snapshots section.

In search for an answer and fix, I read the following:
https://pve.proxmox.com/wiki/Storage
https://forum.proxmox.com/threads/t...-does-not-support-taking-new-snapshots.82444/
https://forum.proxmox.com/threads/e...-does-not-support-taking-new-snapshots.86849/
https://forum.proxmox.com/threads/t...-does-not-support-taking-new-snapshots.73944/

None of the above answer "clearly" my situation. Or I'm too dumb to understand.

My Storage configuration is as follow:

# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content images,iso,backup
        shared 0

My snapshot-enabled Linux VM is as follow (took one as an example):

# qm config 101
agent: 1,fstrim_cloned_disks=1
bios: seabios
boot: 
cores: 2
hotplug: disk,usb
localtime: 0
memory: 5000
meta: creation-qemu=6.1.0,ctime=1639951077
name: Garuda-Dr460nized
net0: virtio=FA:0C:AF:7B:71:10,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local:101/vm-101-disk-0.qcow2,discard=on,size=40G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=450970f5-2b61-4268-8947-aba825bb07cc,manufacturer=Q3J1Y2lhbA==,base64=1
sockets: 1
spice_enhancements: foldersharing=1,videostreaming=all
vga: qxl,memory=64
vmgenid: 3f2c1c22-e516-4028-818e-363cee488172

My Windows VM config is as follow:

# qm config 100
agent: 1,fstrim_cloned_disks=1
bios: seabios
boot: order=ide2;ide0
cores: 2
efidisk0: local:100/vm-100-disk-1.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
ide0: local:100/vm-100-disk-0.qcow2,discard=on,size=70G,ssd=1
ide2: none,media=cdrom
machine: pc-q35-6.1
memory: 5000
meta: creation-qemu=6.1.0,ctime=1639847268
name: Windows
net0: e1000=3E:0D:F8:87:EB:92,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=e6cf1ba3-5910-5e36-8468-6f7b071cf7e7
sockets: 1
spice_enhancements: foldersharing=1,videostreaming=all
tpmstate0: local:100/vm-100-disk-2.raw,size=16896,version=v2.0
vga: qxl,memory=64
vmgenid: 74d4bba3-bf66-4d86-af8a-e17241c4fe51

According to https://pve.proxmox.com/wiki/Storage, a Directory storage with QCOW2 disk should allow snapshots. And it does on my Linux VMs. But not Windows... and then I noticed the following tpmstate0: local:100/vm-100-disk-2.raw,size=16896,version=v2.0

Is it possible the reason why I cannot take snapshots on my Windows VMs is cuz the TPM State disk is RAW and not QCOW2?
If yes, how can I allow snapshots from there?
If no, then what prevents me from taking snapshots on this VM?


Thanks!

 

[FGD-Garuda]

It appears it does, TPM seems to be the problem when using Directory Storage...
I have switched back to lvm-thin by reinstalling Proxmox on a better machine and HD, I restored my Windows VM backup (zstd), log shows all fine, but it won't start at all. I'm guessing switching Storage type is not something that VM likes. However I can take Snapshots now, although it fails when it try to snapshot that TPM disk. But of course! lolll

I created a new VM, will attach the Windows qcow2 to it and see if it works.
If it doesn't, and for some reason I am expecting that, I will then try to figure out how to import (and not attach) only the Windows qcow2 disk (not the EFI nor the TPM). Hopefully a simple copy from shell will do the trick. If yes, that would be the solution to the OP. We shall see...

 

[FGD-Garuda]

Well while trying to extract zst and vma I ran into the famous issue of not having enough disk space on the pve/root. A lot of posts about that but nothing 100% clear for my use case, so I might as well write down the steps in case anyone has a similar use case and falls here in the future.

USE CASE
- Fresh install of Proxmox with 1 disk as ext4
- NO VM on the disk (that is an important assumption)
- Shows like this

# lvs
LV   VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
data pve twi-aotz-- 330.51g             0.00   0.51                           
root pve -wi-ao----  96.00g                                                   
swap pve -wi-ao----  16.00g

- I want to increase root to 150G
- Therefore that will reduce data to 280.51G

I have ssh to the machine, I do not know if you can simply use the Shell from within Promox.

lvremove /dev/pve/data
lvextend --resizefs --size 150G /dev/pve/root
lvcreate -L +280.51G -n data pve
lvconvert --type thin-pool pve/data

Then I went back in GUI, Datacenter/Add/LVM-Thin and fill up the ID, pool, etc.
You're done!

Now back on importing my disk contained in the zst.

 

[FGD-Garuda]

I failed.

I can't get the migrated VM from Directory to lvm-thin to boot up. SeaBIOS and OVMF don't want to boot. OVMF is a beast, 10s of posts on the forum about it not booting up, seems tricky, I tried anyway. Even booting from file like boot64.efi or whatever did not work.

I can't get Windows to reinstall either, says PC doesn't meet requirements. I bet it's TPM related, even though I do have the TPM 2.0 disk attached. I will hack into the ISO to remove the dependency, reinstall VM, re-use original disk UUID and if that doesn't work, then call my software provider and tell them I changed "computer" so I need my license to work on that new installation.

If anyone finds a solution plz post, I'm sure I and others will need to perform such type of migration someday.

EDIT: Seems I was right. https://dannyda.com/2021/10/19/how-...ration-does-not-support-taking-new-snapshots/


Lesson learned here: if you plan on using Snapshot feature on a Windows VM (due to TPM disk) and your Proxmox is installed on either ext4 or xfs, then do not use Directory storage, stick with the default lvm-thin that Proxmox installs.

 

[FGD-Garuda]

Jesus!! I made it!
It was the stupidest, easiest thing on earth!

Once you import your backup from your old Proxmox into the new Proxmox installation (remember, I changed physical machine), you just have to remove the EFI disk and add a new one, then boot!

How come after almost 2 full days working on this I haven't seen one single post, wiki page or google link mentioning that, oh well...

I am with SeaBIOS, not OVMF, but it works.
I just need to accelerate the VM, graphics are slow, as slow as they were on my old machine (10yo 2-core laptop!!). I know to improve performance there are VirtIO drivers to install or some tweaks to tick so I'll dig into that.

 

[cmax13007]

Unfortunately "tpmstate0: [file=]<volume> [,size=<DiskSize>] [,version=<v1.2|v2.0>] - Configure a Disk for storing TPM state. The format is fixed to raw." per the PROXMOX docs and its very annoying, https://pve.proxmox.com/wiki/Manual:_qm.conf.

To work with snapshots, I remove the tpmstate0 from the .conf file every time I take a snapshot. I added the .conf line to my VM notes so that I can recall it anytime and avoid confusion. When I need to restore a snapshot, I add the tmpstate0 back to the .conf file using WinSCP, and everything works fine. This method is inconvenient, but it works for snapshotting.

-CMAX

 

[DaCHeF36]

Unfortunately "tpmstate0: [file=]<volume> [,size=<DiskSize>] [,version=<v1.2|v2.0>] - Configure a Disk for storing TPM state. The format is fixed to raw." per the PROXMOX docs and its very annoying, https://pve.proxmox.com/wiki/Manual:_qm.conf.

To work with snapshots, I remove the tpmstate0 from the .conf file every time I take a snapshot. I added the .conf line to my VM notes so that I can recall it anytime and avoid confusion. When I need to restore a snapshot, I add the tmpstate0 back to the .conf file using WinSCP, and everything works fine. This method is inconvenient, but it works for snapshotting.

-CMAX

Perfect. Thank a lot.

 

[fabian]

you can use a storage that supports snapshots for raw images, such as ZFS, LVM-thin or Ceph. I think it should be (technically) possible to also support directory storages but it would need a mechanism to map the qcow2 image, since the software TPM only works with raw image files (it even used to only support directories, the raw image file support was contributed by us )

bug tracker entry: https://bugzilla.proxmox.com/show_bug.cgi?id=4693 please CC to keep updated with developements

 

[vix9]

Just adding into this thread now. Running Windows 11 on a ProxMox cluster with NFS backing is now a little bit more difficult. As I'm sure many people here do, I run ProxMox + TrueNAS NFS. I use the ZFS snapshots on TrueNAS as one level of protection, but being able to take a quick Snap of a Windows VM before doing something unusual or difficult was great. I've been trying to get ahead of the Windows 10 EoL and get 11 deployed, but now I am unable to snapshot in Proxmox without @cmax13007 config hack. Being able to store the TPM files as qcow2 would be a great addition.

 

[wrobelda]

you can use a storage that supports snapshots for raw images, such as ZFS, LVM-thin or Ceph. I think it should be (technically) possible to also support directory storages but it would need a mechanism to map the qcow2 image, since the software TPM only works with raw image files (it even used to only support directories, the raw image file support was contributed by us )

bug tracker entry: https://bugzilla.proxmox.com/show_bug.cgi?id=4693 please CC to keep updated with developements

I am on a ZFS storage and can't get Snapshots to run:

bios: ovmf
boot: order=virtio0
cores: 8
cpu: host
efidisk0: local:128/vm-128-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
hookscript: local:snippets/bridgefix.sh
machine: pc-q35-8.1
memory: 4096
meta: creation-qemu=8.1.2,ctime=1701210401
name: Win11New
net0: virtio=xx:xx:xx:xx:xx:xx,bridge=vmbr0
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=xxx-xx-xx-xx-xx
sockets: 1
tpmstate0: local:128/vm-128-disk-2.raw,size=4M,version=v2.0
vga: virtio
virtio0: local:128/vm-128-disk-1.qcow2,backup=0,discard=on,iothread=1,size=32G
vmgenid: xx-xx-xx-xx-xx

Any idea what's going on here? I am surprised even CLI offers a mere "snapshot feature is not available" as an error. A more verbose error couldn't hurt...

 

[wrobelda]

OK, I found this post https://forum.proxmox.com/threads/unable-to-take-vm-snapshot.107695/post-462960 and see that 'local' must be a separate ZFS pool, not just a directory located on Proxmox's own ZFS pool. Frankly speaking,tThis is quite frustrating, I did not see any mentioning of that anywhere in the docs. Especially the 7.1. Storage Types section of https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_storage_types doesn't make it any more clear on what should the strategy be here.

EDIT: I also found that local-zfs did exist originally, but was removed by proxmox when I added it (temporarily) to a cluster: https://forum.proxmox.com/threads/local-zfs-lost-after-joining-cluster.118492/

 

[Nimo]

is this being worked on or will this never be implemented?
I am using zfs over iscsi for Win11 and stored the tpm on local zfs because tpm does not support zfs over iscsi. Snapshots are not possible in this configuration and the hack to remove the tpm before taking snapshots is necessary. This is very unsatisfactory though and a solution to this would be highly appreciated.

 

[fabian]

please see the bugzilla entry linked higher up in this thread and subscribe to it if you want to be kept in the loop regarding a fix.

 

[kennysamuerto]

Unfortunately "tpmstate0: [file=]<volume> [,size=<DiskSize>] [,version=<v1.2|v2.0>] - Configure a Disk for storing TPM state. The format is fixed to raw." per the PROXMOX docs and its very annoying, https://pve.proxmox.com/wiki/Manual:_qm.conf.

To work with snapshots, I remove the tpmstate0 from the .conf file every time I take a snapshot. I added the .conf line to my VM notes so that I can recall it anytime and avoid confusion. When I need to restore a snapshot, I add the tmpstate0 back to the .conf file using WinSCP, and everything works fine. This method is inconvenient, but it works for snapshotting.

-CMAX

Hello,

I have many doubts about how to proceed correctly, so that TPM works, and does not generate problems, and to be able to make snapshots.

I understand that from the configuration file of the virtual machine, the line is removed. With this it has allowed me to make the snapshot, so far, perfect;

How should I proceed further? The configuration file now shows two configurations, the original, and the snapshot;

Do I have to add the line again for TPM to work normally, to the initial configuration part, or to the snapshot part?

And then, to recover the snapshot, you indicate that we have to add the line to the conf, but to which part? shouldn't we add it before to not lose this functionality?

 

[fabian]

there is no guarantee that the procedure you describe works, because the TPM state is not actually snapshotted in that case.. please wait for a proper fix to be developed unless you understand what is going on under the hood..