Dead containers after disk cloning

K

klausN00b

New Member
Nov 9, 2026
4
1
1
Good (morning, afternoon, or evening),

I’m writing to report what happened.

I started playing around with Proxmox 6 months ago—first with small test containers, then a few VMs, and finally some scripts from the community.



Now, after a lot of testing, I’m running 4 repurposed nodes with SSDs and salvaged RAM.



That’s exactly why I realized that one of my first nodes had a 128 GB drive.

I decided to buy three NVMe drives.

Today I decided to replace the first one.
I downloaded Clonezilla, looked up some info online, and saw that it was also recommended on this forum.
I cloned the drive, device-to-device in beginner mode.

Everything was OK according to Clonezilla.

I booted the NVMe drive, but the containers wouldn’t open.
When I ran lvs (if I remember correctly), the CTs showed 0.
So I cloned the drive again, copying at the bit level.
I left the other settings unchanged.
Same result.
So I figured, “Fine, I’ll create a backup of the containers and then reinstall them,” only to discover that the original SSD has the same problem.

Now, the fact that I’ll have to recreate those containers and that I didn’t make backups is my own fault.

I’m interested in understanding what went wrong, where I went wrong, and how to avoid this problem in the future.

If anyone could help me analyze the problem and figure out the cause, they’d be the best person in the world.

If there were a way to avoid having to re-import 30 GB of photos into Immich, it would be the kind of mistake we all wish we didn’t have to deal with.


Thanks to everyone for stopping by this thread.
 
Welcome Klaus,
klausN00b said:
Everything was OK according to Clonezilla.
Click to expand...
I do not use LVM but I believe to remember that it might have problems to handle LVM-thin correctly.

Try another tool. For example use classic "dd", e.g. dd if=/dev/mysourcedevice of=/dev/destinationdev bs=1M status=progress or ddrescue (which is my favorite).

You might also just restore from backup, that should always work on "new"/different hardware. (Edit: if we are not talking about the OS-device but the VM/LXC storage.)

klausN00b said:
I’m running 4 repurposed nodes
Click to expand...
In that case I would have created a cluster. This has a lot of requirements, new pitfalls and is possibly total overkill for your use case. But in that case you could have just "moved" a VM/Container from one node to another (and back, later on) using the PVE internal features :-)

In any case: have fun :-)
 
Last edited:
I admit I'm saying a little more than I know, but I recall that this could be a storage/volume UUID or device reference identification problem when an OS is booted with both the source and destination drives being present at the same time. So your containers should still be there, you just have to find them.
 
See also:
N

Thread 'Why healthy SSD's LVM corrupts after taking it out of host and inserting back?'

What may be the nature of the failure (beside unnoticed events I could not notice in the process)? The SSD is healthy.

1. Take the only SSD (LVM thin) out of proxmox host
2. Place it into another host
3. Boot new host into some Live USB environment, like Clonezilla
4. Clone the disk (deny fschk on source disk) into new/target disk (!the topic is about old/source disk, not this new one)
5. Try to put new disk into proxmox host, observe failures and give up trying to fix (root boots, but ext4 is not found on some partitions, complains about superblocks, etc)
6. Move original source disk...
 
  • Like
Reactions: Kingneutron and UdoB
in order
UdoB said:
I do not use LVM but I believe to remember that it might have problems to handle LVM-thin correctly.

Try another tool. For example use classic "dd", e.g. dd if=/dev/mysourcedevice of=/dev/destinationdev bs=1M status=progress or ddrescue (which is my favorite).

You might also just restore from backup, that should always work on "new"/different hardware. (Edit: if we are not talking about the OS-device but the VM/LXC storage.)
Click to expand...

Thanks for the reply.
If it happens again, I’ll definitely do that.
UdoB said:
In that case I would have created a cluster. This has a lot of requirements, new pitfalls and is possibly total overkill for your use case. But in that case you could have just "moved" a VM/Container from one node to another (and back, later on) using the PVE internal features :-)

In any case: have fun :-)
Click to expand...



As for the cluster, I might have gone overboard—a friend of mine says he would have done the same with two Kubernetes nodes and less hassle XD.
But I needed something challenging.
At work, I’m stuck between being a developer and handling requests as if I were a tech lead—without actually being one. I haven’t done anything new or stimulating in over a year now.

Thank you so much for taking the time to stop by and give me some constructive feedback.
 
J-Rod said:
I admit I'm saying a little more than I know, but I recall that this could be a storage/volume UUID or device reference identification problem when an OS is booted with both the source and destination drives being present at the same time. So your containers should still be there, you just have to find them.
Click to expand...
Thanks for the suggestion.
I tried several tools, but I didn't find anything on the disk.
Fortunately, the CTs on this node weren't important.

Thank you so much for taking the time to stop by and give me some constructive feedback.
 
Neobin said:
See also:
N

Thread 'Why healthy SSD's LVM corrupts after taking it out of host and inserting back?'

What may be the nature of the failure (beside unnoticed events I could not notice in the process)? The SSD is healthy.

1. Take the only SSD (LVM thin) out of proxmox host
2. Place it into another host
3. Boot new host into some Live USB environment, like Clonezilla
4. Clone the disk (deny fschk on source disk) into new/target disk (!the topic is about old/source disk, not this new one)
5. Try to put new disk into proxmox host, observe failures and give up trying to fix (root boots, but ext4 is not found on some partitions, complains about superblocks, etc)
6. Move original source disk...
Click to expand...
This describes my exact situation.
I'll take this to heart, and I think I'll spread the word everywhere online so those darn search engines will index it.
Don't clone Proxmox with Clonezilla.

Thank you so much for taking the time to stop by and give me some constructive feedback.
 
  • Like
Reactions: Kingneutron
FYI LVM and especially lvm-thin are abstractions, and have quirks. You can use e.g. REAR (Relax and Recover) to recreate the structure of the lvm (and -thin), but to properly recreate LXCs / VMs, the Right Way is to restore from backup (with PBS you get dedup) or migrate them to another server. You can do migration without a cluster by using PDM.

https://relax-and-recover.org/rear-user-guide/basics/getting-started.html

It creates a bootable ISO recovery image. You can test restores into a VM.

There are a few good reliable ways to clone proxmox (OS only), one is to use fsarchiver (I have scripts for this) and there is another way if you use ZFS boot/root. You could also conceivably use tar, but that is kind of the long way around and involves extra steps. Fsarchiver can also do cool things like restore ext4 to XFS on the fly, and has proven very consistent for me.

https://github.com/kneutron/ansitest/tree/master/proxmox

> If there were a way to avoid having to re-import 30 GB of photos into Immich, it would be the kind of mistake we all wish we didn’t have to deal with

Look into immich-go , you can set it up for bulk uploads and let it run overnight. Any customization you did such as Favorites will probably have to be re-done. At least you still have the original photos, and didn't rely on something crazy like a cloud-only backup.

https://github.com/simulot/immich-go

My advice is to re-deploy immich in a VM under docker, and back it up regularly. CasaOS makes this rather easy, but there are other alternatives
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back