Summary:
When using Proxmox VE 9.2.3 with local Btrfs storage, LXC containers whose ext4 root filesystem is created with the `mmp` feature exhibit an approximately 43 second delay on the first mount/start after rolling back to a snapshot.
Removing the `mmp` feature from the container's ext4 filesystem eliminates the delay completely.
The delay appears to correspond to ext4's Multiple Mount Protection (MMP) timeout.
Environment:
Proxmox VE: 9.2.3
Kernel: 7.0.12-1-pve
Storage backend: local-btrfs
Root filesystem: Btrfs (installed using the automated answer file)
Container template: debian-13-standard_13.1-2_amd64.tar.zst
Command used for container creation:
```
pct create 111 local-btrfs:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst \
--hostname hermes \
--unprivileged 1 \
--cores 2 --memory 2048 --swap 512 \
--rootfs local-btrfs:16 \
--net0 name=eth0,bridge=vmbr0,ip=192.168.1.111/24,gw=192.168.1.1 \
--features nesting=1 \
--onboot 1 \
--ssh-public-keys /root/ssh_key.pub
```
Reproduction:
1. Create container.
2. Start container.
3. Create snapshot.
4. Modify any file inside the container.
5. Roll back to the snapshot.
6. Start or mount the container.
Example:
```
pct snapshot 111 test
pct enter 111
touch hello
exit
pct rollback 111 test
time pct start 111
```
or
```
time pct mount 111
```
---
Observed behavior:
The first mount/start after rollback consistently takes approximately:
```
43-44 seconds
```
Subsequent starts are immediate.
Boot inside the container is normal:
```
systemd-analyze
Startup finished in ~800 ms
```
The delay occurs before userspace starts.
Investigation:
The delay can be reproduced outside Proxmox by mounting the container filesystem manually.
```
LOOP=$(losetup -Pf --show $(pvesm path local-btrfs:111/vm-111-disk-0.raw))
time mount "$LOOP" /mnt/test
```
Result:
```
~43 seconds
```
Therefore the delay is not caused by LXC startup itself.
e2fsck output:
Running:
```
e2fsck -fn "$LOOP"
```
produces:
```
MMP interval is 10 seconds and total wait time is 42 seconds.
Please wait...
```
which closely matches the observed mount delay.
ext4 features:
The filesystem created by Proxmox contains:
```
Filesystem features:
...
mmp
...
```
Proxmox source:
The installed Proxmox source contains:
```perl
'mkfs.ext4', '-O', 'mmp', '-E', "root_owner=$root_uid:$root_gid", $dev,
```
located in:
```
/usr/share/perl5/PVE/LXC.pm
```
Searching the installed Perl sources shows this is the only reference to `mmp`.
Experiment:
After creating a fresh container, before taking any snapshots, I removed the MMP feature:
```
LOOP=$(losetup -Pf --show $(pvesm path local-btrfs:111/vm-111-disk-0.raw))
tune2fs -O ^mmp "$LOOP"
e2fsck -fy "$LOOP"
losetup -d "$LOOP"
```
Verification:
```
tune2fs -l
```
shows the `mmp` feature is no longer present.
I then:
1. Started the container.
2. Created a snapshot.
3. Modified files.
4. Rolled back.
5. Started the container again.
Result:
```
time pct start 111
real 0m1.773s
```
The ~43 second delay disappeared completely.
Question:
Is enabling `-O mmp` for every ext4 LXC root filesystem intentional?
If so, is the 42-43 second mount delay after snapshot rollback expected?
If not, should `mmp` perhaps be omitted for loop-backed local LXC root filesystems, where multiple concurrent mounts cannot normally occur?
Thank you all
Akash
When using Proxmox VE 9.2.3 with local Btrfs storage, LXC containers whose ext4 root filesystem is created with the `mmp` feature exhibit an approximately 43 second delay on the first mount/start after rolling back to a snapshot.
Removing the `mmp` feature from the container's ext4 filesystem eliminates the delay completely.
The delay appears to correspond to ext4's Multiple Mount Protection (MMP) timeout.
Environment:
Proxmox VE: 9.2.3
Kernel: 7.0.12-1-pve
Storage backend: local-btrfs
Root filesystem: Btrfs (installed using the automated answer file)
Container template: debian-13-standard_13.1-2_amd64.tar.zst
Command used for container creation:
```
pct create 111 local-btrfs:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst \
--hostname hermes \
--unprivileged 1 \
--cores 2 --memory 2048 --swap 512 \
--rootfs local-btrfs:16 \
--net0 name=eth0,bridge=vmbr0,ip=192.168.1.111/24,gw=192.168.1.1 \
--features nesting=1 \
--onboot 1 \
--ssh-public-keys /root/ssh_key.pub
```
Reproduction:
1. Create container.
2. Start container.
3. Create snapshot.
4. Modify any file inside the container.
5. Roll back to the snapshot.
6. Start or mount the container.
Example:
```
pct snapshot 111 test
pct enter 111
touch hello
exit
pct rollback 111 test
time pct start 111
```
or
```
time pct mount 111
```
---
Observed behavior:
The first mount/start after rollback consistently takes approximately:
```
43-44 seconds
```
Subsequent starts are immediate.
Boot inside the container is normal:
```
systemd-analyze
Startup finished in ~800 ms
```
The delay occurs before userspace starts.
Investigation:
The delay can be reproduced outside Proxmox by mounting the container filesystem manually.
```
LOOP=$(losetup -Pf --show $(pvesm path local-btrfs:111/vm-111-disk-0.raw))
time mount "$LOOP" /mnt/test
```
Result:
```
~43 seconds
```
Therefore the delay is not caused by LXC startup itself.
e2fsck output:
Running:
```
e2fsck -fn "$LOOP"
```
produces:
```
MMP interval is 10 seconds and total wait time is 42 seconds.
Please wait...
```
which closely matches the observed mount delay.
ext4 features:
The filesystem created by Proxmox contains:
```
Filesystem features:
...
mmp
...
```
Proxmox source:
The installed Proxmox source contains:
```perl
'mkfs.ext4', '-O', 'mmp', '-E', "root_owner=$root_uid:$root_gid", $dev,
```
located in:
```
/usr/share/perl5/PVE/LXC.pm
```
Searching the installed Perl sources shows this is the only reference to `mmp`.
Experiment:
After creating a fresh container, before taking any snapshots, I removed the MMP feature:
```
LOOP=$(losetup -Pf --show $(pvesm path local-btrfs:111/vm-111-disk-0.raw))
tune2fs -O ^mmp "$LOOP"
e2fsck -fy "$LOOP"
losetup -d "$LOOP"
```
Verification:
```
tune2fs -l
```
shows the `mmp` feature is no longer present.
I then:
1. Started the container.
2. Created a snapshot.
3. Modified files.
4. Rolled back.
5. Started the container again.
Result:
```
time pct start 111
real 0m1.773s
```
The ~43 second delay disappeared completely.
Question:
Is enabling `-O mmp` for every ext4 LXC root filesystem intentional?
If so, is the 42-43 second mount delay after snapshot rollback expected?
If not, should `mmp` perhaps be omitted for loop-backed local LXC root filesystems, where multiple concurrent mounts cannot normally occur?
Thank you all
Akash