Erase Data fills RAM

S

SL-FH

New Member
Proxmox Subscriber
Jul 3, 2025
9
1
3
I use PVE 9 with an DELL ME4 iSCSI Storage. This storage is thin provisioned but don't handle unmap command. To avoid data loss zeroing out removed disks is enabled.

storage.cfg
Code: 
lvm: pve001
        vgname vg_pve001
        base st001_01:0.0.21.scsi-3600c0ff00043f8ec590f3f6801000000
        content images,rootdir
        saferemove 1
        saferemove_throughput -1073741824
        shared 1

This task don't check if the disk is enabled and writes into RAM until is full.

zero-out data on image vm-28999-disk-2 (/dev/vg_rtx2st001_pve001/del-vm-28999-disk-2)


/dev is a RAM FS and I think before deletion the LV isn't enabled and the fill process needs to avoid creating files.
 
We're experiencing exactly the same issue, but in our case it's with LVM over IBM FlashSystem via Fibre Channel.

After upgrading to Proxmox VE 9, we’ve noticed that some LVM volumes remain inactive, they are not recognized as real block devices by the node. When one of those inactive volumes is deleted, instead of activating the LVM and writing zeros to the underlying device, the erase process creates a file under /dev/, which resides in RAM.

As a result, the system quickly fills up all available RAM, leading to out-of-memory conditions and even virtual machines being stopped when the node memory is completely exhausted.

This behavior did not occur before the upgrade, and it appears related to how Proxmox 9 handles inactive LVM volumes during erase/zero-out operations.
 
We’ve been debugging the recent “erase data fills RAM” issue and found a clear behavioral change in the LVM storage plugin after the latest refactor.

In the new implementation, the logic of free_image and free_lvm_volumes was heavily rewritten to support qcow2 and snapshot handling.
However, this introduced a functional regression affecting raw volumes.

In the previous implementation (free_image), the LV was always activated and refreshed before performing the zero-out or removal operation:

Perl: 
my $cmd = ['/sbin/lvchange', '-aly', "$vg/$volname"];
run_command($cmd, errmsg => "can't activate LV '$vg/$volname' to zero-out its data");
$cmd = ['/sbin/lvchange', '--refresh', "$vg/$volname"];
run_command($cmd, errmsg => "can't refresh LV '$vg/$volname' to zero-out its data");

In the new code, the activation logic is now inside an if ($format eq 'qcow2') block, which means that:

  • If the format is qcow2, the LV gets properly activated.
  • If the format is raw, that block is skipped entirely, and execution continues directly to free_lvm_volumes().

Perl: 
if ($format eq 'qcow2') {
    ...
    my $cmd = ['/sbin/lvchange', '-aly', $path];
    run_command($cmd, errmsg => "can't activate LV '$path' to zero-out its data");
    ...
}
return free_lvm_volumes($class, $scfg, $storeid, $volnames);

"free_lvm_volumes" never checks if the logical volume is active or visible before trying to zero it out.

When the LV is inactive, this path resolves under /dev, and the zeroing process writes directly into RAM — filling memory completely and triggering OOM conditions on the node.

Code References:
free_image: https://git.proxmox.com/?p=pve-stor...2cd9f5e0111b784fe7c;hb=refs/heads/master#l697

free_lvm_volumes: https://git.proxmox.com/?p=pve-stor...2cd9f5e0111b784fe7c;hb=refs/heads/master#l282
 
spirit said:
Hi,thanks for the report. I have send patch to the mailing recently to improve the performance of secure erase, it's not yet commited. But I didn't notice this bug with volume activation. I'll look at it next week. (and I'll forward it the dev mailing)
Click to expand...
First of all, thanks for the quick response and for confirming that the issue will be reviewed by the dev team.



DANGER

If you don’t have advanced Linux knowledge, stop reading here and do not apply this patch. Wait for the official fix to be released through the Proxmox repositories.

DANGER​


Important notice:
  • This is an unofficial temporary/emergency workaround.
  • Do not apply this in production unless you fully understand the risks and need an urgent fix.
  • It’s always better to wait for the official upstream patch from the Proxmox team.

For anyone affected by the same problem and needing an emergency workaround before the official fix lands, here’s what we’ve tested in our environment (LVM over Fibre Channel with saferemove=1):

Add the following lines to /usr/share/perl5/PVE/Storage/LVMPlugin.pm inside the free_lvm_volumes function:

Diff: 
--- LVMPlugin.pm.original       2025-10-17 12:23:51.016231633 +0200
+++ LVMPlugin.pm.workaround     2025-10-17 12:25:23.495587526 +0200
@@ -295,6 +295,14 @@
         for my $name (@$volnames) {
             print "zero-out data on image $name (/dev/$vg/del-$name)\n";

+            my $act_del = ['/sbin/lvchange', '-aly', "$vg/del-$name"];
+            eval { run_command($act_del, errmsg => "can't activate LV '$vg/del-$name' before zero-out") };
+            warn $@ if $@;
+
+            my $ref_del = ['/sbin/lvchange', '--refresh', "$vg/del-$name"];
+            eval { run_command($ref_del, errmsg => "can't refresh LV '$vg/del-$name' before zero-out") };
+            warn $@ if $@;
+
             my $cmd = [
                 '/usr/bin/cstream',
                 '-i',
@@ -334,6 +342,15 @@
     if ($scfg->{saferemove}) {
         for my $name (@$volnames) {
             # avoid long running task, so we only rename here
+
+            my $act = ['/sbin/lvchange', '-aly', "$vg/$name"];
+            eval { run_command($act, errmsg => "can't activate LV '$vg/$name' before rename") };
+            warn $@ if $@;
+
+            my $ref = ['/sbin/lvchange', '--refresh', "$vg/$name"];
+            eval { run_command($ref, errmsg => "can't refresh LV '$vg/$name' before rename") };
+            warn $@ if $@;
+
             my $cmd = ['/sbin/lvrename', $vg, $name, "del-$name"];
             run_command($cmd, errmsg => "lvrename '$vg/$name' error");
         }


After editing the file, restart the node or the Proxmox services to reload the storage plugin.
This ensures the LVM volumes are explicitly activated before renaming or zero-out, preventing /dev from filling the RAM during saferemove.
 
  • Like
Reactions: fba
You must log in or register to reply here.
Top Bottom