How to remove local (Directory) storage properly?

iGadget · May 19, 2021

In my attempts to understand how Proxmox storage works in a cluster, I took a spare disk and tried to add it through the web GUI via
node -> Disks -> Directory -> Create: Directory.
I selected my spare disk but the only filesystems I could pick were either EXT4 or XFS. I went for EXT4.

Now I've learned I want to use ZFS instead, so I need to remove the current configuration again. But there's nothing about this in the Proxmox Admin Guide.

So, how should I remove local (Directory) storage properly?
This is what I did / tried:

- Removed the storage on the Datacenter level. Result: it's still visible via node -> Disks -> Directory.
- At that same spot, it's impossible to remove anything, you can only add storage.
- The PVE Admin Guide states all storage configuration resides in /etc/pve/storage.conf, but in fact that file only contains cluster-wide mounts, *not* local mounts.
- The guide also states /etc/fstab, but it's empty, except for /proc.
- Seems it's actually in systemd as I discovered - in /etc/systemd/system/mult-user-target.wants/ to be precise:

Bash:

root@node2:/etc/systemd/system/multi-user.target.wants# ls |grep mount
mnt-pve-static_data.mount

Bingo. Let's remove it:

Bash:

root@node2:/etc/systemd/system/multi-user.target.wants# systemctl disable mnt-pve-static_data.mount
root@node2:/etc/systemd/system/multi-user.target.wants# systemctl stop mnt-pve-static_data.mount

Ok, now just remove the partition via cfdisk and we're done, right?
Wrong - the partition / directory is still visible in the web GUI. Hmmmm...
- Reboot the system then? Nope. Still there. What gives?!
- Searched for string 'static_data' in all files in /etc/. 1 result: /etc/systemd/system/mnt-pve-static_data.mount
It's inactive according to systemctl:

Bash:

root@node2:/etc/systemd/system/# systemctl |grep static_data.mount
root@node2:/etc/systemd/system/#

- Searched for string 'static_data' in all files in /var/ : only some results in the logs, nothing else.
- Searched for string 'static_data' in all files in /usr/ : only some results in binaries en libraries.
That can't be it either.

So where on earth does PVE store this information? And how do I remove it? What am I missing?

Dominic · May 20, 2021

Note that the file you mention is a link

Code:

➜  ~ ls -lah /etc/systemd/system/multi-user.target.wants/mnt-pve-static_data.mount
lrwxrwxrwx 1 root root 45 May 20 09:11 /etc/systemd/system/multi-user.target.wants/mnt-pve-static_data.mount -> /etc/systemd/system/mnt-pve-static_data.mount

Please try the following

Remove the storage static_data from /etc/pve/storage.cfg if not done so yet. This is the same as removing it in Datacenter->Storage in the GUI.
Remove the systemd file rm /etc/systemd/system/mnt-pve-static_data.mount. This should remove the entry from the GUI.
Unmount the storage umount /mnt/pve/static_data (or reboot)
Wipe file systems on the disk wipefs --no-act --backup /dev/sdb Replace the disk with your real disk and --no-act with --all.

You should then be able to create a ZFS storage in the GUI: Node->Disks->ZFS->Create: ZFS button

iGadget · May 20, 2021

Thanks for your reply, @Dominic !

Dominic said:
Note that the file you mention is a link

Yes, I was aware of that. I just didn't know that leaving the original file in /etc/systemd/system would cause the disk still to appear in node -> Disks -> Directory.

Dominic said:
1. Remove the storage static_data from /etc/pve/storage.cfg if not done so yet. This is the same as removing it in Datacenter->Storage in the GUI.

That's the first thing I did, expecting it to make the storage 'removable' on the node-level as well, but alas.

Dominic said:
2. Remove the systemd file rm /etc/systemd/system/mnt-pve-static_data.mount. This should remove the entry from the GUI.

YES! That was what I was looking for. IMHO this *really* needs to be in the Admin Guide

Dominic said:
3. Unmount the storage umount /mnt/pve/static_data (or reboot)

IMHO this should be done before step 2, via systemctl disable mnt-pve-static_data.mount --now. Then a reboot is not needed either.

Dominic said:
4. Wipe file systems on the disk wipefs --no-act --backup /dev/sdb Replace the disk with your real disk and --no-act with --all.
You should then be able to create a ZFS storage in the GUI: Node->Disks->ZFS->Create: ZFS button

Actually, I used cfdisk for this, but it's nice to have alternatives.

As for this info not being in the Admin Guide, do you want me to file a bug for that in Bugzilla?

Dominic · May 21, 2021

Great to hear that it worked!

do you want me to file a bug for that in Bugzilla?

Not necessary, I just sent a patch

https://lists.proxmox.com/pipermail/pve-devel/2021-May/048259.html

kenkin360 · Oct 12, 2021

For PVE7:

Either via the direct console or SSH

Code:

nano /usr/share/perl5/PVE/Storage/Plugin.pm

Press [ctrl+w] and type '/var/lib/vz' (quotes not included) then press [Enter] to find the nearby location we're going to comment out

Add '=comment-out' (quotes not included) at the beginning and above the line

Code:

# make sure we have a reasonable 'local:' storage

Add '=cut' (quotes not included) at the beginning and above the line

Code:

foreach my $storeid (keys %$ids) {

So that the sub `parse_config` may look like

Perl:

sub parse_config {
    my ($class, $filename, $raw) = @_;

    my $cfg = $class->SUPER::parse_config($filename, $raw);
    my $ids = $cfg->{ids};
=comment-out
    # make sure we have a reasonable 'local:' storage
    # we want 'local' to be always the same 'type' (on all cluster nodes)
    if (!$ids->{local} || $ids->{local}->{type} ne 'dir' ||
        ($ids->{local}->{path} && $ids->{local}->{path} ne '/var/lib/vz')) {
        $ids->{local} = {
            type => 'dir',
            priority => 0, # force first entry
            path => '/var/lib/vz',
            'prune-backups' => 'keep-all=1',
            content => {
                backup => 1,
                images => 1,
                iso => 1,
                rootdir => 1,
                snippets => 1,
                vztmpl => 1,
            },
        };
    }

    # make sure we have a path
    $ids->{local}->{path} = '/var/lib/vz' if !$ids->{local}->{path};

    # remove node restrictions for local storage
    delete($ids->{local}->{nodes});
=cut
    foreach my $storeid (keys %$ids) {
        my $d = $ids->{$storeid};
        my $type = $d->{type};

        my $def = $defaultData->{plugindata}->{$type};

        if ($def->{content}) {
            $d->{content} = $def->{content}->[1] if !$d->{content};
        }
        if (grep { $_ eq $type }  @SHARED_STORAGE) {
            $d->{shared} = 1;
        }
    }

    return $cfg;
}

And then reboot the host.

Now you can run `pvesm remove local` via the console or through the Web UI to remove the `local` storage.

hyperbola · Feb 20, 2022

kenkin360 said:

For PVE7:

Either via the direct console or SSH

Code:

nano /usr/share/perl5/PVE/Storage/Plugin.pm

Press [ctrl+w] and type '/var/lib/vz' (quotes not included) then press [Enter] to find the nearby location we're going to comment out

Add '=comment-out' (quotes not included) at the beginning and above the line

Code:

# make sure we have a reasonable 'local:' storage

Add '=cut' (quotes not included) at the beginning and above the line

Code:

foreach my $storeid (keys %$ids) {

So that the sub `parse_config` may look like

Perl:

sub parse_config {
    my ($class, $filename, $raw) = @_;

    my $cfg = $class->SUPER::parse_config($filename, $raw);
    my $ids = $cfg->{ids};
=comment-out
    # make sure we have a reasonable 'local:' storage
    # we want 'local' to be always the same 'type' (on all cluster nodes)
    if (!$ids->{local} || $ids->{local}->{type} ne 'dir' ||
        ($ids->{local}->{path} && $ids->{local}->{path} ne '/var/lib/vz')) {
        $ids->{local} = {
            type => 'dir',
            priority => 0, # force first entry
            path => '/var/lib/vz',
            'prune-backups' => 'keep-all=1',
            content => {
                backup => 1,
                images => 1,
                iso => 1,
                rootdir => 1,
                snippets => 1,
                vztmpl => 1,
            },
        };
    }

    # make sure we have a path
    $ids->{local}->{path} = '/var/lib/vz' if !$ids->{local}->{path};

    # remove node restrictions for local storage
    delete($ids->{local}->{nodes});
=cut
    foreach my $storeid (keys %$ids) {
        my $d = $ids->{$storeid};
        my $type = $d->{type};

        my $def = $defaultData->{plugindata}->{$type};

        if ($def->{content}) {
            $d->{content} = $def->{content}->[1] if !$d->{content};
        }
        if (grep { $_ eq $type }  @SHARED_STORAGE) {
            $d->{shared} = 1;
        }
    }

    return $cfg;
}

And then reboot the host.

Now you can run `pvesm remove local` via the console or through the Web UI to remove the `local` storage.

This indeed works!

Tim Hörnle - UJAM · Aug 1, 2023

It would be great if the Proxmox team could implement that we could just use "pvesm remove local" without having to edit the system source code.

Search

Search

How to remove local (Directory) storage properly?

iGadget

Member

Dominic

Proxmox Retired Staff

iGadget

Member

Dominic

Proxmox Retired Staff

kenkin360

New Member

hyperbola

New Member

Tim Hörnle - UJAM

Member

We value your privacy