How to remove local (Directory) storage properly?

iGadget

Member
Apr 9, 2020
26
5
8
42
In my attempts to understand how Proxmox storage works in a cluster, I took a spare disk and tried to add it through the web GUI via
node -> Disks -> Directory -> Create: Directory.
I selected my spare disk but the only filesystems I could pick were either EXT4 or XFS. I went for EXT4.

Now I've learned I want to use ZFS instead, so I need to remove the current configuration again. But there's nothing about this in the Proxmox Admin Guide.

So, how should I remove local (Directory) storage properly?
This is what I did / tried:

- Removed the storage on the Datacenter level. Result: it's still visible via node -> Disks -> Directory.
- At that same spot, it's impossible to remove anything, you can only add storage.
- The PVE Admin Guide states all storage configuration resides in /etc/pve/storage.conf, but in fact that file only contains cluster-wide mounts, *not* local mounts.
- The guide also states /etc/fstab, but it's empty, except for /proc.
- Seems it's actually in systemd as I discovered - in /etc/systemd/system/mult-user-target.wants/ to be precise:

Bash:
root@node2:/etc/systemd/system/multi-user.target.wants# ls |grep mount
mnt-pve-static_data.mount
Bingo. Let's remove it:

Bash:
root@node2:/etc/systemd/system/multi-user.target.wants# systemctl disable mnt-pve-static_data.mount
root@node2:/etc/systemd/system/multi-user.target.wants# systemctl stop mnt-pve-static_data.mount
Ok, now just remove the partition via cfdisk and we're done, right?
Wrong - the partition / directory is still visible in the web GUI. Hmmmm...
- Reboot the system then? Nope. Still there. What gives?!
- Searched for string 'static_data' in all files in /etc/. 1 result: /etc/systemd/system/mnt-pve-static_data.mount
It's inactive according to systemctl:

Bash:
root@node2:/etc/systemd/system/# systemctl |grep static_data.mount
root@node2:/etc/systemd/system/#

- Searched for string 'static_data' in all files in /var/ : only some results in the logs, nothing else.
- Searched for string 'static_data' in all files in /usr/ : only some results in binaries en libraries.
That can't be it either.

So where on earth does PVE store this information? And how do I remove it? What am I missing?
 

Dominic

Proxmox Retired Staff
Retired Staff
Mar 18, 2019
1,388
179
68
Note that the file you mention is a link
Code:
➜  ~ ls -lah /etc/systemd/system/multi-user.target.wants/mnt-pve-static_data.mount
lrwxrwxrwx 1 root root 45 May 20 09:11 /etc/systemd/system/multi-user.target.wants/mnt-pve-static_data.mount -> /etc/systemd/system/mnt-pve-static_data.mount

Please try the following
  1. Remove the storage static_data from /etc/pve/storage.cfg if not done so yet. This is the same as removing it in Datacenter->Storage in the GUI.
  2. Remove the systemd file rm /etc/systemd/system/mnt-pve-static_data.mount. This should remove the entry from the GUI.
  3. Unmount the storage umount /mnt/pve/static_data (or reboot)
  4. Wipe file systems on the disk wipefs --no-act --backup /dev/sdb Replace the disk with your real disk and --no-act with --all.
You should then be able to create a ZFS storage in the GUI: Node->Disks->ZFS->Create: ZFS button
 
Last edited:
  • Like
Reactions: iGadget

iGadget

Member
Apr 9, 2020
26
5
8
42
Thanks for your reply, @Dominic !
Note that the file you mention is a link
Yes, I was aware of that. I just didn't know that leaving the original file in /etc/systemd/system would cause the disk still to appear in node -> Disks -> Directory.
1. Remove the storage static_data from /etc/pve/storage.cfg if not done so yet. This is the same as removing it in Datacenter->Storage in the GUI.
That's the first thing I did, expecting it to make the storage 'removable' on the node-level as well, but alas.
2. Remove the systemd file rm /etc/systemd/system/mnt-pve-static_data.mount. This should remove the entry from the GUI.
YES! That was what I was looking for. IMHO this *really* needs to be in the Admin Guide :)
3. Unmount the storage umount /mnt/pve/static_data (or reboot)
IMHO this should be done before step 2, via systemctl disable mnt-pve-static_data.mount --now. Then a reboot is not needed either.
4. Wipe file systems on the disk wipefs --no-act --backup /dev/sdb Replace the disk with your real disk and --no-act with --all.
You should then be able to create a ZFS storage in the GUI: Node->Disks->ZFS->Create: ZFS button
Actually, I used cfdisk for this, but it's nice to have alternatives.

As for this info not being in the Admin Guide, do you want me to file a bug for that in Bugzilla?
 

kenkin360

New Member
Oct 12, 2021
9
3
3
41
For PVE7:

Either via the direct console or SSH

Code:
nano /usr/share/perl5/PVE/Storage/Plugin.pm

Press [ctrl+w] and type '/var/lib/vz' (quotes not included) then press [Enter] to find the nearby location we're going to comment out

Add '=comment-out' (quotes not included) at the beginning and above the line
Code:
# make sure we have a reasonable 'local:' storage

Add '=cut' (quotes not included) at the beginning and above the line
Code:
foreach my $storeid (keys %$ids) {

So that the sub `parse_config` may look like

Perl:
sub parse_config {
    my ($class, $filename, $raw) = @_;

    my $cfg = $class->SUPER::parse_config($filename, $raw);
    my $ids = $cfg->{ids};
=comment-out
    # make sure we have a reasonable 'local:' storage
    # we want 'local' to be always the same 'type' (on all cluster nodes)
    if (!$ids->{local} || $ids->{local}->{type} ne 'dir' ||
        ($ids->{local}->{path} && $ids->{local}->{path} ne '/var/lib/vz')) {
        $ids->{local} = {
            type => 'dir',
            priority => 0, # force first entry
            path => '/var/lib/vz',
            'prune-backups' => 'keep-all=1',
            content => {
                backup => 1,
                images => 1,
                iso => 1,
                rootdir => 1,
                snippets => 1,
                vztmpl => 1,
            },
        };
    }

    # make sure we have a path
    $ids->{local}->{path} = '/var/lib/vz' if !$ids->{local}->{path};

    # remove node restrictions for local storage
    delete($ids->{local}->{nodes});
=cut
    foreach my $storeid (keys %$ids) {
        my $d = $ids->{$storeid};
        my $type = $d->{type};

        my $def = $defaultData->{plugindata}->{$type};

        if ($def->{content}) {
            $d->{content} = $def->{content}->[1] if !$d->{content};
        }
        if (grep { $_ eq $type }  @SHARED_STORAGE) {
            $d->{shared} = 1;
        }
    }

    return $cfg;
}

And then reboot the host.

Now you can run `pvesm remove local` via the console or through the Web UI to remove the `local` storage.
 
  • Like
Reactions: hyperbola

hyperbola

New Member
Feb 20, 2022
2
0
1
23
For PVE7:

Either via the direct console or SSH

Code:
nano /usr/share/perl5/PVE/Storage/Plugin.pm

Press [ctrl+w] and type '/var/lib/vz' (quotes not included) then press [Enter] to find the nearby location we're going to comment out

Add '=comment-out' (quotes not included) at the beginning and above the line
Code:
# make sure we have a reasonable 'local:' storage

Add '=cut' (quotes not included) at the beginning and above the line
Code:
foreach my $storeid (keys %$ids) {

So that the sub `parse_config` may look like

Perl:
sub parse_config {
    my ($class, $filename, $raw) = @_;

    my $cfg = $class->SUPER::parse_config($filename, $raw);
    my $ids = $cfg->{ids};
=comment-out
    # make sure we have a reasonable 'local:' storage
    # we want 'local' to be always the same 'type' (on all cluster nodes)
    if (!$ids->{local} || $ids->{local}->{type} ne 'dir' ||
        ($ids->{local}->{path} && $ids->{local}->{path} ne '/var/lib/vz')) {
        $ids->{local} = {
            type => 'dir',
            priority => 0, # force first entry
            path => '/var/lib/vz',
            'prune-backups' => 'keep-all=1',
            content => {
                backup => 1,
                images => 1,
                iso => 1,
                rootdir => 1,
                snippets => 1,
                vztmpl => 1,
            },
        };
    }

    # make sure we have a path
    $ids->{local}->{path} = '/var/lib/vz' if !$ids->{local}->{path};

    # remove node restrictions for local storage
    delete($ids->{local}->{nodes});
=cut
    foreach my $storeid (keys %$ids) {
        my $d = $ids->{$storeid};
        my $type = $d->{type};

        my $def = $defaultData->{plugindata}->{$type};

        if ($def->{content}) {
            $d->{content} = $def->{content}->[1] if !$d->{content};
        }
        if (grep { $_ eq $type }  @SHARED_STORAGE) {
            $d->{shared} = 1;
        }
    }

    return $cfg;
}

And then reboot the host.

Now you can run `pvesm remove local` via the console or through the Web UI to remove the `local` storage.
This indeed works!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!