How to remove local (Directory) storage properly?

I

iGadget

Member
Apr 9, 2020
26
7
8
44
In my attempts to understand how Proxmox storage works in a cluster, I took a spare disk and tried to add it through the web GUI via
node -> Disks -> Directory -> Create: Directory.
I selected my spare disk but the only filesystems I could pick were either EXT4 or XFS. I went for EXT4.

Now I've learned I want to use ZFS instead, so I need to remove the current configuration again. But there's nothing about this in the Proxmox Admin Guide.

So, how should I remove local (Directory) storage properly?
This is what I did / tried:

- Removed the storage on the Datacenter level. Result: it's still visible via node -> Disks -> Directory.
- At that same spot, it's impossible to remove anything, you can only add storage.
- The PVE Admin Guide states all storage configuration resides in /etc/pve/storage.conf, but in fact that file only contains cluster-wide mounts, *not* local mounts.
- The guide also states /etc/fstab, but it's empty, except for /proc.
- Seems it's actually in systemd as I discovered - in /etc/systemd/system/mult-user-target.wants/ to be precise:

Bash: 
root@node2:/etc/systemd/system/multi-user.target.wants# ls |grep mount
mnt-pve-static_data.mount
Bingo. Let's remove it:

Bash: 
root@node2:/etc/systemd/system/multi-user.target.wants# systemctl disable mnt-pve-static_data.mount
root@node2:/etc/systemd/system/multi-user.target.wants# systemctl stop mnt-pve-static_data.mount
Ok, now just remove the partition via cfdisk and we're done, right?
Wrong - the partition / directory is still visible in the web GUI. Hmmmm...
- Reboot the system then? Nope. Still there. What gives?!
- Searched for string 'static_data' in all files in /etc/. 1 result: /etc/systemd/system/mnt-pve-static_data.mount
It's inactive according to systemctl:

Bash: 
root@node2:/etc/systemd/system/# systemctl |grep static_data.mount
root@node2:/etc/systemd/system/#

- Searched for string 'static_data' in all files in /var/ : only some results in the logs, nothing else.
- Searched for string 'static_data' in all files in /usr/ : only some results in binaries en libraries.
That can't be it either.

So where on earth does PVE store this information? And how do I remove it? What am I missing?
 
Note that the file you mention is a link
Code: 
➜  ~ ls -lah /etc/systemd/system/multi-user.target.wants/mnt-pve-static_data.mount
lrwxrwxrwx 1 root root 45 May 20 09:11 /etc/systemd/system/multi-user.target.wants/mnt-pve-static_data.mount -> /etc/systemd/system/mnt-pve-static_data.mount

Please try the following
  1. Remove the storage static_data from /etc/pve/storage.cfg if not done so yet. This is the same as removing it in Datacenter->Storage in the GUI.
  2. Remove the systemd file rm /etc/systemd/system/mnt-pve-static_data.mount. This should remove the entry from the GUI.
  3. Unmount the storage umount /mnt/pve/static_data (or reboot)
  4. Wipe file systems on the disk wipefs --no-act --backup /dev/sdb Replace the disk with your real disk and --no-act with --all.
You should then be able to create a ZFS storage in the GUI: Node->Disks->ZFS->Create: ZFS button
 
Last edited:
  • Like
Reactions: iGadget
Thanks for your reply, @Dominic !
Dominic said:
Note that the file you mention is a link
Click to expand...
Yes, I was aware of that. I just didn't know that leaving the original file in /etc/systemd/system would cause the disk still to appear in node -> Disks -> Directory.
Dominic said:
1. Remove the storage static_data from /etc/pve/storage.cfg if not done so yet. This is the same as removing it in Datacenter->Storage in the GUI.
Click to expand...
That's the first thing I did, expecting it to make the storage 'removable' on the node-level as well, but alas.
Dominic said:
2. Remove the systemd file rm /etc/systemd/system/mnt-pve-static_data.mount. This should remove the entry from the GUI.
Click to expand...
YES! That was what I was looking for. IMHO this *really* needs to be in the Admin Guide :)
Dominic said:
3. Unmount the storage umount /mnt/pve/static_data (or reboot)
Click to expand...
IMHO this should be done before step 2, via systemctl disable mnt-pve-static_data.mount --now. Then a reboot is not needed either.
Dominic said:
4. Wipe file systems on the disk wipefs --no-act --backup /dev/sdb Replace the disk with your real disk and --no-act with --all.
You should then be able to create a ZFS storage in the GUI: Node->Disks->ZFS->Create: ZFS button
Click to expand...
Actually, I used cfdisk for this, but it's nice to have alternatives.

As for this info not being in the Admin Guide, do you want me to file a bug for that in Bugzilla?
 
For PVE7:

Either via the direct console or SSH

Code: 
nano /usr/share/perl5/PVE/Storage/Plugin.pm

Press [ctrl+w] and type '/var/lib/vz' (quotes not included) then press [Enter] to find the nearby location we're going to comment out

Add '=comment-out' (quotes not included) at the beginning and above the line
Code: 
# make sure we have a reasonable 'local:' storage

Add '=cut' (quotes not included) at the beginning and above the line
Code: 
foreach my $storeid (keys %$ids) {

So that the sub `parse_config` may look like

Perl: 
sub parse_config {
    my ($class, $filename, $raw) = @_;

    my $cfg = $class->SUPER::parse_config($filename, $raw);
    my $ids = $cfg->{ids};
=comment-out
    # make sure we have a reasonable 'local:' storage
    # we want 'local' to be always the same 'type' (on all cluster nodes)
    if (!$ids->{local} || $ids->{local}->{type} ne 'dir' ||
        ($ids->{local}->{path} && $ids->{local}->{path} ne '/var/lib/vz')) {
        $ids->{local} = {
            type => 'dir',
            priority => 0, # force first entry
            path => '/var/lib/vz',
            'prune-backups' => 'keep-all=1',
            content => {
                backup => 1,
                images => 1,
                iso => 1,
                rootdir => 1,
                snippets => 1,
                vztmpl => 1,
            },
        };
    }

    # make sure we have a path
    $ids->{local}->{path} = '/var/lib/vz' if !$ids->{local}->{path};

    # remove node restrictions for local storage
    delete($ids->{local}->{nodes});
=cut
    foreach my $storeid (keys %$ids) {
        my $d = $ids->{$storeid};
        my $type = $d->{type};

        my $def = $defaultData->{plugindata}->{$type};

        if ($def->{content}) {
            $d->{content} = $def->{content}->[1] if !$d->{content};
        }
        if (grep { $_ eq $type }  @SHARED_STORAGE) {
            $d->{shared} = 1;
        }
    }

    return $cfg;
}

And then reboot the host.

Now you can run `pvesm remove local` via the console or through the Web UI to remove the `local` storage.
 
  • Like
Reactions: abaker and hyperbola
kenkin360 said:
For PVE7:

Either via the direct console or SSH

Code: 
nano /usr/share/perl5/PVE/Storage/Plugin.pm

Press [ctrl+w] and type '/var/lib/vz' (quotes not included) then press [Enter] to find the nearby location we're going to comment out

Add '=comment-out' (quotes not included) at the beginning and above the line
Code: 
# make sure we have a reasonable 'local:' storage

Add '=cut' (quotes not included) at the beginning and above the line
Code: 
foreach my $storeid (keys %$ids) {

So that the sub `parse_config` may look like

Perl: 
sub parse_config {
    my ($class, $filename, $raw) = @_;

    my $cfg = $class->SUPER::parse_config($filename, $raw);
    my $ids = $cfg->{ids};
=comment-out
    # make sure we have a reasonable 'local:' storage
    # we want 'local' to be always the same 'type' (on all cluster nodes)
    if (!$ids->{local} || $ids->{local}->{type} ne 'dir' ||
        ($ids->{local}->{path} && $ids->{local}->{path} ne '/var/lib/vz')) {
        $ids->{local} = {
            type => 'dir',
            priority => 0, # force first entry
            path => '/var/lib/vz',
            'prune-backups' => 'keep-all=1',
            content => {
                backup => 1,
                images => 1,
                iso => 1,
                rootdir => 1,
                snippets => 1,
                vztmpl => 1,
            },
        };
    }

    # make sure we have a path
    $ids->{local}->{path} = '/var/lib/vz' if !$ids->{local}->{path};

    # remove node restrictions for local storage
    delete($ids->{local}->{nodes});
=cut
    foreach my $storeid (keys %$ids) {
        my $d = $ids->{$storeid};
        my $type = $d->{type};

        my $def = $defaultData->{plugindata}->{$type};

        if ($def->{content}) {
            $d->{content} = $def->{content}->[1] if !$d->{content};
        }
        if (grep { $_ eq $type }  @SHARED_STORAGE) {
            $d->{shared} = 1;
        }
    }

    return $cfg;
}

And then reboot the host.

Now you can run `pvesm remove local` via the console or through the Web UI to remove the `local` storage.
Click to expand...
This indeed works!
 
  • Like
Reactions: kenkin360
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back