[Solved] Directory drive migration

[whytf]

Hello,

I am planning on reinstalling VE into raid1, however only thing left that is not backed up is a 4TB HDD, that is formatted as "directory disk", with data on it.

The question is, however, how do I add/mount it into the new installation, without reformatting? I have backed up storage.conf part with that specific drive and made note of the /dev/disk/by-uuid/****, but I don't think that it is enough for the new installation.

Thanks for help.

 

[bbgeek17]

Is this disk separate from your root disk where you plan to reinstall PVE?

If it is a completely separate disk, then as long as you dont accidentally erase data on it - the data will be left intact by the re-installation.
The UUID should not change - its based on information in disk firmware, so, barring any unforeseen user induced problems, the "storage.cfg" entry should be sufficient to regain access to data on that disk.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[whytf]

Is this disk separate from your root disk where you plan to reinstall PVE?

If it is a completely separate disk, then as long as you dont accidentally erase data on it - the data will be left intact by the re-installation.
The UUID should not change - its based on information in disk firmware, so, barring any unforeseen user induced problems, the "storage.cfg" entry should be sufficient to regain access to data on that disk.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Yes the drive is a separate drive from the boot drive, I am not sure about the storage.conf, because it assumes the path to /mnt/pve/xxxx, where the disk is mounted to right now, unless proxmox mounts these disks automatically? Even for that, it would need to know the right name for that drive.

 

[bbgeek17]

yes, /mnt/pve is a service directory maintained by PVE. You should not be mounting thing there manually. Share your storage.cfg here so forum has a full picture.
You can also add /etc/fstab and out of of "df", and "mount"

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[whytf]

yes, /mnt/pve is a service directory maintained by PVE. You should not be mounting thing there manually. Share your storage.cfg here so forum has a full picture.
You can also add /etc/fstab and out of of "df", and "mount"

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

yep, no problem

storage.conf:

dir: FourTB
        path /mnt/pve/FourTB
        content vztmpl,images,iso,snippets,rootdir,backup
        is_mountpoint 1
        nodes pve1

fstab:

# <file system> <mount point> <type> <options> <dump> <pass>
/dev/pve/root / ext4 errors=remount-ro 0 1
/dev/pve/swap none swap sw 0 0
proc /proc proc defaults 0 0

df:

Filesystem               1K-blocks       Used  Available Use% Mounted on
udev                      32855256          0   32855256   0% /dev
tmpfs                      6578744       1360    6577384   1% /run
/dev/mapper/pve-root     221651032    7261364  204768332   4% /
tmpfs                     32893704      67488   32826216   1% /dev/shm
tmpfs                         5120          0       5120   0% /run/lock
/dev/sdd1               3844550452 1308952108 2340231084  36% /mnt/pve/FourTB
/dev/fuse                   131072         28     131044   1% /etc/pve
tmpfs                      6578740          0    6578740   0% /run/user/0

mount:

sysfs                    on  /sys                                                 type  sysfs        (rw,nosuid,nodev,noexec,relatime)
proc                     on  /proc                                                type  proc         (rw,relatime)
udev                     on  /dev                                                 type  devtmpfs     (rw,nosuid,relatime,size=32855256k,nr_inodes=8213814,mode=755,inode64)
devpts                   on  /dev/pts                                             type  devpts       (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs                    on  /run                                                 type  tmpfs        (rw,nosuid,nodev,noexec,relatime,size=6578744k,mode=755,inode64)
/dev/mapper/pve-root     on  /                                                    type  ext4         (rw,relatime,errors=remount-ro)
securityfs               on  /sys/kernel/security                                 type  securityfs   (rw,nosuid,nodev,noexec,relatime)
tmpfs                    on  /dev/shm                                             type  tmpfs        (rw,nosuid,nodev,inode64)
tmpfs                    on  /run/lock                                            type  tmpfs        (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2                  on  /sys/fs/cgroup                                       type  cgroup2      (rw,nosuid,nodev,noexec,relatime)
pstore                   on  /sys/fs/pstore                                       type  pstore       (rw,nosuid,nodev,noexec,relatime)
bpf                      on  /sys/fs/bpf                                          type  bpf          (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1                on  /proc/sys/fs/binfmt_misc                             type  autofs       (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=33932)
mqueue                   on  /dev/mqueue                                          type  mqueue       (rw,nosuid,nodev,noexec,relatime)
hugetlbfs                on  /dev/hugepages                                       type  hugetlbfs    (rw,relatime,pagesize=2M)
debugfs                  on  /sys/kernel/debug                                    type  debugfs      (rw,nosuid,nodev,noexec,relatime)
tracefs                  on  /sys/kernel/tracing                                  type  tracefs      (rw,nosuid,nodev,noexec,relatime)
fusectl                  on  /sys/fs/fuse/connections                             type  fusectl      (rw,nosuid,nodev,noexec,relatime)
configfs                 on  /sys/kernel/config                                   type  configfs     (rw,nosuid,nodev,noexec,relatime)
ramfs                    on  /run/credentials/systemd-sysusers.service            type  ramfs        (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs                    on  /run/credentials/systemd-tmpfiles-setup-dev.service  type  ramfs        (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs                    on  /run/credentials/systemd-sysctl.service              type  ramfs        (ro,nosuid,nodev,noexec,relatime,mode=700)
/dev/sdd1                on  /mnt/pve/FourTB                                      type  ext4         (rw,relatime)
ramfs                    on  /run/credentials/systemd-tmpfiles-setup.service      type  ramfs        (ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc              on  /proc/sys/fs/binfmt_misc                             type  binfmt_misc  (rw,nosuid,nodev,noexec,relatime)
sunrpc                   on  /run/rpc_pipefs                                      type  rpc_pipefs   (rw,relatime)
lxcfs                    on  /var/lib/lxcfs                                       type  fuse.lxcfs   (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/fuse                on  /etc/pve                                             type  fuse         (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)
tmpfs                    on  /run/user/0                                          type  tmpfs        (rw,nosuid,nodev,relatime,size=6578740k,nr_inodes=1644685,mode=700,inode64)

 

[bbgeek17]

is there anything else in storage.cfg related to this disk? Can you post entire config file?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[whytf]

There isn't anything else in storage.conf related to that disk, the other configuration in it is related to other LVM SSDs

 

[bbgeek17]

There isn't anything else in storage.conf related to that disk, the other configuration in it is related to other LVM SSDs

Well the configuration stanza says that that directory should be expected to be a mountpoint, ie dont use it until something is mounted. Your fstab has no reference to the disk/path. But the disk has to be mounted somehow.

Did you create a systemd unit for this disk? Examine your "systemctl" output. Somewhere in your system this disk is mounted, but its not done by PVE based on what you said/presented.
You can find some useful points here, for example: https://medium.com/@benmorel/creating-a-linux-service-with-systemd-611b5c8b91d6

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[whytf]

I did not create a systemd unit, that drive was formatted/created with proxmox gui, as a directory drive

 

[bbgeek17]

This is a task log of creation of a "directory" type storage on a brand new disk:

Spoiler: task log of a create

Task viewer: Directory Storage test - Create



# /sbin/sgdisk -n1 -t1:8300 /dev/sdc
The operation has completed successfully.
# /sbin/mkfs -t ext4 /dev/sdc1
mke2fs 1.46.5 (30-Dec-2021)
Creating filesystem with 4194043 4k blocks and 1048576 inodes
Filesystem UUID: 36979a7c-d674-4b81-98d7-b1329b723eba
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000

Allocating group tables: 0/128 done
Writing inode tables: 0/128 done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: 0/128 done

# /sbin/blkid /dev/sdc1 -o export
Created symlink /etc/systemd/system/multi-user.target.wants/mnt-pve-test.mount -> /etc/systemd/system/mnt-pve-test.mount.
TASK OK

As you can see a disk is formatted with your preferred filesystem, then a systemd unit is created for you. A new storage configuration entry is created as well. This procedure is a convenience for GUI users, it hides the steps you'd have to do manually from CLI.

You can :
a) examine and record placement and context of corresponding systemd units. Manually recreate them after reinstall.
b) Simply mount the disk via /etc/fstab

You will not be able to create "directory" storage from GUI as the disk is no longer raw, it has partition and filesystem. You'd have to really put an effort to erase that data.

good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[whytf]

Thank you, this was the missing last part to be able to reinstall.