Issue with question marks on all storages (and Looking for some reassurances)

[paxmobile]

Hello to all experts out there, I'm looking for someone who tells me that having a major issue almost every week is normal.
I think that my system has enough power (Pentium quadcore CPU and 24GB for 14 LXC's, 1 Win11 and 1 Ubuntu VM) my hardware should not too poor for the testing purpose I'm using it, the system has mostly only me as user and I never experienced lags or performance issues.

Anyway now the problem is that when I right click any of the Storage I get "Status: unknown" and I notice the question marks on the storages. I have to confess that I did not jet notice any malfunctioning but I prefer to be safe than sorry.

 

[vshab]

Maybe your storage is starting to fail? Can you check Node > Disks? What is your storage configuration?

 

[gfngfn256]

Is there a reason your PVE instance is not updated?
Searching further, I notice from an older post of yours that you seem to have downgraded your version, (maybe different node?). Reason?

 

[paxmobile]

Nothing seems wrong here except the question marks on the left column. I suggest The 1TB Sata SSD I monuted at the beginning of 2024 shouldn't be the issue. Does the attached pic help ?
Altrough I blamed about having so many errors, I have to admit that I always found a solution, understood the cause, fixed the issue and learned something new; 75% of the credits go to the savvies of this forum, thanks.

 

[gfngfn256]

Not sure why you chose not to answer my above post.

In any case, could you show the image of {node} , Disks.

Also provide output for cat /etc/pve/storage.cfg

Has your node ever been part of a cluster?

When is the last time you updated your setup? When is the last time you rebooted?

 

[paxmobile]

Ops.. I'm sleepy today...


root@paxmeshcentral:~# cat /etc/pve/storage.cfg
nfs: PROXMOX
export /volume1/PROXMOX
path /mnt/pve/PROXMOX
server 10.10.10.20
content iso,backup,images,snippets,vztmpl,rootdir
nodes paxmeshcentral
preallocation off
prune-backups keep-all=1

dir: USBHDD500
path /mnt/pve/USBHDD500
content iso,images,backup,rootdir,snippets,vztmpl
is_mountpoint 1
nodes paxmeshcentral
shared 0

dir: local
path /var/lib/vz
content vztmpl,snippets,rootdir,backup,images,iso
prune-backups keep-all=1
shared 0

dir: USBCHD500
path /mnt/pve/USBCHD500
content iso,rootdir,vztmpl,snippets,images,backup
is_mountpoint 1
nodes paxmeshcentral
shared 0

dir: USBTERABCK
path /mnt/pve/USBTERABCK
content iso,vztmpl,snippets,rootdir,backup,images
preallocation off
prune-backups keep-all=1
shared 0

Joining my two nodes to a Cluster is on my todo list. Already have a nodename on this machine.

 

[gfngfn256]

Ops.. I'm sleepy today...

You are still asleep - as you still have not answered all my questions. (See above).

2 irregularities I note:

1. You have no lvmthin: local-lvm in you installation. This is normally the default. Is this by choice?
2. Of your 3 USB disks (configured as dir storage), it appears that /dev/sdd is neither mounted nor in use.

 

[paxmobile]

You're perfectly right, thinpool was missing since approx one month, but I thought I can do without it as I don't use the snapshot function (correction to this way of thinking well accepted).
sdd is a new hdd I connected yesterday and was waiting to configure.
Now I'm afraid of rebooting Proxmox, have a stange feeling...

 

[gfngfn256]

thinpool was missing since approx one month

And you have no idea why? That is rather perplexing. I assume your main OS disk is /dev/sda & your local: lvm-thin was on /dev/sda3 . You will need to investigate. Thinking you did not need it so why bother investigating, amounts to negligence. What happened before it went missing?

sdd is a new hdd

So one of your already PVE storage-configured USB disks, is not presently being detected by the node. Do you know why?

Now I'm afraid of rebooting Proxmox

When was the last time you rebooted?

Why is your PVE instance not fully updated? (See my question/s above).

Joining my two nodes to a Cluster is on my todo list

In your present state - don't even think about it. Note that for HA, you will need at least three nodes (one can be a qdevice).


In all cases, I very much hope you have fully restorable backups (external) of all your LXCs & VMs.

IF YOU CHOOSE TO RESPOND - DO SO TO ALL MY QUESTIONS.

 

[paxmobile]

And you have no idea why? That is rather perplexing. I assume your main OS disk is /dev/sda & your local: lvm-thin was on /dev/sda3 . You will need to investigate. Thinking you did not need it so why bother investigating, amounts to negligence. What happened before it went missing?

I'm testing Proxmox since 2023 on 2 nodes that are not clustered. For now I'm doing this only in my spare time, no real negligence as it is not a production grade setup. I'm procrastinating a reinstall of PM as I have all the VMs and LXCs backupped twice.

So one of your already PVE storage-configured USB disks, is not presently being detected by the node. Do you know why?

I'm sorry I got confused, the 1TB HDD sdc is strangely working but appears as not in use, not mounted, not GPT.

When was the last time you rebooted?

Why is your PVE instance not fully updated? (See my question/s above).

I just updated the host, than running
service pvestatd restart
did fix the issue > successfully rebooted the node

In your present state - don't even think about it. Note that for HA, you will need at least three nodes (one can be a qdevice).

In all cases, I very much hope you have fully restorable backups (external) of all your LXCs & VMs.

IF YOU CHOOSE TO RESPOND - DO SO TO ALL MY QUESTIONS.

I'm always inspired by your mind quickness answering this issues but will try to slower the answers after more studying, given I hate the day for having only 24h.

All storages are now normal (without question marks). USBTERABCK is indeed sdd but appears as not mounted, not GPT and under usage it's no. Even as a rookie I understand that's not normal, but don't know what to try.

Anyway I'm trying hard to not reinstall proxmox altrough, of course, I have multiple backup stages of all VM's and LXC's.

 

[gfngfn256]

the 1TB HDD sdc

USBTERABCK is indeed sdd

is it sdc or sdd?

How do / did you go about adding the USB drives to Proxmox?

What do the following output:

cat /etc/fstab

pveversion -v

If your local: lvm-thin still hasn't reappeared since your update, I think you need a COMPLETE reinstall. Do so AFTER completely wiping the OS disk.

 

[paxmobile]

is it sdc or sdd?
It's sdd the origin of strange cause it is working as USBTERABCK and nightly backups 14 LXCs and 2 VMs

How do / did you go about adding the USB drives to Proxmox?
Cannot remember

What do the following output:

cat /etc/fstab
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/pve/root / ext4 errors=remount-ro 0 1
UUID=515C-8A6F /boot/efi vfat defaults 0 1
/dev/pve/swap none swap sw 0 0
proc /proc proc defaults 0 0
[/QUOTE]

[QUOTE="gfngfn256, post: 715818, member: 182598"]
pveversion -v

proxmox-ve: 8.1.0 (running kernel: 6.5.11-8-pve)
pve-manager: 8.1.4 (running version: 8.1.4/ec5affc9e41f1d79)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.5: 6.5.11-8
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
ceph-fuse: 17.2.7-pve2
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.4-1
proxmox-backup-file-restore: 3.1.4-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.4
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-3
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.2.0
pve-qemu-kvm: 8.1.5-2
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1

If your local: lvm-thin still hasn't reappeared since your update, I think you need a COMPLETE reinstall. Do so AFTER completely wiping the OS disk.

No, since lvm-thin hasn't reappeared after the update, and if there is no command to create the thin from sda I will reinstall PM, this time destroiying the partition with (I guess) GParted, install the latest PM and import the VM's and LXC's ? What's the main reason for not living without lvm-thin ?

 

[gfngfn256]

So your fstab does not contain entries for any of your USB drives. With Proxmox they won't be magically mounted, unless they are told to either manually with a mount command or permanently by adding an entry in fstab.

Check output (& post) of the lsblk command to check what is actually mounted.

strange cause it is working as USBTERABCK and nightly backups 14 LXCs and 2 VMs

You will probably find those backups being stored locally in the mountpoint /mnt/pve/USBTERABCK (not on the USB disk). To test; disconnect the USB drive & see if the backups are still there at the mountpoint.

proxmox-ve: 8.1.0 (running kernel: 6.5.11-8-pve)

Clearly your Proxmox is NOT updated. How did you update?. You probably have not set the repositories correctly.

To fix your problems, search these valuable forums for the above. (See below).

since lvm-thin hasn't reappeared after the update

this time destroiying the partition

No, I said WIPE the whole OS disk. I suspect maybe something isn't quite right with the partitioning etc.

As I said above; make sure you have external & restorable backups of all your VMs & LXCs. (Remember my above conclusion that those backups may not be on those USB disks currently attached!).

As I've indicated you will probably need a complete reinstall - paying attention to detail, such as how to correctly add storages in Proxmox.