CIFS remote mount disconnect every X hours

[Ancalagon]

Hi guys, I'm strugling in the last month with my PVE and remote NAS connection via CIFS. At least two times per day my PVE doesn't has the connection to the NAS with an exclamation mark icon near the storage name. The share name is 'Entertainment' and is on an Unraid NAS in my LAN= with fix IP and worked that way for the last three years.

CIFS status
root@pve:/mnt/pve# pvesm status
unable to activate storage '[B]Entertainment[/B]' - directory '/mnt/pve/Entertainment' does not exist or is unreachable
Name Type Status Total Used Available %
[B]Entertainment cifs inactive 0 0 0 0.00%[/B]
NvME zfspool active 942931968 663361912 279570056 70.35%
PBS pbs active 48822443852 27261873216 21560570636 55.84%
local dir active 98497780 30440864 63007368 30.91%
local-lvm lvmthin active 833396736 327941615 505455120 39.35%

The server is reachable of course:
root@pve:/mnt/pve# ping 192.168.2.100
PING 192.168.2.100 (192.168.2.100) 56(84) bytes of data.
64 bytes from 192.168.2.100: icmp_seq=1 ttl=64 time=0.012 ms
64 bytes from 192.168.2.100: icmp_seq=2 ttl=64 time=0.016 ms

Detail of my folder with mounts:
root@pve:/mnt/pve# ls -la
ls: cannot access 'Entertainment': Stale file handle
total 20
drwxr-xr-x 6 root root 4096 Mar 10 2023 .
drwxr-xr-x 5 root root 4096 Sep 14 2023 ..
drwxr-xr-x 2 root root 4096 Mar 10 2023 temp_download
[B]d????????? ? ? ? ? ? Entertainment[/B]
drwxr-xr-x 2 root root 4096 Jan 19 2023 unraid_bak

I'm using linux since 2005 and never seen something like this. If I reboot the PVE the problem fix itself for a few hours.
In LOGS the first message says "unable to activate storage 'Entertainment' - directory '/mnt/pve/Entertainment' does not exist or is unreachable" and form that moment the NAS is not reachable and tha folder "Entertainment" has that strange proprieties d??????.

To solve it without a server reboot I need to force umount and then it work for some hours. The problem seems to be from PVE side. Seems.
umount -f -l /mnt/pve/Entertainment

Any idea?

 

[Moayad]

Hi,

For more information what happened exactly, could you check the syslog when the mount is unmounted on both side? You can check the syslog with specific time using Proxmox VE Web UI or from CLI using journalctl e.g.:

journalctl --since "2024-07-01 08:00"  > /tmp/$(hostname)-Syslog.txt

 

[Ancalagon]

Logs are not helping, my mistake to not mentioned earlier. The only thing in logs that I get is

Jul 02 12:00:07 pve pvestatd[1549]: unable to activate storage 'Entertainment' - directory '/mnt/pve/Entertainment' does not exist or is unreachable

Nothing sospicious before. When that's happening NAS is reachable and the destination folder had all that strange permissions

 

[Moayad]

Hi,

Did you check the older syslog i.e., before the `Entertainment` unmounted? Hard to know how fix the issue without syslog.

 

[Ancalagon]

Yes, I'd loog into syslog and journalctl, nothing seems to indicate that there is a problem.
The issue with email is that is not configured, PVE has always LAN and WAN connection, I'd monitored with Unifi console.

Jul 04 13:54:29 pve kernel: eth0: entered promiscuous mode
Jul 04 13:54:31 pve kernel: eth0: left promiscuous mode
Jul 04 13:55:31 pve kernel: eth0: entered promiscuous mode
Jul 04 13:55:33 pve kernel: eth0: left promiscuous mode
Jul 04 13:56:33 pve kernel: eth0: entered promiscuous mode
Jul 04 13:56:35 pve kernel: eth0: left promiscuous mode
Jul 04 13:57:05 pve postfix/qmgr[1519]: 01C444009DF: from=<root@pve.localdomain>, size=13123, nrcpt=1 (queue active)
Jul 04 13:57:06 pve postfix/smtp[3068857]: connect to gmail-smtp-in.l.google.com[2a00:1450:4013:c06::1b]:25: Network is unreachable
Jul 04 13:57:35 pve kernel: eth0: entered promiscuous mode
Jul 04 13:57:36 pve postfix/smtp[3068857]: connect to gmail-smtp-in.l.google.com[64.233.184.27]:25: Connection timed out
Jul 04 13:57:36 pve postfix/smtp[3068857]: connect to alt1.gmail-smtp-in.l.google.com[2a00:1450:4013:c16::1a]:25: Network is unreachable
Jul 04 13:57:37 pve kernel: eth0: left promiscuous mode
Jul 04 13:58:06 pve postfix/smtp[3068857]: connect to alt1.gmail-smtp-in.l.google.com[142.250.153.26]:25: Connection timed out
Jul 04 13:58:06 pve postfix/smtp[3068857]: connect to alt2.gmail-smtp-in.l.google.com[2404:6800:4003:c00::1b]:25: Network is unreachable
Jul 04 13:58:06 pve postfix/smtp[3068857]: 01C444009DF: to=<mymail@gmail.com>, relay=none, delay=133039, delays=132979/0.01/60/0, dsn=4.4.1, status=deferred (connect to alt2.gmail-smtp-in.l.google.com[2404:6800:4003:c00::1b]:25: Network is unreachable)
Jul 04 13:58:37 pve kernel: eth0: entered promiscuous mode
Jul 04 13:58:39 pve kernel: eth0: left promiscuous mode
Jul 04 13:59:39 pve kernel: eth0: entered promiscuous mode
Jul 04 13:59:41 pve kernel: eth0: left promiscuous mode
Jul 04 14:00:41 pve kernel: eth0: entered promiscuous mode
Jul 04 14:00:43 pve kernel: eth0: left promiscuous mode
Jul 04 14:01:09 pve pvestatd[1549]: unable to activate storage 'Entertainment' - directory '/mnt/pve/Entertainment' does not exist or is unreachable
Jul 04 14:01:19 pve pvestatd[1549]: unable to activate storage 'Entertainment' - directory '/mnt/pve/Entertainment' does not exist or is unreachable
Jul 04 14:01:29 pve pvestatd[1549]: unable to activate storage 'Entertainment' - directory '/mnt/pve/Entertainment' does not exist or is unreachable
Jul 04 14:01:40 pve pvestatd[1549]: unable to activate storage 'Entertainment' - directory '/mnt/pve/Entertainment' does not exist or is unreachable
Jul 04 14:01:43 pve kernel: eth0: entered promiscuous mode
Jul 04 14:01:45 pve kernel: eth0: left promiscuous mode

I think something is changeing the directory. As I said when the problem occur, the mont directory in PVE is unusable, I can't event go inside of it because is messed up. If I do "ls -l" I get strange information about permission of the folder

d????????? ? ? ? ? ? Entertainment