Hi all,
I had a FreeNAS share mounted in my Proxmox environment as NFS storage. Due to some hardware issues we had to take the FreeNas appliance offline.
Since we've restarted the first Proxmox VE node the entire cluster started acting weird:
- when trying to start a VM (on any node) if fails with:
- can't temporarily remove the NFS storage as it will error out with Connection Timed Out (in the web console) and pvesm remove NAS_SHARE will just freeze.
- the Proxmox specific services are showing as running but pvestatd status is advising that STORAGE 'NAS_SHARE' IS NOT ONLINE
Any ideas if the unavailable storage is causing this?
- if YES how can I remove it temporarily so that I can resume cluster normal operation?
- If NO then what exactly is causing this behavior and how can I fix it ?
Here is the info on my Proxmox VE environment:
Thanks in advance,
B.
I had a FreeNAS share mounted in my Proxmox environment as NFS storage. Due to some hardware issues we had to take the FreeNas appliance offline.
Since we've restarted the first Proxmox VE node the entire cluster started acting weird:
- when trying to start a VM (on any node) if fails with:
Code:
TASK ERROR: start failed: command '/usr/bin/kvm -id 202 -name VM-001 -chardev 'socket,id=qmp,path=/var/run/qemu-server/202.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qemu-server/202-event.qmp,server,nowait' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/202.pid -daemonize -smbios 'type=1,uuid=d6f39b00-43a3-4cd5-9f6f-33e10f09124e' -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga std -vnc unix:/var/run/qemu-server/202.vnc,x509,password -no-hpet -cpu 'kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,enforce' -m 16384 -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:411bb56e5518' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' -drive 'file=/mnt/SSD_NODE_22/images/202/vm-202-disk-1.qcow2,if=none,id=drive-sata0,format=qcow2,cache=none,aio=native,detect-zeroes=on' -device 'ide-drive,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap202i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=EE:E9:CA:97:07:57,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard'' failed: got timeout- can't temporarily remove the NFS storage as it will error out with Connection Timed Out (in the web console) and pvesm remove NAS_SHARE will just freeze.
- the Proxmox specific services are showing as running but pvestatd status is advising that STORAGE 'NAS_SHARE' IS NOT ONLINE
Any ideas if the unavailable storage is causing this?
- if YES how can I remove it temporarily so that I can resume cluster normal operation?
- If NO then what exactly is causing this behavior and how can I fix it ?
Here is the info on my Proxmox VE environment:
Code:
root@is-node-22:~# pveversion -v
proxmox-ve: 5.2-2 (running kernel: 4.15.18-8-pve)
pve-manager: 5.2-10 (running version: 5.2-10/6f892b40)
pve-kernel-4.15: 5.2-11
pve-kernel-4.15.18-8-pve: 4.15.18-28
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: not correctly installed
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-41
libpve-guest-common-perl: 2.0-18
libpve-http-server-perl: 2.0-11
libpve-storage-perl: 5.0-30
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.2+pve1-3
lxcfs: 3.0.2-2
novnc-pve: 1.0.0-2
proxmox-widget-toolkit: 1.0-20
pve-cluster: 5.0-30
pve-container: 2.0-29
pve-docs: 5.2-9
pve-firewall: 3.0-14
pve-firmware: 2.0-6
pve-ha-manager: 2.0-5
pve-i18n: 1.0-6
pve-libspice-server1: 0.14.1-1
pve-qemu-kvm: 2.12.1-1
pve-xtermjs: 1.0-5
qemu-server: 5.0-38
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3Thanks in advance,
B.
