[SOLVED] Web access fails

[petrs]

Dear all,
I would like to kindly ask you for a help. I am quite new in Proxmox technology, I am comming from VMware...
I have one physical server with PVE installed (updated/upgraded). I am still playing with PVE and VMs to get familiar with the PVE.
However, I have one annoying issue - from time to time, I simply cannot acces the web interface even the NMAP scan indicates that port 8006 is opened.
I can SSH in trying to restart the PVE node, however, without any luck:
reboot gives me: failed to open initctl fifo no such device or address
I also tried other ways how to restart daemons as indicated by several forum posts however, with following error message "bash: /usr/sbin/service: input/output error".
Thank you for your help,
Petr

 

[syfy323]

Your storage went away during runtime. Are you sure the device is in good condition?

 

[petrs]

Thanks for reply.
Well, I am not sure what to say. There are two new SSDs in HW raid 1, it is DELL HW, quite old, correct, that is reality with small customers.
I have no indication that there is HW failure.
What I can say from my experience is that PVE box works perfectly when I am connected and I am trying creating, deleting, backuping etc. tasks and the box is under heavy workload. When I do not touch the box for a few days, it results in the failure I just described. That is why it makes me quite confused...

 

[syfy323]

You can try "dmesg" on your console (might work) when this happens.

 

[petrs]

Still the same: bash: /usr/bin/dmesg: input/output error

 

[syfy323]

Ok, you definitely lost access to your storage backend which holds the OS.
Did you install directly to disk or did you install proxmox to a flash drive which the server boots?

 

[petrs]

No flash drive, directly to the disks, but managed by the DELL HW raid controller (raid 1). Therefore, it could be raid controller issue, however, I just replaced HDDs, deleted ESXi and installed PVE - isn't is compatibility issue? The simplest thing what I can do is to connect SSDs to the motherboard and let PVE to manage the raid.

 

[syfy323]

That's also a safe solution. ESXi does not support mdraid or lvm, there you are forced to use a HW RAID controller. On most other solutions, software based solutions are available and most of them are equally or even better from performance perspective.

You can also check if an update for firmware is available for your controller, just in case you hit a bug because debian uses a more recent driver.

 

[petrs]

First thing before installing the PVE was firmware upgrade of all server components to the last version but of course, there could be a bug.
So, your final recommendation is to go away from the HW raid controller, right?

 

[syfy323]

First thing before installing the PVE was firmware upgrade of all server components to the last version but of course, there could be a bug.
So, your final recommendation is to go away from the HW raid controller, right?

Depends. I don't generally recommend a change in hardware, just because there are problems with unknown cause.
On the other hand, it might take weeks / months to understand whats happening there, which might resolve on it's own when switching hardware.

IMHO
I had lot's of broken HW RAIDs, card tells me everything is in order, clean state - but data is definitely corrupted.
For some years now, I heavily utilize Ceph for larger environments and ZFS or LVM on small ones.
Till now, all these installs worked fine, no data corruption. I will never touch a HW RAID again, if I can.

 

[petrs]

Well, as we are not able to identify the issue now and the HW modification is the only way we can see now, I will definitely modify the HW and continue playing with the PVE. It is impressive piece of technology and I am starting to love it!
I believe we can mark this thread as solved.
Thank you for taking the time and helping me!
Regards,
Petr

 

[syfy323]

Sure, no problem

 

[petrs]

I am back after some time just to confirm - removing HW RAID fixed the problem. Now I am on ZFS RAID managed by Proxmox and it just works without any issue!