Proxmox severs keep going into Read-only mode and breaking Inodes

[laptopdude90]

I'm running Proxmox on a Dell PE 2950 server with 32GB of RAM. In it are 2 brand new Seagate Barracudas in RAID 1 (Less than a week old). The VM is running Ubuntu 10.04. The virtual HDD is RAW with writethrough. It is 1000GB. For some reason, every few days the VM goes into read-only mode. Rebooting the VM fixes it, and it repairs some Inodes. I'm worried that one day a reboot won't fix it. I don't have this problem with any of my smaller servers.

Help!

 

[mir]

Is this software RAID?
You could try option 'Direct Sync'.

 

[laptopdude90]

Is this software RAID?
You could try option 'Direct Sync'.

No, it's hardware RAID. Perc 5i.

 

[mir]

No, it's hardware RAID. Perc 5i.

This sounds like malfunctioning hardware then. Either the controller or the disks.

 

[laptopdude90]

This sounds like malfunctioning hardware then. Either the controller or the disks.

I have other servers that read/write much more frequently than that one, so I don't think that's the problem. I think it has something to do with the size.

 

[mir]

I have other servers that read/write much more frequently than that one, so I don't think that's the problem. I think it has something to do with the size.

What has that to do with this case? Hardware can get defected at any time. It does not correlate to the numbers of read and write.

 

[laptopdude90]

What has that to do with this case? Hardware can get defected at any time. It does not correlate to the numbers of read and write.

All the servers are running on the same host.

 

[mir]

Ok. I misunderstood.

Are all VM's using the same cache setting?
What kind of file system is used inside the VM's that fails and the ones that don't fail?

 

[laptopdude90]

It's happened again! That's the second time today!

Edit: I'm keeping it offline until I figure out what to do.

Edit 2: A possible solution might be to have a 40GB OS drive and a 1TB storage drive.

 

[mir]

And what OS is used in the failing VM's?

 

[laptopdude90]

And what OS is used in the failing VM's?

Ubuntu 10.04 for all of them.

 

[laptopdude90]

Yes, they're all using the same cache setting.

All the default in Ubuntu 10.04. EXT3, I think.

 

[RobFantini]

are backups or heavy network traffic occurring at the time the issue occurs?

 

[laptopdude90]

are backups or heavy network traffic occurring at the time the issue occurs?

There never really is heavy traffic.

 

[spirit]

Hi,
I using some 2950 in our datacenter, and I never had any problem with them.

As they are pretty old, are you sure that the perc cache battery is not dead ?

 

[laptopdude90]

Hi,
I using some 2950 in our datacenter, and I never had any problem with them.

As they are pretty old, are you sure that the perc cache battery is not dead ?

No error on the screen, plus wouldn't that make all the VMs break?

 

[spirit]

No error on the screen, plus wouldn't that make all the VMs break?

I have already sometime this kind of problem, with a failing battery, perc controller going from writeback->writethrough,writethrough->writeback because of a failing battery.

Just to be sure that it's not a perc cache problem, can you force the perc in writethrough ?

 

[udo]

Ubuntu 10.04 for all of them.

Hi,
you know, that ubuntu has an special kernel to run inside VMs?

Without this kernel I got sometime kernelpanics on a normaly loaded VM - after switching the guest kernel all works ok.

Udo

 

[laptopdude90]

I have already sometime this kind of problem, with a failing battery, perc controller going from writeback->writethrough,writethrough->writeback because of a failing battery.

Just to be sure that it's not a perc cache problem, can you force the perc in writethrough ?

Unfortunately I can't, I have some important VMs on that server. I do have an identical server, Dell PE 2950, with a dead battery, but that one works fine.

 

[abubin]

wow..didn't know ubuntu 10.04 have this kind of problem. I am interested to know how you switch to using this guest kernel?