ext3 external journal on ssd - nfs timeout

[JustaGuy]

I learned today that the journal of an ext3 partition can be placed on a separate 400MB LV that could reside on a SSD.

Has anyone here experienced success with this kind of config?
I'm interested in hearing people's thoughts for or against it.


Here's why.

I happened across this as a possible solution after finding today that one VM would occasionally fail to initialize all it's hardware properly. The one with 28 NICs.

This would only occur during a backup and when the nfs server was reporting that it was timing out.

The nfs server would only time out when one particular disk had an ext3 flush that would take ~5 seconds to complete. The timeouts would show as lasting ~15 seconds in syslog, every minute or so.

The 'storage' page on PVE's web interface would always return the emperl error if I attempted to load it during the 10 seconds of nfs timeout.

So far an interim fix was to add a line to /etc/pve/storage.cfg under the nfs mounts that says

options timeo=500

This value may be far too high, but down 14 seconds with a 'nice 7' process accessing the mount led me to believe 50 seconds makes sense for if +2 VMs are on that disk too.

Also I changed /etc/fstab for the exported mounts to

noatime,commit=1

Just these 2 changes seems to cut the lag in half, allows the big VM to initialize properly, and lets the 'storage' page load on the pve interface during the flushes.

There's still significant sluggishness that the one slow disk spreads around the place coming through the nfs, it's just lessened some.

Since each physical machine has some SSD and a journal only takes 400MB- unless I hear why not to I think I'm going to try this & see if it takes off the remainder of this lag.

In theory it makes sense, but I know there are things I don't even know I don't know until I do know that that's what I don't know, you know?


So I just figured I'd ask before I call it a day, see if anyone has anything to add.
Thanks ahead of time.

 

[zzigge]

Well, I run a few servers in that configuration, single x25-m as boot disk and with the journal for the vz filesystem that resided on a dm raid5 on sata disks. I do it this way knowing full well a few caveats 1. no raid for the boot volume, or log of the vz filesystem - I found a forum post somewhere from a ext3 developer reasuring me somewhat that loss of the journal should cause at most 5 minutes of lost data. 2. the intel ssd does not guarantee written data in case of power failure, disabling the writecache can rectify this but reduces the lifespan of the disk 3. software raid is not recommended or supported by the proxmox developers 4. the snapshot backup method does not seem to work with an external ext3 journal, I only just discovered this and have not yet figured out what to do about it. I'm not quite convinced of the usefulness of the pve integrated backups anyway, there is no restore, no history, and as far as I've gathered they recopy everything every time... think cp, not rsync That said, with this configuration I got 3 8gb quadcore servers with 2500 fsyncs/second and 3tb usable storage for the price of one DL380, meaning I have a cluster and a spare. Sofar, thanks to openvz live migrations I have 100% uptime.