Xtreemfs distribuited file system, interesting for Proxmox roadmap?

[mmenaz]

Hi, I've been told good things about Xtreemfs
http://www.xtreemfs.org/
and a demo video here:
http://www.youtube.com/watch?v=6WP0V5ABMUA
I know nothing about distributed file system, but like to know Proxmox team opinion on this FS and if they can consider worth implementing besides / in addition to sheepdog or cheph (see roadmap)
N.B. I only know that xtreemfs is used by a proprietary cloud solution (I disapprove this way of create business on Free software so much... they use free tools and proprietary chains, as far as I understand)
Thanks in advance

 

[dietmar]

This is a filesystem, so you can already use it with PVE.

 

[mmenaz]

Forgive my ignorance, and I write not to argue but to understand. What is the difference then with Sheepdog or Ceph, that are also distributed file system but are in the Proxmox roadmap like you have to implement / integrate them with Proxmox?
And also, is Xtreemfs any better than those two, or worse in usual Proxmox deploy scenario (local lan, 1Gbit switched)?

 

[dietmar]

What is the difference then with Sheepdog or Ceph, that are also distributed file system

Both sheepdog and RADOS are distributed block storage implementations (not file systems).

 

[SamTzu]

As far as future filesystems go btrfs seems more promising (than ext3 now in use) because it does not "format" the storage devices in the traditional way (practically no time used to start using the device) and because it can handle duplicate files without the need for additional space. Imagine that you have a 100 Virtual Machines on the same host all created from the same template. Only 1 of those servers would actually take the storage space
btrfs is superior FS when it comes to Snapshots.
You can actually convert your ext3 now to btrfs allready and test drive it with your Proxmox.
Here you can find easy to understand instructions on how to convert ext3 -> btrfs.
http://www.howtoforge.com/how-to-convert-an-ext3-ext4-root-file-system-to-btrfs-on-ubuntu-12.10

 

[mmenaz]

As far as I know, for VM stored in host files, btrfs has *terrible* performances due to it's "copy on write". Did you really tried it with Proxmox? What FSYNC performances did you get? And was good the I/O from the inside of the VM?
A part from virtualization, my only test on a PC has been done with a netbook... I had to reformat to ext4 because of the slugginess and continuous swap, now works far better.
In any case, btrfs lacks the Xtreemfs performances and features that are interesting for virtualization

 

[dietmar]

In any case, btrfs lacks the Xtreemfs performances

Really - you did some benchmarks? Usually, file-systems using dedup (checksums) are extremely slow.

 

[fknorn]

I'm also looking into using XtreemFS with Proxmox.

What I'm unsure about, is it possible to use it as shared storage in a HA cluster for holding (and supporting live migration of) containers? As far as I understand, this is currently only possible with either GlusterFS or NFS.

Thanks!

 

[SamTzu]

As far as I know, for VM stored in host files, btrfs has *terrible* performances due to it's "copy on write". Did you really tried it with Proxmox? What FSYNC performances did you get? And was good the I/O from the inside of the VM?
A part from virtualization, my only test on a PC has been done with a netbook... I had to reformat to ext4 because of the slugginess and continuous swap, now works far better.
In any case, btrfs lacks the Xtreemfs performances and features that are interesting for virtualization

Nah. I'm not that in to it
But I would like Proxmox to support btrfs so we could easily/quickly add additional (backup) storage devices to existing Proxmox setups. What ext lacks is compression. This is a BIG thing when you have a lot of data to backup.