GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interrupt

[tytanick]

Hi guys.
I have two nodes and made DRBD 2 primary cluster.
But i am really disapointed by that i need to have all VMs stored as RAW and i cant put on DRBD storage any files for example ISO files.

Thats why i want to try GlusteFS.
Do you guys think its good idea ?
2 nodes with glusteFS and i also want to have live migration option in case of server 1 will fail.
Also i want to store VMs as qemu2 to eat less more GB and store some iso.
Tell me will that work, wont be any problems ?

Any other alternative that is tested for production use ?
P.S - I CANT HAVE EXTERNAL ANY storage, and all data should be replicated.

One more question.
Lets say that i have glustefs,
Node1 running VM 101
Node2 running VM 102

And the connection between 2 nodes will fail, some changes will be done to 101 (on node1) and 102 (on node2)
and the after lets say 2 hours, connection will be restored.
Will GlusterFS figure out that files were modified so it will se that node1 was using only file of VM 101, and node 2 files of VM 102
So VM101 (file on node1) will rewrite all VM101 (file on node2)
and vice versa, VM102 file running on node2 will rewrite all 102 data on node1 ?
And after recconect admin wont have do anything to restore data ?

Because in my opinion it should be a problem. Problem would be only if node1 and node2 were accesing the same VM101 file, but it wont gonna happen

Thx guys for the support !

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

Accually after a lot of searching i found interesting article:
GlusterFS on two server nodes, each node has mounted its resurce and running VMs on it.
It has 100% redudancy
it allows live migration
also backups can be done on this cluster and other iso stored
also 100% redudancy.


Tell me any cons about this setup why not to use it on two node cluster (100% replication and in case of one node failure, manual restart of VMs on second node)
Or is this perfect setup for two node cluster with no external NAS ?

Here is articke: http://www.sdmachelp.com/wiki/doku.php/linux/gluster-kvm-hosts

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

Ok i did this.
Only 2 nodes.
Each had glusterfs-server running and sharing identical ext4 space.
Mounted on each node as localhost/name_of_storage
http://scr.hu/2o2c/z8isp

I will test this for live migration in few minutes and post here results.
I guess that noone tried this so far ?

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

after force reboot of node1 i have in my VM journal commit I/O error
eh. and i thought it will be sooo great.

so any other posibility to use qcow2 in replication mode - something else that drbd and gluster ?
in drbd i have to use raw images so many GB are wasted.

 

[mir]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

I have tried it and it works splendid. You cannot migrate CT's due to a kernel bug.

One note: On GlusterFS the recommended file system is XFS.

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

you think thats the case ?
because i "reboot -f" node1 that was running gluster and when i started on node2 my VM while node1 was rebooting.
VM just crashed, ext4 is fucked up
you think that is XFS case ?

 

[mir]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

you think thats the case ?
because i "reboot -f" node1 that was running gluster and when i started on node2 my VM while node1 was rebooting.
VM just crashed, ext4 is fucked up
you think that is XFS case ?

Gluster community recommends XFS because of performance. Regarding reboot -f: If you do this don't expect any file system to survive since reboot -f is an unclean shutdown which prevents disk cache to flush uncommitted data to disk.

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

does reboot -f works like power outtage ?
i want to simulate power failure

 

[mir]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

does reboot -f works like power outtage ?
i want to simulate power failure

Yes. That's why UPS and/or RAID with BBU is recommended.

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

but what about hardware failre, suddenly whole server fires and second will get corupted data ?
Am i correct or drbd worked much better or i am mistaken ?

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

XFS worked ok
qcow2 -
node1 running VM, then reboot -f
while rebooting i restarted VM on node2 and everything seems ok

but, 1 sec before i reboot -f node1, i made "touch aaaaaaa.txt" on VM running on node1
and when i restarted this VM on node2 there wasnt be any file. so it is not fast as i thought, or there are some other caches

 

[tytanick]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

ok so anone want to share your feelings about glusterfs and (manual HA in case of hardware failure ?)
Or glusterfs vs drbd ?

I like gluster because i can store normal files on top of it and compressed qemu.

 

[acidrop]

Re: GlusteFS + qcow2 + Live Migration AND GlusterFS question about connection interru

ok so anone want to share your feelings about glusterfs and (manual HA in case of hardware failure ?)
Or glusterfs vs drbd ?

I like gluster because i can store normal files on top of it and compressed qemu.

Each one has pros and cons. I would prefer DRBD because it's faster than Gluster and more mature (long time tested). Cons are that it does not scale so well.
It would be better if you follow Glusterfs or DRBD community forums for more questions about these strorage systems.

regards