creating central storage with 3 nodes - without extra SAN

[bill_mustermann]

Hi,

i have 3 test proxmox 4 server installed.

on the first one i have created a test VM. Now i want to create a central storage but i dont wanna use an extra SAN oder something like that. all i have are these 3 servers.

what can i do?

how do i move/migrate the test vm from the local storage to the new central storage?

regards,
bill

 

[Q-wulf]

check:
https://pve.proxmox.com/wiki/Storage_Model

i'd do GlusterFS or Ceph (without knowing how many disks [HDD/SSD] you have in each node)


Test-Vm to differet storage:
- least painless is to do a backup via Vzdump, then a restore to the network storage

 

[bill_mustermann]

Hi,

is there a documentation how to setup GlusterFS on my 3 Nodes?

regards,
bill

 

[vcp_ai]

Hi , GlusterFS is the simplest way (not the better performance one) to do what you need. I have a similar config.

Just install glusterfs-server (direct from proxmox repo apt-get install glusterfs-server ) on at least two of your servers.
Then follow instructions elsewhere (google) to set up at least one 'brick' replicated on these two servers.
This has to be done manually on each server.

Then you can use Proxmox GUI to use them on every node of your cluster.

Highly recomended is a dedicated network for this pourpose.

Regards

 

[vcp_ai]

how do i move/migrate the test vm from the local storage to the new central storage?

Once you have your new 'central' storage running, you can just move your Vm's disk to this new storage.

VM> Hardware > Select your Disk > Move Disk


You can do it even online
​

 

[Q-wulf]

[...]

VM> Hardware > Select your Disk > Move Disk
[...]​

Oh, I forgot about that, quite embarrassing.

 

[bill_mustermann]

Hi,

GlusterFS is working now but there seems nothing to happen:
create full clone of drive ide0 (local:100/vm-100-disk-1.raw)
Formatting 'gluster://127.0.0.1/default/images/100/vm-100-disk-1.raw', fmt=raw size=59055800320
transferred: 0 bytes remaining: 59055800320 bytes total: 59055800320 bytes progression: 0.00


regards,
bill

 

[vcp_ai]

Search the forum for Online Migration. There are several causes.
One of them that I've had is using disk type = SATA (qemu problem)
http://forum.proxmox.com/threads/13931-migrate-failed-State-blocked-by-non-migratable-device

 

[bill_mustermann]

Hi,

hmmm semms like there is happening someting:

create full clone of drive ide0 (local:100/vm-100-disk-1.raw)
Formatting 'gluster://127.0.0.1/default/images/100/vm-100-disk-1.raw', fmt=raw size=59055800320
transferred: 0 bytes remaining: 59055800320 bytes total: 59055800320 bytes progression: 0.00 %
transferred: 590558003 bytes remaining: 58465242317 bytes total: 59055800320 bytes progression: 1.00 %

regards,
bill

 

[robhost]

What about DRBD9?

 

[Erk]

What about DRBD9?

I think it's considered to still be in development. Though it does support a 3 node mirror now without stacked layers which could be good. I would be interested in hearing any reports of it's performance on Proxmox.

 

[robhost]

Works fine here with PVE 4. Speed depends on you network layout and number of replicas. With dedicated 1000M we have ~70MB/s write speed, which is quite good.
Reading is almost as fast as from local storage, because reads are local.

DRBB 9 is stable, see https://www.linbit.com/en/p/products/drbd9

 

[bill_mustermann]

Hi,

are there performance differences between drbd9 and glusterfs?

is there already a documentation how to use drbd with 3 nodes on proxmox?

regards,
bill

 

[robhost]

Here: http://pve.proxmox.com/wiki/DRBD9