Help for Storage Model strategy

[morfair]

Now I have two servers (nodes) in cluster and have not network storage. Each node has its own local storage for its own CT and VM.
I want to use migration and backup to neighboring nodes. What strategy for data storage should I choose? Also I want to easily add new nodes later.

 

[udo]

Now I have two servers (nodes) in cluster and have not network storage. Each node has its own local storage for its own CT and VM.
I want to use migration and backup to neighboring nodes. What strategy for data storage should I choose? Also I want to easily add new nodes later.

Hi,
for two nodes I prefer DRBD. For "easy add new nodes" you need an networkstorage (iSCSI, FC, NFS, CEPH).
For backup is an external host (NFS) a good choice.

Udo

 

[morfair]

Hi,
for two nodes I prefer DRBD. For "easy add new nodes" you need an networkstorage (iSCSI, FC, NFS, CEPH).
For backup is an external host (NFS) a good choice.

Udo

I don't want to use DRDB because this is not perfect work in primary/primary mode. Also when I add more nodes DRDB can not work with them.
I wanted to create partitions at each node and share their via iSCSI, then use these target at their owners for CT and VM (after being added the targets to cluster). New drive /dev/sdX appear at all nodes (also at himself node). I created LVM LV over this iSCSI drive, but the logical volumes saw only one node... This method did not work =(

 

[nandex]

I think that you could create a Server NFS intermediate and use from it your storage.... It's an idea.

Good Luck!

 

[morfair]

I think that you could create a Server NFS intermediate and use from it your storage.... It's an idea.

Good Luck!

Do you offer to do so? For example three nodes with their local storage (hdd), NFS mount in /etc/fstab, in Proxmox's WEB-interface storages added as Direcotry:

Node #1:
/var/lib/vz01 - local fs
/var/lib/vz02 - nfs to node2:/var/lib/vz02
/var/lib/vz03 - nfs to node3:/var/lib/vz03

Node #2:
/var/lib/vz01 - nfs to node1:/var/lib/vz01
/var/lib/vz02 - local fs
/var/lib/vz03 - nfs to node3:/var/lib/vz03

Node #3:
/var/lib/vz01 - nfs to node1:/var/lib/vz01
/var/lib/vz02 - nfs to node2:/var/lib/vz02
/var/lib/vz03 - local fs

Do I understand correctly? Then what to do with KVM?..

 

[nandex]

Is your NFS a shared storage?.
You need to define a server with NFS, for example Node#1.
1. Directory NFS, for example /srv with subdirectories for images, isos and backups.
#mkdir /srv
#mkdir /srv/backup
#mkdir /srv/isos
#mkdir /srv/images

2. Install nfs #apt-get install nfs-kernel-server

3. Configure exports

/srv/backup 192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)
/srv/images 192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)
/srv/isos 192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)

4. Restart NFS service #/etc/init.d/nfs-kernel-server restart

Add nfs to proxmox.

a) Go to Datacenter → Storage. Add y NFS Share.
b) ID put the name.
c) Server name of NFS Server, In that case Node #1.
d) Open Exports y detects directories automatically.
e) Content -> select content.
f) Put shared.

Add.

 

[udo]

I don't want to use DRDB because this is not perfect work in primary/primary mode.

work well for me at two cluster (pve1.9 and pve2.2)

Also when I add more nodes DRDB can not work with them.

you can always use a couple of hosts, or start with drbd to have shared storage and if you expand, buy an "real" shared storage also.

I wanted to create partitions at each node and share their via iSCSI, then use these target at their owners for CT and VM (after being added the targets to cluster). New drive /dev/sdX appear at all nodes (also at himself node). I created LVM LV over this iSCSI drive, but the logical volumes saw only one node... This method did not work =(

Sorry, but this look not very durable!!
If one host fail - you lost the data from thios node! This is not shared storage this is dangerous and has nothing to do with cluster...

Udo

 

[morfair]

Is your NFS a shared storage?.
You need to define a server with NFS, for example Node#1.
1. Directory NFS, for example /srv with subdirectories for images, isos and backups.
#mkdir /srv
#mkdir /srv/backup
#mkdir /srv/isos
#mkdir /srv/images

2. Install nfs #apt-get install nfs-kernel-server

3. Configure exports

/srv/backup 192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)
/srv/images 192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)
/srv/isos 192.168.0.0/255.255.240.0(rw,sync,no_subtree_check,no_root_squash)

4. Restart NFS service #/etc/init.d/nfs-kernel-server restart

Add nfs to proxmox.

a) Go to Datacenter → Storage. Add y NFS Share.
b) ID put the name.
c) Server name of NFS Server, In that case Node #1.
d) Open Exports y detects directories automatically.
e) Content -> select content.
f) Put shared.

Add.

In this case CT of second and third nodes will be placed at hdd of first node. It's not right.
Note I don't want use NFS strongly.

 

[blurrrr]

If money isn't the point, go and grab some external storage-device, that would be the easiest way. There are some really nice ones out there ( http://www.violin-memory.com/products/6000-flash-memory-array/ or http://www.ramsan.com/products/pcie-storage/ramsan-70 )

I would prefer: external storage-system with redundant raidcontroller with raid10 (+Hotspare-HDD) or Raid 60 - connection to Servers via FC oder iSCSI.

 

[morfair]

If money isn't the point, go and grab some external storage-device, that would be the easiest way. There are some really nice ones out there ( http://www.violin-memory.com/products/6000-flash-memory-array/ or http://www.ramsan.com/products/pcie-storage/ramsan-70 )

I would prefer: external storage-system with redundant raidcontroller with raid10 (+Hotspare-HDD) or Raid 60 - connection to Servers via FC oder iSCSI.

I would not have external storage. No money...
But I have some servers (nodes) with their HDD. I would like to use theirs own HDD and only for theirs own CT and VM. Only in migration at neighboring node transfer data at same node's hdd.

 

[morfair]

How to store the CTs and VMs on the node's hdd which is a hypervisor for them? Only in migration to move data to the hard disk target node.