Help with making a functional Proxmox setup out of this

[NetVent]

I'm assisting a community organization to virtualize their small infrastructure. I've used Proxmox primarily for personal use so far.

Through some awesome local support and donations from IT companies they got some older, but IMO still very usable, hardware:
2x Dell R610 servers that are almost identically configured with 16GB and 24GB RAM. About 600GB of space in each server (4x 150 GB drives)
1x 16 port GigE switch.
1x fiber channel storage switch with 4x HBAs.
A number of workstations but nothing good for virtualization.

Goals:
- Virtualize their existing "servers" which are just desktops right now as they keep breaking. They host a website, shared file server, calendar, CRM app, some other webapps, windows apps, etc.
- It's used hardware, so make it easy to recover from hardware failure (so one R610 dying). Manual fail-over is fine...
- Backup critical files (in VMs) and VM images themselves. Surprisingly they do have an LTO3 tape drive in one of the existing desktops.
- Rely on open-source technology as much as possible (limited budget so can't afford much in terms of licensing).

I've been reading up on proxmox clusters and replicated storage. A little (ok A LOT) overwhelmed with choices.

What could be done with what they got? I would welcome any advise and suggestions. Thanks!

 

[m.ardito]

you could do a two nodes cluster somehow http://pve.proxmox.com/wiki/Two-Node_High_Availability_Cluster
has some con, but the pro is needs only two nodes, and is simple (drbd HA is more complex)
if there is budget a safe external shared storage for backups at least would be even better.

Marco

 

[udo]

I'm assisting a community organization to virtualize their small infrastructure. I've used Proxmox primarily for personal use so far.

Through some awesome local support and donations from IT companies they got some older, but IMO still very usable, hardware:
2x Dell R610 servers that are almost identically configured with 16GB and 24GB RAM. About 600GB of space in each server (4x 150 GB drives)
1x 16 port GigE switch.
1x fiber channel storage switch with 4x HBAs.
A number of workstations but nothing good for virtualization.

Goals:
- Virtualize their existing "servers" which are just desktops right now as they keep breaking. They host a website, shared file server, calendar, CRM app, some other webapps, windows apps, etc.
- It's used hardware, so make it easy to recover from hardware failure (so one R610 dying). Manual fail-over is fine...
- Backup critical files (in VMs) and VM images themselves. Surprisingly they do have an LTO3 tape drive in one of the existing desktops.
- Rely on open-source technology as much as possible (limited budget so can't afford much in terms of licensing).

I've been reading up on proxmox clusters and replicated storage. A little (ok A LOT) overwhelmed with choices.

What could be done with what they got? I would welcome any advise and suggestions. Thanks!

Hi,
against Marcos suggestion, I prefer an 3-node cluster. Take simply the LTO-3 Workstation as third node (only for quorum - and as storage-node for backup).
With the two Dell you can use DRBD (do you have an raid controller in the servers?) to allow live migration / failover.

With an Backup-VM (like bareos) you can run all backups inside the VMs and store the data on the LTO-3 LW.
I do the same - 1 LTO-6 Jukebox is directly connected to one pve-node and this node act also as bareos-sd. The backup-server itself is virtualized (kvm). Run very well!

For the FC-Switch I missed FC-Storage?!

Udo

 

[m.ardito]

Hi,
against Marcos suggestion, I prefer an 3-node cluster. Take simply the LTO-3 Workstation as third node (only for quorum - and as storage-node for backup).

Yes (udo is much more expert than me, btw) I agree 3 nodes is better
But he said "with what they got" and I assumed that the "lto workstation" could be also a Windows one, or not so reliable/redundant or maybe not always on.
Of course If you can have 3 "nodes" is always better. I work with two node since years (no HA) and never had troubles, but if you can go for 3.

Marco

 

[NetVent]

you could do a two nodes cluster somehow http://pve.proxmox.com/wiki/Two-Node_High_Availability_Cluster
has some con, but the pro is needs only two nodes, and is simple (drbd HA is more complex)
if there is budget a safe external shared storage for backups at least would be even better.

Marco

Currently there is no budget for external shared storage. Aside from some desktop/software purchases, their IT infrastructure is built on donated equipment. It is possible that another server gets donated but best I can do right now is to re-purpose one of the workstations and fill it up with drives. But that doesn't sound like the shared storage you must be thinking of.

 

[NetVent]

Hi,
against Marcos suggestion, I prefer an 3-node cluster. Take simply the LTO-3 Workstation as third node (only for quorum - and as storage-node for backup).
With the two Dell you can use DRBD (do you have an raid controller in the servers?) to allow live migration / failover.

The CPUs in older workstations, which includes the one with LTO-3 tape drive, do not appear to have VT extensions. I was under impression that to run Proxmox you needed VT (sort of how VMware requires it).

The two Dells do have embedded RAID controllers. I don't recall which ones. Might be the most basic ones. Does the controller have to have a specific feature set to support DRBD?

With an Backup-VM (like bareos) you can run all backups inside the VMs and store the data on the LTO-3 LW.
I do the same - 1 LTO-6 Jukebox is directly connected to one pve-node and this node act also as bareos-sd. The backup-server itself is virtualized (kvm). Run very well!

Thank you for the suggestion. I don't have experience with bareos but am looking into it as something to use. Again, a little overwhelmed with choices, terminology, and things like mtx. So plenty of research to do.

For the FC-Switch I missed FC-Storage?!

Yes, a fiber channel switch along with a few controller cards (HBAs) were donated. There is no central shared storage. So not sure how this helps them. But it's there. Perhaps someone has a suggestion.

 

[NetVent]

Yes (udo is much more expert than me, btw) I agree 3 nodes is better
But he said "with what they got" and I assumed that the "lto workstation" could be also a Windows one, or not so reliable/redundant or maybe not always on.
Of course If you can have 3 "nodes" is always better. I work with two node since years (no HA) and never had troubles, but if you can go for 3.

Marco

I do believe workstation with LTO3 is a Windows machine which is also one of their file servers. Someone had set it up to backup their shared files. There are 2 other shared drives that reside in different workstations. Yeah it's a bit of a mess. Workstation with LTO3 can be re-purposed... not sure how they will do backups in the interim. But if it helps, I can do it.
It's CPU might not support VT for virtualization. Not sure if Proxmox can even run on it.

I've been doing research and still trying to understand how to create a cluster with what they got and without shared storage. Everyone I read, it says 2-node cluster is possible but not a good idea. Even if I re-purpose another machine to make it a 3-node cluster... they still have no shared storage.

 

[udo]

I do believe workstation with LTO3 is a Windows machine which is also one of their file servers. Someone had set it up to backup their shared files. There are 2 other shared drives that reside in different workstations. Yeah it's a bit of a mess. Workstation with LTO3 can be re-purposed... not sure how they will do backups in the interim. But if it helps, I can do it.
It's CPU might not support VT for virtualization. Not sure if Proxmox can even run on it.

Hi,
pve need an 64bit cpu. Without VT you can't use kvm, but for quorum it's enough (and OpenVZ is also possible).

I've been doing research and still trying to understand how to create a cluster with what they got and without shared storage. Everyone I read, it says 2-node cluster is possible but not a good idea. Even if I re-purpose another machine to make it a 3-node cluster... they still have no shared storage.

DRBD can used for this scenario (like suggested before) http://pve.proxmox.com/wiki/DRBD

Udo

 

[udo]

Currently there is no budget for external shared storage. ...
but best I can do right now is to re-purpose one of the workstations and fill it up with drives.

Hi,
if you think about that, you have another choice for storage: ceph!

But if your hosts are not powerfull and with 1GB Nics the performance it not really fast (but perhaps fast enough)... so you should try that to see if the performance meet the usage.

Udo

 

[m.ardito]

I do believe workstation with LTO3 is a Windows machine which is also one of their file servers. Someone had set it up to backup their shared files. There are 2 other shared drives that reside in different workstations. Yeah it's a bit of a mess. Workstation with LTO3 can be re-purposed... not sure how they will do backups in the interim. But if it helps, I can do it. It's CPU might not support VT for virtualization. Not sure if Proxmox can even run on it.
I've been doing research and still trying to understand how to create a cluster with what they got and without shared storage. Everyone I read, it says 2-node cluster is possible but not a good idea. Even if I re-purpose another machine to make it a 3-node cluster... they still have no shared storage.

a two node cluster behaves very well (if hardwre is reliable) but requires more manual administration and could temporarily lock some service, until you restore its normal state.

a third node could have different roles
- as a quorum resolver (with two nodes, if 1 is unavailable the cluster goes in locked state (you can recover manually until the other node is back again). A per the wiki it could be just "A small (about 10MB) block device".
- as a shared storage (will allow live migration): you need nfs at least.
- as an external storage (will allow easy recover from backups even if not shared) eg: 1 or 2 windows workstation could serve 2 ntfs shares and you can mount each one in a different server and use it as "local". if you backup there, you can merge (or sync) those two windows folders afterwards in some way.
- if LTO tape is external, it could be perhaps passed through to a vm, and easily connected to its host.

[edit] you could also use a folder on each node for local backup and another mounted on the other node as "remote backup" and alternate bakup jopbs on local and remote storage, or use only local and then sync both local folders with rsync... you get the idea[/edit]

as external storage, any low budget nas with 2 disk in raid1 is better than any 1 disk workstation, imho. apart the 2 disks it could cost under 200€ new and you could have iscsi and nfs and much more in that small box. eg: active directory if you need that.

see this eg: https://www.qnap.com/i/it/product/model.php?II=131&event=3 but there are others. You could plan to buy that as soon as you get funds or donations.

imho you can start in a simple way and then evolve it. or wait for some more money or hardware.

as for ceph, wiki says "For a production system you need 3 servers minimum. For testing you can get by with less," http://pve.proxmox.com/wiki/Storage:_Ceph
and "For smaller deployments, it is also possible to run Ceph services directly on your Proxmox VE nodes. Recent hardware has plenty of CPU power and RAM, so running storage services and VMs on same node is possible." http://pve.proxmox.com/wiki/Ceph_Server

so, it may be a way or not. nfs is definitely simpler, I would go with the small external nas.


Marco