recommended setup for small size network

Lirio

Member
Apr 15, 2017
2
1
8
46
Hi, I've been using Proxmox single node for a couple of months now and I'm loving it.
It serves its purpose well to have different vm's like web, smtp and samba server.
It is setup only on a Core i5 with 8gb of ram.

Now, I want to expand proxmox. I want to create another node to act as backup server or replication machine so that just in case my 1st node goes down I can easily up the vm at 2nd node. With this purpose in mind, can you please guide me what is the best setup for my system.

Currently I have the below hardware. Total of 3 cpus.

1x proxmode running on single node with 1tb hd, core i5 & 8gb ram
2x bare pc with the same specs as node 1

Thank you.
 
  • Like
Reactions: chudnovs
Hi Mir, thanks for the quick reply, unfortunately purchasing of new hardware is not the main option. Target is to utilize un-used hardware for this project.
Though I have an option to backup the vm on node 1 and restore it on node 2 then replication will be done on mysql but i also want to replicate the whole vm itself.

thanks
 
If don't want to use the proper hardware then forget about this.
Really @mir? There is no other way than using "proper hardware"? @Lirio has two more identical nodes but you think they belong to the trash, there is nothing that can be done with them? Honestly, that's your best advice?

Hi Mir, thanks for the quick reply, unfortunately purchasing of new hardware is not the main option. Target is to utilize un-used hardware for this project.
Though I have an option to backup the vm on node 1 and restore it on node 2 then replication will be done on mysql but i also want to replicate the whole vm itself.

From your posts I reckon you want some kind of high availability. For that, you have several options, but ultimately you have to decide between these based on your needs and budget:

1. Have a dedicated storage node (NFS served by OpenMediaVault for example, or any kind of NAS) on one of the computers, and run the VMs off of it on the other two Proxmox nodes. If one of the compute nodes fail, you can migrate the VMs to the other. If the NFS node (or NAS) fails, you have downtime.

2. Run your VMs on one node, on ZFS local storage. Sync your VMs to another, similar node with ZFS send/receive. If your first node goes down, you can start the identical VMs on your second node.

3. Create a DRBD mirror between the storage of two nodes, and run the VMs on top of that. With HA cluster configured your VMs will stay online.

4. Create a 3-node Proxmox cluster, and a Ceph pool that has a monitor and at least one OSD in all nodes. If you don't need high IO performance, you can do this with gigabit Ethernet. The default Ceph pool has a replication ("size") of 2, so your VMs will stay online even if a single node goes down. Ceph is also very flexible, you can simply add more disks later should you need more storage capacity and performance, but the IO performance will be much lower than local storage.

Planning a highly available cluster like this is not a trivial task, so research your options carefully.
 
Last edited:
Really @mir? There is no other way than using "proper hardware"? @Lirio has two more identical nodes but you think they belong to the trash, there is nothing that can be done with them? Honestly, that's your best advice?
No, I have not said his current hardware is trash but I think you have forgotten to analyze the hardware at hand! Each node has a single disk, presumably HDD so given any advice using this for HA will only lead to a frustrated user blaming proxmox for a bad solution. Running HA on a single disk on each node will be dead slow and only cause grief and disappointment. This is the reason I proposed buying a dedicated storage node.
 
No, I have not said his current hardware is trash but I think you have forgotten to analyze the hardware at hand! Each node has a single disk, presumably HDD so given any advice using this for HA will only lead to a frustrated user blaming proxmox for a bad solution. Running HA on a single disk on each node will be dead slow and only cause grief and disappointment. This is the reason I proposed buying a dedicated storage node.

sorry to say it mir, but that is what you sounded like :).

"do as I say or don't do it at all" >:-(

however with OP available hardware he can in fact run a somewhat HA setup.
not perfect by no means but doable.
if I was to fix your response, I would say to OP something like,

"
Hello <OP>, while the hardware you already have is capable of running HA configuration, it would not be an optimal
setup and the whole thing would be slow and difficult to setup and manage.
if you want to try and set it up anyway here what you do: <insert the possible setup outline here>.
However if you need/want a robust and workable configuration, I would add a shared storage like a dedicated qnap or similar unit to the mix, if budget allows it. "

This would have been a nice and friendly response that also provides a useful information to OP.

now that said, I would like to add my 2 centse to the thread.

@Lirio > you do have almost enough hardware to setup and run a nice HA Proxmox cluster.

to outline the things :

A. HA Proxmox cluster require 3 nodes. while it is possible to run it on 2 nodes it will not be a smooth sailing.
B. any kind of HA setup needs an external shared storage. or at very least a decent specks on member nodes when it comes to network and storage.
i.e. if you would go with what you have with no external shared storage, you would need at least couple of extra NIC cards in each node to use solely for storage synchronization/replication use as in at least 2 nic cards to be used only for DRDB or Ceph to move/replicate data between the nodes real time fast enough for all to run smoothly.

you might also want to get a dedicated NIC card for cluster management useage. so to recap,
rach node might need at least 4 NICs if you go with 1GB speed.
1 for management
1 for network usage (this is the one you will use to connect all VMs to the network etc. while you can use the same port for management it is not recommended for HA setup. )
2 or more nics dedicated for shared storage use (Cifs or drdb)
if you can you can have 1-2 1GB ports for all network usage and get a 10Gb cards to use for storage nees.
now with dedicated 10Gb ports you can run Cifs/drdb setup on what you have quite comfortably. NOT perfect but comfortably, at least until your needs overtake the capabilities of the hardware.

if budget permits, in addition to what was said above, i would add an SSD to each node to install OS and run the VMs from, leaving the slower spinner totally for data storage.
and would plan to maybe adding second spinner and converting to at least RAID-1 setup, using ZFS or whatever to make the nodes more stable and data exchange faster.
if you want to make all this even more stable, plan for raid-1 OS install.
this way if OS disk fails you can just replace one and keep going. Proxmox VE 4.3+ support install on ZFS raid-1 directly.
 
  • Like
Reactions: chudnovs
May be, it is a bit late, to add to this thread, but I'd rather wanted to inquire an update to the same question, from the modern perspective.
As I have read @Lirio 's post, I do not see him wanted anything like a "true HA". The matter is the same for myself, as I only want some sync of the current VM (say, once per hour), to be able to start it on another host, if I boot on another PC.

In my case, I plan to have a 24/7 running host with low-power CPU, and several smaller HDDs united into redundant raid. Powerful hosts will be booted only for some purpose, like heavy task, or many users working at once. Otherwise, I will have a modest laptop, as my primary working vehicle.

So, my desire would be to have a common working environment, whatever computer I boot on.

Forgot to mention, that the whole setup is rather a home lab, having almost no critical or delicate data.

What would local wizards advise?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!