Switching from VMware to Proxmox. Need help designing new environment.

G

gemcneill

New Member
May 17, 2016
5
0
1
44
Hello Everyone,

I am been watching Proxmox Ve for the past year now and I think I am ready to move my environment to Proxmox VE. I want to make sure I build it proper to replace what I have now with vmware and I was hoping those with more experience could help me here conceptually. I will list what I have, what I want to implement and what I expect from that implementation. I would appreciate any constructive feed back.

What I have now


Vmware ESXi 5.5 environment setup in a blade server with local storeage (3 blades)

Windows Domain Environment (all windows servers 2008 R2)

Replibit for server backup (uses zfs to create backups of vms that are stored on appliance and can even run vm on appliance.)

Replibit vault (off site replication of main replibit box that has all vm backups)

Replibit is backing up each server as often as 15 min. This is a complete backup (entire vm as it is). Replibit can export to any type of vm (vmware, kvm, hyperv) or even baremetal. Also it has the ability to mount the vm and see the drives so I can pull files from that iteration of the vm. The longest I will keep the backups are 45 days before they are dropped. The appliance can even in an emergency situation run a vm live from the appliance if a server fails, and I can export and restore later.

What I want to replace it with

Proxmox VE environment with local ZFS storage (wont bother with details of box at this point)

Second Identical Proxmox VE with local zfs Storage (used for replication offsite for DR)

Plan is to move all vms to the new proxmox VE. Use Proxmox VE with zfs to enable snapshotting of the VMs to use as backups and store on the local appliance. Setup Secondary proxmox VE and use PVE-zsync to replicate the primary proxmox VE . Be able to access snapshots as backups to restore files or folders or even switch to it and make it the active VM. Lastly in the event of a a total lost of hardware at the primary site, the secondary Proxmox VE will be at a secondary site with all vms and snapshots of vms replicated exactly to that hardware at a secondary physical site.

Now I don't know if I can even do this, and second if this is even a good idea. I know that I would not have HA in this setup which is fine for me at this point. My main take aways is can I use snapshotting in Proxmox with the power of ZFS to use as backups. This eliminates the need for a replibit appliance. I believe that the DR should work as I envision as long as PVE-zsync will replicate the envrionment (vms, and snapshots) exactly to the secondary box at the offsite location.

Any insight would be appreciated
 
Should work if your machines have a way more power than the ones you already have. Do not underestimate the demands of ZFS on your hardware while serving as a virtualization environment. You should have plenty of RAM and only use at max half of it (rest is for ZFS). Using SSD-only should also be a good choice (not speaking of L2ARC and ZIL, one RAW power). This can be consumer grade SSDs which are not that good for ZFS, but still much, much better than disks.
 
Thanks for the advice. I was aware of the memory requirements but did not know the cpu was that taxing. However the server is easily over built for the environment so there would be no issue. As for the disk requirements was looking at using normal spin disks (3.5 sas 4tb 7200) setup in zfs's raid 10 . Intended to follow what I found here

http://www.zfsbuild.com/2010/04/15/explanation-of-arc-and-l2arc/

Basically as much ram as possible for ARC and then 2 ssds for L2ARC and then a SLC SATADOM for Zil. I would like to go to ssd but the power loss protection not being an option for most consumer ssd as well as enterprise ssds with plp are very costly I was hoping this would be a viable alternative.

What do you think of that configuration?
 
CPU is not that demanding, but of course more than a hardware raid controller.

Do you have no option to use "real" SAS drives instead of NL-SAS? How many 4 TB drives do you want to use? How many VMs do you plan to use?

How fast does your SLC SATADOM sync writes? That's really cruicial for a fast setup. For L2ARC, you do not need more than one SSD, because it cannot be mirrored and it is only a cache, which can and will fail eventually.

Do you have no UPS for your system that you can use "consumer grade" for storage?
 
I would have a UPS for the setup of course, but I am just being extra careful. I am just worried something would happen like a power outage over night and something goes wrong with the UPS. Looks like I can make it a lot less complicated if I just use SSDs instead of trying to setup using spin disks and caching.

However one thing that I am really glossing over that worries me is how VM snapshots are done and using them as a form of backup. Remember I am using a replibit appliance for backup which works very well. I am trying to remove the need for it by creating 2 proxmox boxes that are the same. As I understand it there is away to do the exact same thing that replibit is doing within proxmox (replibit uses zfs and kvm). Am I using something that should not be used this way? I come from the vmware world and what echos in my head is that "snap shots are not backups" , however with using replibit, that is kinda what is being used. I am ok with continuing to use my backup solution, but I was hoping I wouldn't need it with 2 proxmox boxes where I create snapshots for backups and then replicate everything to the second box for DR.
 
And please use two separate UPS for your server (hopefully it has two PSUs) if you can. Off-site ZFS replication is really awesome!
 
vkhera said:
Only if the system crashes. ZFS is supposed to survive a ZIL failure -- all that data is still in RAM.
Click to expand...
No, when data is sent to Zil it is no longer in RAM - that is the hole idea. If you had no Zil then the data would go directly to the disks. So if the disks containing the data is toasted then the data is gone.
 
mir said:
That's one man's opinion. Why do you trust him more than the ones who designed and implemented ZFS at SUN, the large providers of ZFS storage OS's etc. ?
Click to expand...

The ZIL is *never* read unless it is replaying after a crash. You cannot lose data until your system crashes, and the only window of opportunity to lose the data is during the interval where the flush happens. After the SLOG device fails, further ZIL data is written to the pool as if that device did not exist.

To quote the official docs (from Oracle) http://docs.oracle.com/cd/E19253-01/819-5461/gfgaa/index.html
  • Log devices can be unreplicated or mirrored, but RAID-Z is not supported for log devices.

  • If a separate log device is not mirrored and the device that contains the log fails, storing log blocks reverts to the storage pool.
 
I am learning a great deal here, thank you to everyone chiming in. Looks like if I go with a SSD setup with 2 UPS on each power supply I really don't run into an issue with L2ARC and ZIL as I will not need to set them up (unless I am mistaken). As for backing up data looks like from what I read on what mir posted, I need to use znapzend which will do the scheduling of snapshotting and replication as well as dropping off old snapshots. I do have a question about znapzend though. It appears it will dump to another zfs location but in my case I want to have an exact duplicate of the infrastructure. So from my limited knowledge so far here is what I think I need to do

Setup new infrastructure with vms and once everything is running use znapzend to schedule the creation of snapshots and maintenance of them (i.e drop old ones after a specific time). Znapzend can also replicate this data to another zfs store, which would be in this case be the second box off site which will be exactly the same hardware wise. But should I use this? I was thinking that proxmox has an option to replicate itself to another box exactly as it is. My though process was to setup proxmox the same on the second box as the first, minus the vms and data from the first. Then Proxmox would have a replicate ability to say to the second box, be like me in every way, and then do this on a schedule. So ZnapZend would handle the snapshots and proxmox would then replicate everything (including the snapshots) to the second box. The idea would be if the first box had a melt down, I could just drive to the second location and bring that box to the main location and fire it up and it would pick up where it left off from the last replication.

I am sorry I am asking so many questions, I just want to make sure my expectations are correct before I start down this path.
 
"
gemcneill said:
I do have a question about znapzend though. It appears it will dump to another zfs location but in my case I want to have an exact duplicate of the infrastructure. So from my limited knowledge so far here is what I think I need to do
Click to expand...
I think you missed this:
You can configure any number of remote destinations for a fileset. Even no destinations at all if all you need are well managed local snapshots.
Click to expand...
For local only I would create another local pool to use as destination.
 
Ah ok, so are you suggesting creating a second pool to use as the snapshot destination using znapzend? If so that makes sense, but I guess my greater question is, if there is a way to replicate proxmox setup exactly if the hardware is the same. That way znapzend creates my backups using snapshotting and then, I hope, proxmox has a function that allows DR replication to a secondary box. That way I have on my primary box my local backups as well as I have a second box at an off site location that is , through replication, exactly the same as the first, including the backups, but all the vms are off and just waiting just in case of a major failure of the hardware at the primary location. Again I am sorry if I am not being clear on what I want I can still use replibit to do the backup portion and even be the DR if necessary, but I just think it would not be necessary since the new infrastructure would be built on KVM and ZFS , but I could be wrong.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back