Hey everyone!
I've been using Proxmox on my home server for a while and have been quite happy, but I have a potentially bigger scenario here and I'd like to seek the advice of you fine folks.
I work for a local IT company that has a large construction group as a client. I am filling in for their IT Infrastructure Head (also a good friend of mine) while he's off for a couple months on medical leave. Most of their IT infrastructure is in Azure, but they do have a fairly chonky couple of VMware 8.0 hosts on-prem that host a few smaller functions but critically, are also the on-prem storage point for all their Veeam backups. Because Broadcom is Broadcom, they're trying to massively hike their support contract rates for VMware and they're saying screw that. They've been out of support for a few months now. I suggested Proxmox (with their much more reasonably priced support of course) as an alternative and they're interested. Several of us at my employer have experience with Proxmox, but none of us have done VMware migrations, plus this one might have some unique challenges.
I have consulted the wiki page on migrations and read up on it and while it was helpful, I'd like to lay out the exact setup they have here and get feedback on not only the best way to approach the process, but also any potential gotchas that we might run into because they are looking to do several things in the process of this and I'm concerned about both potential risks and most importantly, potential downtime to achieve this.
Currently, they have two hosts:
Host 1
HPE ProLiant DL385 Gen11
128 CPU(s) x AMD EPYC 9374F 32-Core Processor
512GB RAM (180GB in use on average)
2TB SSD (1.25TB in use)
10TB 10K SAS HDD (3.4TB in use)
Host 2
HPE ProLiant DL385 Gen10 Plus
32 CPU(s) x AMD EPYC 7262 8-Core Processor
96GB RAM (75GB in use on average)
240GB SSD (30GB used)
6TB SAS HDD (4TB in use)
At the end of this, they want everything consolidated to just Host 1, though Host 2 could be used as a staging host if needed. There are also 2-3 VMs currently running on Host 2 that we hope to have retired before this migration, which will free up capacity. Everything should be able to comfortably fit on Host 1 when that's done.
In addition, there is a 75TB Synology unit that stores their Veeam backups. The VM that hosts Veeam Backup & Replication on Host 1 directly connects to this device via Raw Device Mapping. According to my friend, this was required due to VMware's virtual disk size limits. From a bit of reading I've done, it sounds like Proxmox might not have this limitation, but I'm curious if people think this would be best maintained as-is or changed to something else. I'm trying to see exactly how the connection works, but the credentials on file for the Synology aren't working so I'm hoping I can figure it out once that's sorted.
I'm not sure how many physical NICs the host has (I'm guessing at least 4), but there are 10 virtual networks setup on the cluster. We might not need a couple of them, but I'm not sure yet.
The Challenges
Here's where things get fun. We are planning this well ahead of time and the client knows down time will be required to do this, but some of these systems host things that can't be down for long during work hours. I'm not sure if achieving this over a weekend is viable given the amount of conversion work and such that will have to be done. If not, we might be able to take up-to-the-minute backups of some of the VMs and temporarily spin them up in Azure to keep things going, though that will pose its own challenges as the data deltas grow. We might need to buy or rent additional storage as well to act as a "bridge" when converting the disks as I assume we're going to need twice as much space for those as we currently have to make converted copies. I'm really hoping that the Veeam backups drive that's attached via Raw Device Mapping isn't in a VMware only format and that we can just detach it from the VMware VM and onto the Proxmox VM. If not, that's going to be...fun. Veeam also officially supports Proxmox now which is good, though I know we'll have to take fresh full backups of the new VMs since well, they're new. I believe the backup store has plenty of free space so that shouldn't be an issue I don't think.
Aside from just getting general feedback on this scenario, here's the specific questions I can think of at the moment:
I'll probably think of more questions and will update as I get them. Sorry for the long post, but we want to make 100% sure of this before we propose it to the client since it's kind of a weird setup. I think we can pull it off, I just want to have all our ducks in a row and since people here are very knowledgeable and helpful, I'm hoping some might be willing to office advice.
Thanks very much!
I've been using Proxmox on my home server for a while and have been quite happy, but I have a potentially bigger scenario here and I'd like to seek the advice of you fine folks.
I work for a local IT company that has a large construction group as a client. I am filling in for their IT Infrastructure Head (also a good friend of mine) while he's off for a couple months on medical leave. Most of their IT infrastructure is in Azure, but they do have a fairly chonky couple of VMware 8.0 hosts on-prem that host a few smaller functions but critically, are also the on-prem storage point for all their Veeam backups. Because Broadcom is Broadcom, they're trying to massively hike their support contract rates for VMware and they're saying screw that. They've been out of support for a few months now. I suggested Proxmox (with their much more reasonably priced support of course) as an alternative and they're interested. Several of us at my employer have experience with Proxmox, but none of us have done VMware migrations, plus this one might have some unique challenges.
I have consulted the wiki page on migrations and read up on it and while it was helpful, I'd like to lay out the exact setup they have here and get feedback on not only the best way to approach the process, but also any potential gotchas that we might run into because they are looking to do several things in the process of this and I'm concerned about both potential risks and most importantly, potential downtime to achieve this.
Currently, they have two hosts:
Host 1
HPE ProLiant DL385 Gen11
128 CPU(s) x AMD EPYC 9374F 32-Core Processor
512GB RAM (180GB in use on average)
2TB SSD (1.25TB in use)
10TB 10K SAS HDD (3.4TB in use)
Host 2
HPE ProLiant DL385 Gen10 Plus
32 CPU(s) x AMD EPYC 7262 8-Core Processor
96GB RAM (75GB in use on average)
240GB SSD (30GB used)
6TB SAS HDD (4TB in use)
At the end of this, they want everything consolidated to just Host 1, though Host 2 could be used as a staging host if needed. There are also 2-3 VMs currently running on Host 2 that we hope to have retired before this migration, which will free up capacity. Everything should be able to comfortably fit on Host 1 when that's done.
In addition, there is a 75TB Synology unit that stores their Veeam backups. The VM that hosts Veeam Backup & Replication on Host 1 directly connects to this device via Raw Device Mapping. According to my friend, this was required due to VMware's virtual disk size limits. From a bit of reading I've done, it sounds like Proxmox might not have this limitation, but I'm curious if people think this would be best maintained as-is or changed to something else. I'm trying to see exactly how the connection works, but the credentials on file for the Synology aren't working so I'm hoping I can figure it out once that's sorted.
I'm not sure how many physical NICs the host has (I'm guessing at least 4), but there are 10 virtual networks setup on the cluster. We might not need a couple of them, but I'm not sure yet.
The Challenges
Here's where things get fun. We are planning this well ahead of time and the client knows down time will be required to do this, but some of these systems host things that can't be down for long during work hours. I'm not sure if achieving this over a weekend is viable given the amount of conversion work and such that will have to be done. If not, we might be able to take up-to-the-minute backups of some of the VMs and temporarily spin them up in Azure to keep things going, though that will pose its own challenges as the data deltas grow. We might need to buy or rent additional storage as well to act as a "bridge" when converting the disks as I assume we're going to need twice as much space for those as we currently have to make converted copies. I'm really hoping that the Veeam backups drive that's attached via Raw Device Mapping isn't in a VMware only format and that we can just detach it from the VMware VM and onto the Proxmox VM. If not, that's going to be...fun. Veeam also officially supports Proxmox now which is good, though I know we'll have to take fresh full backups of the new VMs since well, they're new. I believe the backup store has plenty of free space so that shouldn't be an issue I don't think.
Aside from just getting general feedback on this scenario, here's the specific questions I can think of at the moment:
- Since we have two hosts but one is much less powerful than the other, we can't simply move all the running VMs over to it and then do the automatic ESXi import on each. We don't have to have them all running necessarily and can plan for downtime, but the OS drives are stored on the hosts themselves (with backups from Veeam on that Synology NAS appliance) and there isn't enough space on Host 2 to accommodate all the disks from Host 1 at once. This will be a problem as we obviously can't run two hypervisors on Host 1 at once. Is the solution for this just to acquire or rent additional storage we can attach as a temporary datastore to Host 2 to store all of Host 1's disks until we can import them?
- Any concerns about bringing over that many virtual networks or do we just want to make sure we re-create them like-for-like in Proxmox?
- Any concerns with drivers for hardware on a server like this with Proxmox? Should we get an HPE Debian driver pack for it if one's available or should Proxmox be able to figure it out?
- I've never seen a NAS attached via Raw Device Mapping like this. If anyone has that experience, is it actually a VMware formatted volume or does it just see it as a big-ass hard drive that's directly formatted with the OS' filesystem? I'm hoping it's the latter because it's NTFS and we can hopefully just attach it to the Proxmox VM and it'll just work, but I'm not sure as I can't check the appliance right now.
- Veeam does support restoring VMware backups to a new Proxmox host, similar to automatic import, but Veeam does it all. Anyone have experience with this?
I'll probably think of more questions and will update as I get them. Sorry for the long post, but we want to make 100% sure of this before we propose it to the client since it's kind of a weird setup. I think we can pull it off, I just want to have all our ducks in a row and since people here are very knowledgeable and helpful, I'm hoping some might be willing to office advice.
Thanks very much!
