Hello all! This is my first post, and I haven't used Proxmox before, so I have a lot to ask, but didn't want to do several posts, because I would have had to repeat my long “foreword”.
My Goal: I want to create and host Java-based websites. I'm going to rent some dedicated server(s) on Hetzner, and have the websites as OpenVZ VE in Proxmox. As a Java portal is “heavy”, a complete website stack will require at least 2GB ram, 4GB being more realistic when I start getting users. I would like my websites to be HA, that is to say, a few minutes downtime maximum when a host breaks down. Finally, this should be possible with “standard server configurations”, because it gets quite expensive when you order extra hardware.
If I limit myself to about 150 EUR/month for the hardware, I can get for about that price either an “advanced” server EQ10 (6 cores, 24 GB ram), or two “middle” servers EQ6 (2 * 4 cores, 12 GB ram) or three “small” servers EQ4 (3 * 4 cores, 8 GB ram). In all cases, I get 24 GB ram total, and more than enough CPU, and each server gets 4 IPs. If I keep one IP for the host, then I can have a maximum of three websites per server.
Now, if I want HA, the single EQ10 is out of question, so this leaves me with 2 * EQ6 or 3 * EQ4. That also means that the servers have to have “spare” resources to accommodate extra VEs if one of the other servers fail. If I choose the EQ6, then I can have a maximum of 3 websites, because a) if one server fails, this leaves me with a single server running, b) there is only 3 IPs available on a single machine, and c) 12GB ram is just about enough for 3 websites. With the EQ4, I could have just about 4 sites, if they can get by with 3.75 GB ram, because you don't really get the 8GB due to host OS, Proxmox, … and because there would be 2 servers left if one died. Main disadvantage of the EQ4 is: 3 times setup fee (150 EUR) instead to two for the EQ6s.
Now for my questions:
1) Have you got any advice on the EQ4 vs EQ6 choice, as I have to place my order in the next few days?
2) The “Storage Model” wiki page says: “Note: Currently only KVM guests can benefit from these enhancements, OpenVZ containers must be located on local storage.” It is my understanding that I need some kind of shared/replicated storage to do live migrations of VMs, but DRBD falls under “local storage”. So the only way I could do live migrations of VE (not KVM), is to use DRBD, which is not available in my case because Hetzner's servers have only one network card in the standard configuration. So no live migration for me?
3) Without DRBD, what are my other options for HA, with or without live migrations? I read that ZFS can now run on Linux, and can do stuff like file-system replication, but I also read that it is quite slow, and it's probably not in the Proxmox kernels, so it's build-you-own-kernel time. I should manage that, but that makes host maintenance a pain. Any other options would be welcome.
4) Network configuration: I understand that bridged configuration is not allowed on Hetzner's servers (as well as many others), and I have seen post from many people having problems with this. So this leaves me with “routed” or NAT. It is my understanding that “routed” wastes one IP just for the bridge, and also seems more complicated. Routed would therefore not be applicable to the EQ6 configuration, because this would only leave me 2 IPs for 3 websites. I like the idea of NAT, because this “feels like” managing a LAN behind a firewall (I know, no FW in Proxmox yet, but I can do that in the host from the console).
4a) Can I have a Proxmox cluster with NAT, and have the VE keep the same LAN IP when they move between hosts? (And have the kernel somehow know where to forward the packets to the correct host when a VM “moves”. Could this be configured to happen automatically?) This seems problematic to me as one wants the VE gateway to be “local” to the host, but the gateway can't move with the VE. Can every host have the same gateway (IP) configured?!?
4b) Am I right in thinking that one of the main negative sides of NAT is that the VE does not see the real IP of the client? I think this is the case in VirtualBox. I am also assuming that it is not the case with “routed”. This has a lot of negative implication for a web/mail server. If the answer is yes, is there some way around it? I thought that maybe there was some way using something like HAProxy in the host, instead of NAT, and send the client IP as part of the “protocol” to the webserver.
4c) Any other problems with NAT, compared to “routed”?
4d) How does live migration works, when the public IP cannot be moved? (I can't move the IPs that I get with one Hetzner server to another one, but you can get such an IP at a premium price.) If I assume NAT/HAProxy and a fixed local IP that doesn't change during migration, I can imagine the following scenario: Tell target server to forward target IP to local VE, tell DNS to VE domain point to target IP, move VE, wait until (short) DNS TTL has passed, stop forwarding from original IP, done.
OK, I think that will do for a start.
My Goal: I want to create and host Java-based websites. I'm going to rent some dedicated server(s) on Hetzner, and have the websites as OpenVZ VE in Proxmox. As a Java portal is “heavy”, a complete website stack will require at least 2GB ram, 4GB being more realistic when I start getting users. I would like my websites to be HA, that is to say, a few minutes downtime maximum when a host breaks down. Finally, this should be possible with “standard server configurations”, because it gets quite expensive when you order extra hardware.
If I limit myself to about 150 EUR/month for the hardware, I can get for about that price either an “advanced” server EQ10 (6 cores, 24 GB ram), or two “middle” servers EQ6 (2 * 4 cores, 12 GB ram) or three “small” servers EQ4 (3 * 4 cores, 8 GB ram). In all cases, I get 24 GB ram total, and more than enough CPU, and each server gets 4 IPs. If I keep one IP for the host, then I can have a maximum of three websites per server.
Now, if I want HA, the single EQ10 is out of question, so this leaves me with 2 * EQ6 or 3 * EQ4. That also means that the servers have to have “spare” resources to accommodate extra VEs if one of the other servers fail. If I choose the EQ6, then I can have a maximum of 3 websites, because a) if one server fails, this leaves me with a single server running, b) there is only 3 IPs available on a single machine, and c) 12GB ram is just about enough for 3 websites. With the EQ4, I could have just about 4 sites, if they can get by with 3.75 GB ram, because you don't really get the 8GB due to host OS, Proxmox, … and because there would be 2 servers left if one died. Main disadvantage of the EQ4 is: 3 times setup fee (150 EUR) instead to two for the EQ6s.
Now for my questions:
1) Have you got any advice on the EQ4 vs EQ6 choice, as I have to place my order in the next few days?
2) The “Storage Model” wiki page says: “Note: Currently only KVM guests can benefit from these enhancements, OpenVZ containers must be located on local storage.” It is my understanding that I need some kind of shared/replicated storage to do live migrations of VMs, but DRBD falls under “local storage”. So the only way I could do live migrations of VE (not KVM), is to use DRBD, which is not available in my case because Hetzner's servers have only one network card in the standard configuration. So no live migration for me?
3) Without DRBD, what are my other options for HA, with or without live migrations? I read that ZFS can now run on Linux, and can do stuff like file-system replication, but I also read that it is quite slow, and it's probably not in the Proxmox kernels, so it's build-you-own-kernel time. I should manage that, but that makes host maintenance a pain. Any other options would be welcome.
4) Network configuration: I understand that bridged configuration is not allowed on Hetzner's servers (as well as many others), and I have seen post from many people having problems with this. So this leaves me with “routed” or NAT. It is my understanding that “routed” wastes one IP just for the bridge, and also seems more complicated. Routed would therefore not be applicable to the EQ6 configuration, because this would only leave me 2 IPs for 3 websites. I like the idea of NAT, because this “feels like” managing a LAN behind a firewall (I know, no FW in Proxmox yet, but I can do that in the host from the console).
4a) Can I have a Proxmox cluster with NAT, and have the VE keep the same LAN IP when they move between hosts? (And have the kernel somehow know where to forward the packets to the correct host when a VM “moves”. Could this be configured to happen automatically?) This seems problematic to me as one wants the VE gateway to be “local” to the host, but the gateway can't move with the VE. Can every host have the same gateway (IP) configured?!?
4b) Am I right in thinking that one of the main negative sides of NAT is that the VE does not see the real IP of the client? I think this is the case in VirtualBox. I am also assuming that it is not the case with “routed”. This has a lot of negative implication for a web/mail server. If the answer is yes, is there some way around it? I thought that maybe there was some way using something like HAProxy in the host, instead of NAT, and send the client IP as part of the “protocol” to the webserver.
4c) Any other problems with NAT, compared to “routed”?
4d) How does live migration works, when the public IP cannot be moved? (I can't move the IPs that I get with one Hetzner server to another one, but you can get such an IP at a premium price.) If I assume NAT/HAProxy and a fixed local IP that doesn't change during migration, I can imagine the following scenario: Tell target server to forward target IP to local VE, tell DNS to VE domain point to target IP, move VE, wait until (short) DNS TTL has passed, stop forwarding from original IP, done.
OK, I think that will do for a start.