Proxmox for Renderfarm

[PabloUserT38]

Hi,

I'm running a small VFX studio infrastructure. We have a little renderfarm (8 nodes) and I'm considering using Proxmox to manage it. These are the things I plan to use Proxmox for:

• Manage and monitor the physical servers from a single location
• Installing a single Linux (or Windows) image on each server from a template (so I need to create a template first). That template might need to change more or less frequently. The idea is to configure the first node and automatically replicate it to the others. It's OK to run a single VM taking all the resources on each node if that helps.
• Use Ansible or any other configuration tool to manage updates, installs, etc. (I understand that's something outside the scope of Proxmox and Ansible doesn't care if the servers are part of a cluster, a group or anything).

Is Proxmox a good solution for these needs?

Would a Proxmox cluster provide any advantage to just thinking of these 8 nodes as a simple "group of servers" (I'm not quite sure what the concept of Proxmox cluster brings to the table)?

I'm planning to use Proxmox in other parts of the company (file servers and other systems in the network), so using it on the renderfarm would make it easier to manage the whole infrastructure.

 

[dylanw]

Hi,

Perhaps my opinion is biased, being an employee, but I'll still try to answer your question

Would a Proxmox cluster provide any advantage to just thinking of these 8 nodes as a simple "group of servers" (I'm not quite sure what the concept of Proxmox cluster brings to the table)?

The cluster provides a few advantages for your use case. Primarily, it provides a single interface to all the nodes of the cluster, meaning that administration is much simpler than having to individually manage 8 servers. In addition, with support for CEPH [0][1], you can pool each node's storage together, receiving redundancy in the process.

Manage and monitor the physical servers from a single location

In regard to this, we have a multi-master design, meaning that you can manage your entire cluster from any node of the cluster.

Installing a single Linux (or Windows) image on each server from a template (so I need to create a template first). That template might need to change more or less frequently

We support both cloning VMs and template creation from a VM image. Should you need to change the template, it's probably easiest to just delete the old one and create a new one from the current image. Otherwise, you can skip the template creation altogether and simply clone the latest version of the VM again.

Use Ansible or any other configuration tool to manage updates, installs, etc. (

I am not so familiar with Ansible myself, but I believe there are proxmox users using it for management.

I'm planning to use Proxmox in other parts of the company (file servers and other systems in the network), so using it on the renderfarm would make it easier to manage the whole infrastructure.

Happy to hear it! And in this case, you are correct. The render farm could still communicate on a separate network to the rest of your nodes and have its own resources defined, while also being manageable from the same interface as your other nodes.


[0] https://docs.ceph.com/en/latest/start/intro/
[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_pveceph

 

[kevinshane]

pve will not that much help of a "cpu" render farm, you need as much cpu resources as you want to rendering, so in this case deploying to multi VM will waste OS resouces(plus render software licenses etc), rather consider using Deadline or something else to manage the render node.
However for a gpu render farm, pve can deploy multiple VMs, each VM(render node) has a dedicated gpu passthrough, but still you need a "middle man" to communicating with each node, such as assigning render jobs and monitoring rendering status etc (in this case the middle man is Deadline)

 

[PabloUserT38]

Hi,

Perhaps my opinion is biased, being an employee, but I'll still try to answer your question


The cluster provides a few advantages for your use case. Primarily, it provides a single interface to all the nodes of the cluster, meaning that administration is much simpler than having to individually manage 8 servers. In addition, with support for CEPH [0][1], you can pool each node's storage together, receiving redundancy in the process.


In regard to this, we have a multi-master design, meaning that you can manage your entire cluster from any node of the cluster.


We support both cloning VMs and template creation from a VM image. Should you need to change the template, it's probably easiest to just delete the old one and create a new one from the current image. Otherwise, you can skip the template creation altogether and simply clone the latest version of the VM again.


I am not so familiar with Ansible myself, but I believe there are proxmox users using it for management.


Happy to hear it! And in this case, you are correct. The render farm could still communicate on a separate network to the rest of your nodes and have its own resources defined, while also being manageable from the same interface as your other nodes.


[0] https://docs.ceph.com/en/latest/start/intro/
[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_pveceph

Thanks, Dylan.

I've started to try Proxmox in the renderfarm. So far I have two issues:

1- Renaming nodes: I made a mistake on the name of one of the nodes. I had already installed a VM on it when I realized it. Changing the name was a complete nightmare, to the point I decided to reinstall Proxmox on the two nodes I was trying Proxmox on. Removing nodes from a cluster or renaming them should be part of the GUI and should be EASY. This is a serious hindrance because it forces us to very carefully design our naming conventions beforehand which is not always possible.

2- No unified management of different clusters. This is another serious issue. The renderfarm won't be the only cluster in the company. Not being able to manage them all from a single point is bad. It's not just the comfort of having a great picture of the whole infrasctructure in one place, but also the ability to move resources between clusters which is expected.

I find Proxmox nice and easy for those things it can do out of the box, and terribly difficult for those it can't. Here are other few issues that don't affect me at this moment but would bite in the future:

- No docker management: I won't discuss the technical issues of managing docker images, but these days containers mean docker, not LXC. LXC are useless for me and lack of docker management is an important hindrance (I know the workarounds, but still).
- No erasure coding for CEPH. We're using ZFS at the moment, but when moving to CEPH we'll need that important feature to save on number of drives.
- No modularity/expandability: Cockpit is based on "applications" or modules. That leads to a less cohesive solution, but at the same time, someone can build whatever is missing from the base installation. I know it takes a new whole architecture to make a modular Proxmox but you might want to consider it for the future.

I'm trying Cockpit next week. I know it'll have its drawbacks too. I guess I'll make a decision by late June.

Thanks!

 

[PabloUserT38]

pve will not that much help of a "cpu" render farm, you need as much cpu resources as you want to rendering, so in this case deploying to multi VM will waste OS resouces(plus render software licenses etc), rather consider using Deadline or something else to manage the render node.
However for a gpu render farm, pve can deploy multiple VMs, each VM(render node) has a dedicated gpu passthrough, but still you need a "middle man" to communicating with each node, such as assigning render jobs and monitoring rendering status etc (in this case the middle man is Deadline)

Using a single VM per node makes little to no difference in resources and allows for easy updates and other advantages (like trying different OS on the whole renderfarm). Bare metal installations are silly these days.

Deadline (which we already use) has nothing to do with this.