Hi All,
I am redoing my entire home cluster, adding storage, moved to more powerful servers. Presently copying files around, takes forever, yet gives me time to make decisions. I am having trouble with docker and how to go about implementing it.
In my previous implementation, I was using LXC and unprivileged containers. This worked, "ok". The challenge being, I had to settle for VFS (ugh) or fuse-overlay (also ugh). Neither of these were great solutions.
The main reason I'm leaning towards LXC vs. VM is that I have 3 NVIDIA datacenter cards in my highest powered cluster node, and I don't want to lock their resources to a specific instance with pass-through. There's plenty of power there, and ideally I want to remain as flexible as possible around where I use those resources.
My first thought was that for internal workloads I could use privileged LXC. This gives me less security, yet lets me use standard overlay. I set this up, all looked good, yet when I tried to start spinning up docker containers it worked for hello world, yet failed for more complex containers. Mainly ones that implemented their own python stacks inside of docker. I'm not sure what was causing this, yet it was consistent, and I couldn't find a solution for it using Google-foo. I'd still be open to this if I could figure out why docker didn't like it.
My second thought was to install docker on the Proxmox host. To avoid issues with future upgrades, I'm concerned about this particular solution as well. I want to keep the hosts as clean as possible. Plus, I lose the Proxmox integrated clustering and backup with this option.
My third option is to just drop Proxmox in favor of building my own cluster based upon similar stacks. This is sort of my last option as I enjoy having the community to assist when issues to crop up, as well a team working on a single product. Similarly, I could look at something like OpenNebula as a Proxmox GUI replacement. Yet, I've used Proxmox for such a long time that I'm not sure, I want to learn an entirely new platform.
Since am yet to commit to a solution as I wait for 300 TB of data to move around, I was hoping for some input and opinions. What would you do? Do you know how to fix the privileged LXC's? Are there other options that maybe I should put on my radar?
Thanks in advance for taking the time to help me figure this out!
Keith
I am redoing my entire home cluster, adding storage, moved to more powerful servers. Presently copying files around, takes forever, yet gives me time to make decisions. I am having trouble with docker and how to go about implementing it.
In my previous implementation, I was using LXC and unprivileged containers. This worked, "ok". The challenge being, I had to settle for VFS (ugh) or fuse-overlay (also ugh). Neither of these were great solutions.
The main reason I'm leaning towards LXC vs. VM is that I have 3 NVIDIA datacenter cards in my highest powered cluster node, and I don't want to lock their resources to a specific instance with pass-through. There's plenty of power there, and ideally I want to remain as flexible as possible around where I use those resources.
My first thought was that for internal workloads I could use privileged LXC. This gives me less security, yet lets me use standard overlay. I set this up, all looked good, yet when I tried to start spinning up docker containers it worked for hello world, yet failed for more complex containers. Mainly ones that implemented their own python stacks inside of docker. I'm not sure what was causing this, yet it was consistent, and I couldn't find a solution for it using Google-foo. I'd still be open to this if I could figure out why docker didn't like it.
My second thought was to install docker on the Proxmox host. To avoid issues with future upgrades, I'm concerned about this particular solution as well. I want to keep the hosts as clean as possible. Plus, I lose the Proxmox integrated clustering and backup with this option.
My third option is to just drop Proxmox in favor of building my own cluster based upon similar stacks. This is sort of my last option as I enjoy having the community to assist when issues to crop up, as well a team working on a single product. Similarly, I could look at something like OpenNebula as a Proxmox GUI replacement. Yet, I've used Proxmox for such a long time that I'm not sure, I want to learn an entirely new platform.
Since am yet to commit to a solution as I wait for 300 TB of data to move around, I was hoping for some input and opinions. What would you do? Do you know how to fix the privileged LXC's? Are there other options that maybe I should put on my radar?
Thanks in advance for taking the time to help me figure this out!
Keith