Proxmox on aarch64 (arm64)

[Johannes S]

Ugh. That sounds absolutely horrible for troubleshooting/debugging any problems that occur.

On the other hand together with it's immutable state it should make tampering though bad actors a lot harder.

The same approach is used by kubernetes distribution talos and less extremely TrueNAS ( the latter still has ssh though, talos has not, both expect users to use the api or tools/uis based on the UI) so I can see the appeal from a devops perspective. Personally I prefer the flexibility of ProxmoxVE. Like always: Pick your poison

 

[justinclift]

less extremely TrueNAS

Um, as you mention TrueNAS does have ssh, so it's very much not the same as these things that can't be accessed.

Also, am kind of curious why you'd say not having ssh is a feature for DevOps? How are you expecting DevOps staff to diagnose problems if they don't even have shell access? Maybe these things have terminal/console/tty access instead, which would do effectively the same thing in a pinch...?

 

[Johannes S]

Um, as you mention TrueNAS does have ssh, so it's very much not the same as these things that can't be accessed.

Yes, but the TrueNAS documentation clearly states that you are expected not to alter the system but use the API, the WebUI or the TrueNAS specific cli tools. So although you can use ssh it's less powerful than on a classical Unix box (since you can't do much on the system). Whether this is a good thing or not, depends on your point of view
My point concerning devops is that if you happen to be a former developer who somehow ended up in devops it's very appealing to do everyting with an API and a debug channel (maybe prometheus metrics or an endpoint for syslogs), because that is what you are used to. For sysadmins (which I am and I suspect you too) who are used to do everything via ssh the lack of ssh is an antifeature

 

[justinclift]

Heh Heh Heh. The good developers i know (and work with) very much understand how to use ssh and a shell effectively. Doing everything over an API (etc) is just such a huge waste of time/effort for debugging system problems.

But yeah, I could see how juniors who don't know the fundamentals might not understand the value.

 

[ambo]

That's all there is for Proxmox here ... homelabs .... no income to gain so not worth it commercially. I'd also love it but there is not enough special software for aarch64 that no other architecture has - besides the Mac eco system. It's still a niche market. Powerful aarch64 hardware is not cheaper than x86-64, so only one disadvantage after another.

There is a major factor which I think has been ignored: The US has placed sanctions on China and many Chinese technology companies. This blocks access to many American technologies - such as Intel and AMD CPUs - for these companies. These tech companies have pivoted - and are now putting all of their efforts behind ARM based hardware and software which is compatible with those platforms.

This hardware isn't yet showing up in massive volumes in places like eBay for home labbers - but enterprise grade hardware certainly exists. Systems like TaiShan are being produced in the thousands (perhaps millions). These are being sold both in Asia - and in many other markets around the world as well. A large percentage (potentially the majority) of services in these regions already runs on top of ARM - whether we realise it or not.

Much of the adoption of ARM is happening "behind the curtain". Hyperscalers (like AWS) are adopting ARM CPUs due to the power efficiencies. Even where end customer adoption of ARM based compute instances may be slow - for services like hosted databases its easy to swap out the underlying hardware without anyone noticing.

 

[Johannes S]

Much of the adoption of ARM is happening "behind the curtain". Hyperscalers (like AWS) are adopting ARM CPUs due to the power efficiencies. Even where end customer adoption of ARM based compute instances may be slow - for services like hosted databases its easy to swap out the underlying hardware without anyone noticing.

But hyperscalers won't use ProxmoxVE but their own KVM/qemu-based virtualization stack.

I agree with your other points though

 

[jlauro]

But hyperscalers won't use ProxmoxVE but their own KVM/qemu-based virtualization stack.

I agree with your other points though

There are some smaller cloud companies that do use proxmox, and probably more that use openshift then proxmox, but I know of at least a couple in the thousands of nodes that publicly stated they are based on it. If they would consider using ARM or not is another question. Granted out of literally hundreds of cloud providers, the top 4 have 60% of the market so the remaining ones are relatively small...

A hybrid of their own stack on top of proxmox is a lot easier/quicker than starting from simply KVM/qemu alone.