ARM = Future of Proxmox?

H

harvie

Well-Known Member
Apr 5, 2017
138
23
58
35
I've noticed that ARM based server processors with vast number of cores are starting to appear on the market. (Imagine motherboardboard having four Qualcomm Centriq 2400 CPUs with 48 cores each :) While they might be bit inferior in single thread performance, they might be very well suited for servers with lots of CTs and VMs. Especially given their size and power efficiency.

I think that LXC should be quite ARM compatible, since it does not require any HW support (if it can run Linux, it can probably run LXC). Not sure about KVM, but i think that people were working on this in recent years and there even may be some results already.

I know this is relatively far in the future for Proxmox... But i just wanted to ask if there is will to release ARM version of Proxmox once ARM will be raising in it's popularity.
 
  • Like
Reactions: fireon and Edward Vielmetti
We have already an ARM version of Proxmox VE.

One reason why we did not publish it yet, is the limited availability of suitable server hardware and the use case.
 
  • Like
Reactions: Edward Vielmetti, tschanness and fireon
guletz said:
One hw arm who can use kvm is odroid-c2(4 core, 2 Gb ram, emmc, 7-800 Gbit traffic on network). I have it but I do not have time to test the kvm on it.... shame on me ;)
Click to expand...
Odroid-C2 would be limited by Kernel support. Officially it only supports an older (3.14) kernel and newer kernels have significant issues with USB and Networking (and video, but not so important here). There is a mainline kernel in the works and it might be released with 4.14 soon - but its been "soon" for well over a year now, back to 4.03. Current Proxmox won't play nice with a 3.x kernel.

Perhaps more interesting for Odroid are the hc1 and mc1. These are stripped of frills and designed for clustering. hc1 has single SATA and mount for 2.5 inch drive while mc1 is a cluster-of-4. They support modern kernel (4.09 today, 4.14 imminent). They are arm7 devices though, so 32 bit only.

Of course the C2, HC1 and MC1 are all limited by their very small memory footprint (2GB).

See here:
http://www.hardkernel.com/main/products/prdt_info.php?g_code=G150229074080
http://www.hardkernel.com/main/products/prdt_info.php?g_code=G150152508314

I know this is a Proxmox forum, but I think these types of IOT boards are more suited to Docker Swarm and/or Kubernetes than a full-function KVM/LXC management framework.

Of course they could do really well as a simple, low-cost/low-power quorum node for a 2-node cluster.
 
Last edited:
  • Like
Reactions: Edward Vielmetti and guletz
I'm one of the two devs which bootstrapped PVE on 64 bit ARM, so maybe I can give also some insights here.
At the time it was not straight forward - mainly for the kernels build config, which is a bit hard to get right even now and was worse then.
I'm currently not full on working with it but we use a MP30-AR0 board with an APM X-Gene 1 SoC (8x2.4 Ghz, KVM support, ARMv8) which I periodically update.
One possible nice looking SoC would be the Cavium ThunderX (each 48 cores @ 2.5 GHz, also ARMv8), see https://system76.com/servers/starling
Also https://packet.net already offer them, AFAIK.
Main technical problems are currently the Kernel, it needs some effort to bring one out which supports a lot of SoCs and boards, as a lot of development effort happens in the ARM space this got easier already, especially also with the adoption of PCI(e) as root bus and EFI as firmware (not that UBoot is bad, just EFI is more common and "server people" are used to it).
The second problem is the KVM/QEMU management stack (qemu-server) as we need quite different things form ARM as for amd64, but work toward making this a lot easier is under way.

But no date for an test release known, for Toms and mine reasons stated above.

PigLover said:
I know this is a Proxmox forum, but I think these types of IOT boards are more suited to Docker Swarm and/or Kubernetes than a full-function KVM/LXC management framework.
Click to expand...

I mean, not to say that a cluster of such smaller hardware wouldn't be great for those orchestration tools also LXC, you could surely do nice things with them!
Just LXC is really lightweight (in fact it was the technology used by Docker for a long time, and it wasn't abandoned because it was bad but as they needed other things too and windows support, AFAIK) and if you simple not need the complexity of the orchestration tools and just want to run a few services here and there with the comfort of our HA stack it would be a nice fit too, in my experience.
For KVM i totally agree with you here, on such small devices (in comparison to server hardware) it may seldom make sense to run a full VM there for production use.

PigLover said:
Of course they could do really well as a simple, low-cost/low-power quorum node for a 2-node cluster.
Click to expand...

If you plan this go in the direction of QDevices, you have to do much less (need no multicast, not running instance of your cluster filesystem).
With a Debian Stretch on ARM you should be quite ready to go (corosync-qnetd) would be the package. I hope than I can pick up again my series integrating this in PVE soon, until then one must manually set this up.
 
Last edited:
  • Like
Reactions: Edward Vielmetti, guletz and PigLover
Hi - I manage the Works on Arm program at Packet and would be happy to get Proxmox access to our Arm server cluster for dev/test/CI. We have a working Debian 9 on that system recently and I think the kernel should be new and featureful enough to run Proxmox.

Email contact is ed@packet.net (can't put a link in because of spam filters).
 
  • Like
Reactions: t.lamprecht
What does that mean for running ARM and ARM64 guests on PVE??

(Yes, I would also like to see more then just x86 on Proxmox)
 
stappers said:
What does that mean for running ARM and ARM64 guests on PVE??

(Yes, I would also like to see more then just x86 on Proxmox)
Click to expand...

This itself would mean arm64 VMs and arm/arm64 CT on PVE as a first step.
But to achieve that we need to rework our VM management code a bit, as it's currently quite oriented to the x86 architecture.
Once that is done it should be easier to add other architectures as well, and to add the possibility to run MTTCG (multi thread tiny code generator, the emulator part of QEMU) VMs which may run on any backing hardware, albeit a bit slower - for moving old legacy machines to a virtual environment with all its advantages, or quickly testing something on another architecture, it should be enough.
At the moment only initial work on the code refactoring was started, but this isn't yet up-streamed, the additional archs will probably need quite a bit additional time...
 
I have some Rock64 ARM SBC with 4GB RAM and onboard eMMC storage. I'd be happy to help trying out Proxmox on these devices!
@t.lamprecht any word on the Packet servers? I spoke with them on DockerCon, and they seemed willing to help out open source projects getting ARM support, and told me they have a Slack channel for ARM supporters/devs.
 
Dear tom, t.lamprecht

Hi guys, I saw on the old forum post that we have already an ARM version of Proxmox VE in 2017. Does this image support arm iso now provide testing? My email is 171293193@qq.com, is it convenient to provide your contact method? Thank you
 
I guess ARM is gonna lose some popularity, because RISC-V is all the rage now...
 
yangsm said:
Hi guys, I saw on the old forum post that we have already an ARM version of Proxmox VE in 2017. Does this image support arm iso now provide testing? My email is 171293193@qq.com, is it convenient to provide your contact method? Thank you
Click to expand...

We currently paused the effort a bit as there's no viable modern hardware which "normal" companies (as in not multi billion dollar companies) buy on the market. That said, all projects should build and the experimental integration in our qemu/lxc management is still there - so you always could install a Debian and build the packages yourself if you want to explore/advance this.

harvie said:
I guess ARM is gonna lose some popularity, because RISC-V is all the rage now...
Click to expand...

Yeah, the moment a viable CPU (board) is out there with the virtualization HW extension support and a reasonable performance I'd be all over it. The proprietary nature was always a major disadvantage of ARM, IMO, you see it in mobile and I saw it on every board we tried to touch. Stone-age old custom kernels with various BLOBs, once you ask some normal information (i.e., how to really boot that fricking thing) strong NDA's one with slight moral can never accept will get thrown ones way, etc., etc. PITA to do things.
That's a totally personal opinion, no official stance from us as Proxmox devs, but if there's no major change and readjustment I do not think that ARM will capture any bigger chunk of the server or workstation market ever, besides some special purpose deployments. POWER and MIPS got that, and try to be a lot more open. Let's see what the next few years bring..
 
Hi
@t.lamprecht Sorry to directly ping you here, but as this specifically targets you and your work on arm.

Has this topic ever been picked up again?

With the rising of things like RaspberryPI with more than 4GB RAM, I think it's quite an interesting topic again, to run really low power SBC-Clusters.

Thanks for your reply!
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back