proxmox on arm64

Tmanok said:
Awesome! Good job man.

In that same line of thought, the new M1 Pro and M1 Max came out and according to Apple they will be somewhere in the 30,000 on the benchmark score which is 1.5x my Dell XPS 15's i9-11900H. I gotta hand it to Apple, if this keeps up and they release a gen3 M1 MacMini I might be forced to make a Mini-Cluster as my homelab rofl.

Imagine a MacMini with: 64GB DDR5 per node, an absurd amount of GPU and Neural Cores, PCIe Gen4 SSD, and likely 4x Thunderbolt 4 ports. I could build a Ceph cluster with Thunderbolt drive arrays and Thunderbolt to 40Gbps networking and the whole thing could probably fit in a filing cabinet drawer. It's difficult to state how excited I am about these kinds of prospects, I'm starting to really come around to the idea of an ARM PVE within the next 3-5 years.

Oh and don't forget the 200GB/s and 400GB/s memory bandwidth. Heckin' doodle, that's nuts.
Click to expand...
I just want to point out that only 5 months after my comment the new MacMini supports 128GB of RAM at 800GB/s (DDR5 is only 51.2GB/s per DIMM, the Macintosh uses it within the SoC package) and it has 20 CPU cores now... With Asahi Linux natively supporting the M1 lineup, it is only a matter of time before we start seeing fleets of high performance ARM servers rolling out, they are accelerating at a tremendous rate...
 
Last edited:
  • Like
Reactions: kenji and jnoxon
did anyone get the ssd working i have been having issues for an hour at this point im just using one of those usb 3.0 to sata adapters but the drive keeps disapearing when i try to make a ct or vm.
 
Lucio Magini said:

HPE ProLiant RL300 Servers with Ampere Cloud-Native Silicon Launched​

​

https://www.storagereview.com/news/...ers-with-ampere-cloud-native-silicon-launched​

Click to expand...
Its promising, but there's no GA date and even then, no word on availability no REAL word on board or CPU components availability. It could be just a feeler for customer interest. The only OEM to actually ship Ampere bases servers for non operator (eg, google or amazon) use is Gigabyte, and prices are high and showing backorder. IF Ampere is to become a player in the general compute environment (eg, not at cloud providers) it would have to wait until the current economic conditions improve and/or electronic manufacturing supply chains return to normal.
 
Last edited:
Availability

The HPE ProLiant RL300 Gen11 server will be available worldwide for purchase or as-a-service through HPE GreenLake in calendar year Q3 2022, through HPE and HPE’s Channel partners.
 
  • Like
Reactions: Tmanok
Dear all:
I tried to complie pve on the arm64 server from git source(https://git.proxmox.com/), and now I got into trouble about：
1.I can not find proxmox-tfa and proxmox-time package in the git. could anybody tell me where are they?
2.is there any documents about comply pve from source?

thx for your help and best regards.
 
Last edited:
Pimox is a port of Proxmox to the Raspberry Pi allowing you to build a Proxmox cluster of Rapberry Pi's or even a hybrid cluster of Pis and x86 hardware.
https://github.com/pimox/pimox7
I runing pimox7 on my rockpi and tvbox.it works well.
But I have no money to buy kunpeng920 on xianyu app,so I can't provide useful suggestions 。
 
johnny_loo said:
Dear all:
I tried to complie pve on the arm64 server from git source(https://git.proxmox.com/), and now I got into trouble about：
1.I can not find proxmox-tfa and proxmox-time package in the git. could anybody tell me where are they?
2.is there any documents about comply pve from source?

thx for your help and best regards.
Click to expand...
I found them in the proxmox.git。so it's solved。
 
Hi, i'm trying to compile PVE in Kunpeng 920 (arm64) based on debian(bullseye)

I followed the instructions in root user.

1.apt-get install gnupg2 wget –y

2.add Proxmox source（seems not working...）

apt-get install -y wget curl

wget -qO - http://download.proxmox.com/debian/proxmox-release-bullseye.gpg | apt-key add -

echo "deb [arch=arm64] http://download.proxmox.com/debian/pve bullseye pve-no-subscription " | tee /etc/apt/sources.list.d/bullseye-pvetest.list

3.apt update
apt-get update

4.install compile tools/libs

apt-get install -y build-essential
apt-get install -y asciidoc
apt-get install -y binutils
apt-get install -y bzip2
apt-get install -y gawk
apt-get install -y gettext
apt-get install -y git
apt-get install -y libncurses5-dev
apt-get install -y libz-dev
apt-get install -y patch
apt-get install -y unzip
apt-get install -y zlib1g-dev
apt-get install -y libc6-dev
apt-get install -y subversion
apt-get install -y flex
apt-get install -y uglifyjs
apt-get install -y p7zip
apt-get install -y p7zip-full
apt-get install -y msmtp
apt-get install -y libssl-dev texinfo
apt-get install -y libglib2.0-dev
apt-get install -y xmlto
apt-get install -y qemu-utils
apt-get install -y upx
apt-get install -y libelf-dev
apt-get install -y autoconf
apt-get install -y automake libtool
apt-get install -y autopoint
apt-get install -y device-tree-compiler
apt-get install -y screen

apt-get install -y python3-dev
apt-get install -y python3-sphinx
apt-get install -y lintian
apt-get install -y bc
apt-get install -y bison
apt-get install -y libdw-dev
apt-get install -y libiberty-dev
apt-get install -y libnuma-dev
apt-get install -y libslang2-dev
apt-get install -y lz4
apt-get install -y rsync

#the following command does not work.
apt-get install -y libpve-common-perl

apt-get install -y dh-make
apt-get install -y docbook5-xml

apt-get install -y dpkg-dev debhelper
apt-get install -y dh-python

5. git pve kernel code

git clone git://git.proxmox.com/git/pve-kernel.git

cd pve-kernel

6. edit Makefile

in line 23, looks like

#when the ARCH is arm64, set KERNEL_ARCH arm64 too.
KERNEL_ARCH=arm64


7. create the pve-kernel/debian/rules.d/arm64.mk, looks like:

#see also pve-kernel/debian/rules.d/amd64.mk

KERNEL_BUILD_ARCH = arm64
KERNEL_HEADER_ARCH = $(KERNEL_BUILD_ARCH)
KERNEL_BUILD_IMAGE = Image
KERNEL_IMAGE_PATH = arch/$(KERNEL_BUILD_ARCH)/boot/${KERNEL_BUILD_IMAGE}
KERNEL_INSTALL_FILE = vmlinuz

8. change pve-kernel/debian/rules, the "tools" in line "rysnc..." to "source"

rsync --ignore-... scripts tools)...
--to-->
rsync --ignore-... scripts source)...

9. make it
make all


But the make failed, error is:

root@pvearm01:/home/berg/pve-kernel# make all
test -f "submodules/ubuntu-jammy/README" || git submodule update --init submodules/ubuntu-jammy
test -f "submodules/zfsonlinux/Makefile" || git submodule update --init --recursive submodules/zfsonlinux
cd build; dpkg-buildpackage --jobs=auto -b -uc -us
dpkg-buildpackage: info: source package pve-kernel
dpkg-buildpackage: info: source version 5.15.39-4
dpkg-buildpackage: info: source distribution bullseye
dpkg-buildpackage: info: source changed by Proxmox Support Team <support@proxmox.com>
dpkg-buildpackage: info: host architecture arm64
dpkg-source --before-build .
dpkg-checkbuilddeps: error: Unmet build dependencies: libpve-common-perl
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
make: *** [Makefile:61: pve-kernel-5.15.39-4-pve_5.15.39-4_arm64.deb] Error 3


How can i install the libpve-common-perl or should i compile it from source(https://github.com/proxmox-on-arm64/pve-common)?
 
Last edited:
Let's go on...
i compile the libpve-common-perl as follows

#######compile libpve-common-perl########
#set sbin path
PATH=$PATH:/usr/local/sbin:/usr/sbin:/sbin

cd ~
git clone https://github.com/proxmox-on-arm64/pve-common

cd pve-common

#install required perl components
#perl -MCPAN -e shell
#cpan>install Filesys::Df JSON String::ShellQuote Net::IP Linux::Inotify2 Devel::Cycle
Code: 
echo "install Filesys::Df JSON String::ShellQuote Net::IP Linux::Inotify2 Devel::Cycle" | perl -MCPAN -e shell

#install required libs
apt-get install -y libfilesys-df-perl libjson-perl liblinux-inotify2-perl libstring-shellquote-perl

#compile the libpve-common-perl
make all

#install dependencies.
apt-get install -y libcrypt-openssl-random-perl
apt-get install -y libcrypt-openssl-rsa-perl
apt-get install -y libdevel-cycle-perl
apt-get install -y libmime-base32-perl
apt-get install -y libnet-ip-perl

#install generated libpve-common-perl
dpkg -i libpve-common-perl_5.0-38_all.deb

#########################
#now go on compiling the PVE kernel.

And i get the following error...

...
scripts/selinux/mdp/mdp

sent 3,582,643 bytes received 197,274 bytes 2,519,944.67 bytes/sec
total size is 2,905,145 speedup is 0.77
rm -rf /home/berg/pve-kernel/build/ubuntu-jammy_tmp
touch .headers_compile_mark
cp ubuntu-jammy/include/generated/compile.h debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve/include/generated/compile.h
install -m 0644 ubuntu-jammy/Module.symvers debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve
mkdir -p debian/pve-headers-5.15.39-4-pve/lib/modules/5.15.39-4-pve
ln -sf /usr/src/linux-headers-5.15.39-4-pve debian/pve-headers-5.15.39-4-pve/lib/modules/5.15.39-4-pve/build
touch .headers_install_mark
dh_installdocs -A debian/copyright debian/SOURCE
dh_installchangelogs
dh_installman
dh_strip_nondeterminism
dh_compress
dh_fixperms
debian/rules fwcheck abicheck
make[2]: Entering directory '/home/berg/pve-kernel/build'
make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
debian/scripts/find-firmware.pl debian/pve-kernel-5.15.39-4-pve/lib/modules/5.15.39-4-pve >fwlist.tmp
mv fwlist.tmp fwlist-5.15.39-4-pve
checking fwlist for changes since last built firmware package..
if this check fails, add fwlist-5.15.39-4-pve to the pve-firmware repository and upload a new firmware package together with the 5.15.39-4-pve kernel <---What does this mean....?
sort fwlist-previous | uniq > fwlist-previous.sorted
sort fwlist-5.15.39-4-pve | uniq > fwlist-5.15.39-4-pve.sorted
diff -up -N fwlist-previous.sorted fwlist-5.15.39-4-pve.sorted > fwlist.diff
make[2]: *** [debian/rules:278: fwcheck] Error 1
make[2]: Leaving directory '/home/berg/pve-kernel/build'
make[1]: *** [debian/rules:110: binary] Error 2
make[1]: Leaving directory '/home/berg/pve-kernel/build'
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
make: *** [Makefile:61: pve-kernel-5.15.39-4-pve_5.15.39-4_arm64.deb] Error 2


###################after comment the lines 278/279 in debian/rules, i get errors:##################
...
dh_fixperms
debian/rules fwcheck abicheck
make[2]: Entering directory '/home/berg/pve-kernel/build'
make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
checking fwlist for changes since last built firmware package..
if this check fails, add fwlist-5.15.39-4-pve to the pve-firmware repository and upload a new firmware package together with the 5.15.39-4-pve kernel
sort fwlist-previous | uniq > fwlist-previous.sorted
sort fwlist-5.15.39-4-pve | uniq > fwlist-5.15.39-4-pve.sorted
done, no need to rebuild pve-firmware
debian/scripts/abi-generate debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve/Module.symvers abi-5.15.39-4-pve 5.15.39-4-pve
input file 'debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve/Module.symvers' does not exist
make[2]: *** [debian/rules:284: abi-5.15.39-4-pve] Error 2
make[2]: Leaving directory '/home/berg/pve-kernel/build'
make[1]: *** [debian/rules:110: binary] Error 2
make[1]: Leaving directory '/home/berg/pve-kernel/build'
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
make: *** [Makefile:61: pve-kernel-5.15.39-4-pve_5.15.39-4_arm64.deb] Error 2

----------------------
WOW, so many strange errors...
 
Last edited:
iamfoolberg said:
Let's go on...
i compile the libpve-common-perl as follows

#######compile libpve-common-perl########
#set sbin path
PATH=$PATH:/usr/local/sbin:/usr/sbin:/sbin

cd ~
git clone https://github.com/proxmox-on-arm64/pve-common

cd pve-common

#install required perl components
#perl -MCPAN -e shell
#cpan>install Filesys::Df JSON String::ShellQuote Net::IP Linux::Inotify2 Devel::Cycle
Code: 
echo "install Filesys::Df JSON String::ShellQuote Net::IP Linux::Inotify2 Devel::Cycle" | perl -MCPAN -e shell

#install required libs
apt-get install -y libfilesys-df-perl libjson-perl liblinux-inotify2-perl libstring-shellquote-perl

#compile the libpve-common-perl
make all

#install dependencies.
apt-get install -y libcrypt-openssl-random-perl
apt-get install -y libcrypt-openssl-rsa-perl
apt-get install -y libdevel-cycle-perl
apt-get install -y libmime-base32-perl
apt-get install -y libnet-ip-perl

#install generated libpve-common-perl
dpkg -i libpve-common-perl_5.0-38_all.deb

#########################
#now go on compiling the PVE kernel.

And i get the following error...

...
scripts/selinux/mdp/mdp

sent 3,582,643 bytes received 197,274 bytes 2,519,944.67 bytes/sec
total size is 2,905,145 speedup is 0.77
rm -rf /home/berg/pve-kernel/build/ubuntu-jammy_tmp
touch .headers_compile_mark
cp ubuntu-jammy/include/generated/compile.h debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve/include/generated/compile.h
install -m 0644 ubuntu-jammy/Module.symvers debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve
mkdir -p debian/pve-headers-5.15.39-4-pve/lib/modules/5.15.39-4-pve
ln -sf /usr/src/linux-headers-5.15.39-4-pve debian/pve-headers-5.15.39-4-pve/lib/modules/5.15.39-4-pve/build
touch .headers_install_mark
dh_installdocs -A debian/copyright debian/SOURCE
dh_installchangelogs
dh_installman
dh_strip_nondeterminism
dh_compress
dh_fixperms
debian/rules fwcheck abicheck
make[2]: Entering directory '/home/berg/pve-kernel/build'
make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
debian/scripts/find-firmware.pl debian/pve-kernel-5.15.39-4-pve/lib/modules/5.15.39-4-pve >fwlist.tmp
mv fwlist.tmp fwlist-5.15.39-4-pve
checking fwlist for changes since last built firmware package..
if this check fails, add fwlist-5.15.39-4-pve to the pve-firmware repository and upload a new firmware package together with the 5.15.39-4-pve kernel <---What does this mean....?
sort fwlist-previous | uniq > fwlist-previous.sorted
sort fwlist-5.15.39-4-pve | uniq > fwlist-5.15.39-4-pve.sorted
diff -up -N fwlist-previous.sorted fwlist-5.15.39-4-pve.sorted > fwlist.diff
make[2]: *** [debian/rules:278: fwcheck] Error 1
make[2]: Leaving directory '/home/berg/pve-kernel/build'
make[1]: *** [debian/rules:110: binary] Error 2
make[1]: Leaving directory '/home/berg/pve-kernel/build'
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
make: *** [Makefile:61: pve-kernel-5.15.39-4-pve_5.15.39-4_arm64.deb] Error 2


###################after comment the lines 278/279 in debian/rules, i get errors:##################
...
dh_fixperms
debian/rules fwcheck abicheck
make[2]: Entering directory '/home/berg/pve-kernel/build'
make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
checking fwlist for changes since last built firmware package..
if this check fails, add fwlist-5.15.39-4-pve to the pve-firmware repository and upload a new firmware package together with the 5.15.39-4-pve kernel
sort fwlist-previous | uniq > fwlist-previous.sorted
sort fwlist-5.15.39-4-pve | uniq > fwlist-5.15.39-4-pve.sorted
done, no need to rebuild pve-firmware
debian/scripts/abi-generate debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve/Module.symvers abi-5.15.39-4-pve 5.15.39-4-pve
input file 'debian/pve-headers-5.15.39-4-pve/usr/src/linux-headers-5.15.39-4-pve/Module.symvers' does not exist
make[2]: *** [debian/rules:284: abi-5.15.39-4-pve] Error 2
make[2]: Leaving directory '/home/berg/pve-kernel/build'
make[1]: *** [debian/rules:110: binary] Error 2
make[1]: Leaving directory '/home/berg/pve-kernel/build'
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
make: *** [Makefile:61: pve-kernel-5.15.39-4-pve_5.15.39-4_arm64.deb] Error 2

----------------------
WOW, so many strange errors...
Click to expand...
Hi, is there any good news?
 
also trying to do this with cloud.oracle.com with their free 24 Gig RAM 4x arm64 cpu's 200gb data , and having to use a Ubuntu install , to run a virtual machine to install debian so I can try and get proxmox installed onto that , so I have a extra node that will not cost me anything to keep running

currently hitting a wall here as it seems only amd64 distribution files
https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster

apt install proxmox-ve postfix open-iscsi

The following packages have unmet dependencies:
proxmox-ve : Depends: pve-kernel-5.4 but it is not going to be
installed
Depends: pve-manager but it is not installable
Depends: pve-qemu-kvm but it is not installable
Depends: qemu-server but it is not installable
Depends: spiceterm but it is not installable
Depends: vncterm but it is not installable



ended up working out how to upload a debian qcow2 https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-arm64.qcow2
to a storage bucket
https://cloud.oracle.com/object-storage/buckets
then importing as a custom image
https://cloud.oracle.com/compute/images

edit image details to allow VM.Standard.A1.Flex

and using https://github.com/pimox/pimox7
to install proxmox on it
 
Last edited:
MasterCATZ said:
also trying to do this with cloud.oracle.com with their free 24 Gig RAM 4x arm64 cpu's 200gb data , and having to use a Ubuntu install , to run a virtual machine to install debian so I can try and get proxmox installed onto that , so I have a extra node that will not cost me anything to keep running

currently hitting a wall here as it seems only amd64 distribution files
https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster

apt install proxmox-ve postfix open-iscsi

The following packages have unmet dependencies:
proxmox-ve : Depends: pve-kernel-5.4 but it is not going to be
installed
Depends: pve-manager but it is not installable
Depends: pve-qemu-kvm but it is not installable
Depends: qemu-server but it is not installable
Depends: spiceterm but it is not installable
Depends: vncterm but it is not installable



ended up working out how to upload a debian qcow2 https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-arm64.qcow2
to a storage bucket
https://cloud.oracle.com/object-storage/buckets
then importing as a custom image
https://cloud.oracle.com/compute/images

edit image details to allow VM.Standard.A1.Flex

and using https://github.com/pimox/pimox7
to install proxmox on it
Click to expand...
Did you get this working?
I managed to get Debian 11 latest with Arm64 installed with 24 Gig RAM 4x arm64 using their Ampere CPU.
When I start the apt install proxmox-ve
something seems to disconnect the network connection. I can access it using the cloud shell but no longer with SSH.
 
Last edited:
Jarvar said:
Did you get this working?
I managed to get Debian 11 latest with Arm64 installed with 24 Gig RAM 4x arm64 using their Ampere CPU.
When I start the apt install proxmox-ve
something seems to disconnect the network connection. I can access it using the cloud shell but no longer with SSH.
Click to expand...
you need change your ` /etc/network/internet ` first
 
@bingsin
Thanks. Not sure if we're supposed to use our public IP as the static or the internal VCN one like 10.xx.xx.xx
Have you had success?
I tried the public one, but I could be entering the information incorrectly.
For my gateway I entered corresponding to my public IP

gateway 192.168.xxx.1
 
Last edited:
I have just found a distributor in Germany that could ship the HPE ProLiant RL300 Gen11 – with a beefy Ampere M128-30 that would be the dream machine! I just hope Proxmox VE will be soon ready for arm64 and officially support it.
Until then, I am going to hold back any server upgrades in my company. Really hoping for this to come through in 2023.
Thanks for making PVE such a great product and delivering such great support during all those years!
 
  • Like
Reactions: fpausp
Hi, this is my very first post here.

We are a university in France. After a visit to HPE in Paris last week, we are considering buying a bunch of Proliant RL300 for the next academic year. Our main purpose is to provide both ephemeral and persistent GNU/Linux systems to students in the browser.

I am considering Proxmox VE as a layer to ease the administration of our service. We may start by using KVM instead, and switch to Proxmox VE ounce it officially support AMD64. For completeness, we have other problems to solve before that (mainly our software uses user-mode linux wich is only available through emulation on ARM).

I am pretty new to such technical considerations, and I will appreciate any comment and suggestions!
 
You must log in or register to reply here.
Top Bottom