[SOLVED] ARM64 VM emulation on Proxmox 5.3 AMD64

Mikus

New Member
Apr 13, 2018
18
3
3
43
Hello everyone! Proxmox 5.3 added support for emulating ARM virtual machines (experimental). Anybody tried to create and run ARM-based VM using latest Proxmox 5.3? If so, can you share VM configuration file? Thanks in advance!
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
4,089
369
88
31
Vienna
i believe the only 2 parameters necessary are
Code:
arch: aarch64
bios: ovmf
 
  • Like
Reactions: Mikus

Dark26

Member
Nov 27, 2017
103
6
23
42
If someone can test this option. Perhaps next week i can test this with my pine board.
 

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
3,185
273
88
If someone can test this option. Perhaps next week i can test this with my pine board.
AFAIU the OP wants to emulate ARM (as a VM) on PVE (amd64), not the PVE ARM build.
 

Dark26

Member
Nov 27, 2017
103
6
23
42
AFAIU the OP wants to emulate ARM (as a VM) on PVE (amd64), not the PVE ARM build.
I know.

The question is to know if we can create directly a arm machine on the proxmox amd64 hypervisor, or if we have to import an existing image who was create on a arm hardware.

Actually i have a lxc container who run on my pineH64, and i want to know if this lxc image can run on my proxmox amd64

Dark26
 

Mikus

New Member
Apr 13, 2018
18
3
3
43
Tried with no success so far to install Debian on ARM64 VM. Debian iso used: debian-9.6.0-arm64-netinst.iso

Install GUI is launched, but after option "Install" selection system hangs:

EFI stub: Booting Linux Kernel
EFI stub: EFI_RNG_PROTOCOL unavailable, no randomness supplied
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...

.. and nothing happens ...

Code:
arch: aarch64
bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 1
efidisk0: local-lvm:vm-110-disk-1,size=64M
memory: 2048
name: DebianARM64
net0: virtio=00:00:00:00:00:00,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-110-disk-0,size=32G
scsi2: local:iso/debian-9.6.0-arm64-netinst.iso,media=cdrom,size=203M
scsihw: virtio-scsi-pci
smbios1: uuid=ffffffff-ffff-ffff-ffff-ffffffffffff
sockets: 1
Code:
 /usr/bin/qemu-system-aarch64 -id 110 -name DebianARM64 -chardev socket,id=qmp,path=/var/run/qemu-server/110.qmp,server,nowait -mon chardev=qmp,mode=control -chardev socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5 -mon chardev=qmp-event,mode=control -pidfile /var/run/qemu-server/110.pid -daemonize -smbios type=1,uuid=fde92152-18ca-4323-ba50-72ee4472bcf6 -drive if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//AAVMF_CODE.fd -drive if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/pve/vm-110-disk-1 -smp 1,sockets=1,cores=1,maxcpus=1 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg -vnc unix:/var/run/qemu-server/110.vnc,x509,password -cpu cortex-a57 -m 2048 -device pci-bridge,id=pci.2,chassis_nr=2,bus=pcie.0,addr=0x1f -device pci-bridge,id=pci.1,chassis_nr=1,bus=pcie.0,addr=0x1e -device usb-ehci,id=ehci,bus=pcie.0,addr=0x1 -device usb-tablet,id=tablet,bus=ehci.0,port=1 -device usb-kbd,id=keyboard,bus=ehci.0,port=2 -device virtio-gpu,id=vga,bus=pcie.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:4a918acfd3ef -device virtio-scsi-pci,id=scsihw0,bus=pcie.0,addr=0x5 -drive file=/dev/pve/vm-110-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on -device scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200 -drive file=/var/lib/vz/template/iso/debian-9.6.0-arm64-netinst.iso,if=none,id=drive-scsi2,media=cdrom,aio=threads -device scsi-cd,bus=scsihw0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2,bootindex=100 -netdev type=tap,id=net0,ifname=tap110i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown -device virtio-net-pci,mac=D6:A7:87:68:EC:26,netdev=net0,bus=pcie.0,addr=0x12,id=net0,bootindex=300 -machine accel=tcg,type=virt
 
Last edited:

Dark26

Member
Nov 27, 2017
103
6
23
42
i was thinking, if it 's work some day, it's only with VM, not containers ?
 

Mikus

New Member
Apr 13, 2018
18
3
3
43
Maybe, it will work some day ;) Yesterday I also tried to install Ubuntu 18.04 in the same way, problem persists: after selecting option in Grub system hangs.

However last night after some goooogling I was able to launch Debian installer on ARM64 VM via commandline using qemu-system-aarch64 after bootloader (linux-deb-9-6) and initrd of the Debian installer (initrd.gz) extraction:

Code:
qemu-system-aarch64 -smp 2 -M virt -cpu cortex-a57 -m 1G \
    -initrd /var/lib/vz/template/iso/initrd.gz \
    -kernel /var/lib/vz/template/iso/linux-deb-9-6 -append "root=/dev/ram console=ttyAMA0" \
    -global virtio-blk-device.scsi=off \
    -device virtio-scsi-device,id=scsi \
    -machine accel=tcg,type=virt \
    -netdev user,id=unet -device virtio-net-device,netdev=unet \
    -net user \
    -nographic
 

Mikus

New Member
Apr 13, 2018
18
3
3
43
Tried with no success so far to install Debian on ARM64 VM. Debian iso used: debian-9.6.0-arm64-netinst.iso

Install GUI is launched, but after option "Install" selection system hangs:

EFI stub: Booting Linux Kernel
EFI stub: EFI_RNG_PROTOCOL unavailable, no randomness supplied
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...

.. and nothing happens ...

Code:
arch: aarch64
bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 1
efidisk0: local-lvm:vm-110-disk-1,size=64M
memory: 2048
name: DebianARM64
net0: virtio=00:00:00:00:00:00,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-110-disk-0,size=32G
scsi2: local:iso/debian-9.6.0-arm64-netinst.iso,media=cdrom,size=203M
scsihw: virtio-scsi-pci
smbios1: uuid=ffffffff-ffff-ffff-ffff-ffffffffffff
sockets: 1
Code:
 /usr/bin/qemu-system-aarch64 -id 110 -name DebianARM64 -chardev socket,id=qmp,path=/var/run/qemu-server/110.qmp,server,nowait -mon chardev=qmp,mode=control -chardev socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5 -mon chardev=qmp-event,mode=control -pidfile /var/run/qemu-server/110.pid -daemonize -smbios type=1,uuid=fde92152-18ca-4323-ba50-72ee4472bcf6 -drive if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//AAVMF_CODE.fd -drive if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/pve/vm-110-disk-1 -smp 1,sockets=1,cores=1,maxcpus=1 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg -vnc unix:/var/run/qemu-server/110.vnc,x509,password -cpu cortex-a57 -m 2048 -device pci-bridge,id=pci.2,chassis_nr=2,bus=pcie.0,addr=0x1f -device pci-bridge,id=pci.1,chassis_nr=1,bus=pcie.0,addr=0x1e -device usb-ehci,id=ehci,bus=pcie.0,addr=0x1 -device usb-tablet,id=tablet,bus=ehci.0,port=1 -device usb-kbd,id=keyboard,bus=ehci.0,port=2 -device virtio-gpu,id=vga,bus=pcie.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:4a918acfd3ef -device virtio-scsi-pci,id=scsihw0,bus=pcie.0,addr=0x5 -drive file=/dev/pve/vm-110-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on -device scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=200 -drive file=/var/lib/vz/template/iso/debian-9.6.0-arm64-netinst.iso,if=none,id=drive-scsi2,media=cdrom,aio=threads -device scsi-cd,bus=scsihw0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi2,id=scsi2,bootindex=100 -netdev type=tap,id=net0,ifname=tap110i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown -device virtio-net-pci,mac=D6:A7:87:68:EC:26,netdev=net0,bus=pcie.0,addr=0x12,id=net0,bootindex=300 -machine accel=tcg,type=virt
Additional observations running (actually not running ;) ) this configuration: ARM64 VM's Bios reports 0 MB RAM o_O
 
Last edited:

wbumiller

Proxmox Staff Member
Staff member
Jun 23, 2015
647
88
48
Tried with no success so far to install Debian on ARM64 VM. Debian iso used: debian-9.6.0-arm64-netinst.iso

Install GUI is launched, but after option "Install" selection system hangs:

EFI stub: Booting Linux Kernel
EFI stub: EFI_RNG_PROTOCOL unavailable, no randomness supplied
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...

.. and nothing happens ...
Almost none of the installer ISOs come with graphics drivers. You need to add a serial socket, and connect to that via the xterm.js button.
After installation, when using 'virtio' graphics, you'll be able to see the machine via novnc as well.
 
  • Like
Reactions: Mikus

Mikus

New Member
Apr 13, 2018
18
3
3
43
Almost none of the installer ISOs come with graphics drivers. You need to add a serial socket, and connect to that via the xterm.js button.
After installation, when using 'virtio' graphics, you'll be able to see the machine via novnc as well.
Thank you Wolfgang, very, very useful and helpful! I connected serial console but was in stuck with xterm console message:
Code:
starting serial terminal on interface serial0
I needed only to press enter (select default "Install" option from the grub menu in "background") after this message to start installation.

So, in the end I was able to install and run and play with Debian ARM64 VM on Proxmox 5.3, this new feature works out of the box, thank you for support once more!

Code:
arch: aarch64
bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 1
efidisk0: local-lvm:vm-110-disk-1,size=64M
memory: 2048
name: DebianARM64
net0: virtio=00:00:00:00:00:00,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-110-disk-0,size=32G
scsi1: local:iso/debian-9.6.0-arm64-netinst.iso,media=cdrom,size=203M
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=ffffffff-ffff-ffff-ffff-ffffffffffff
sockets: 1
vga: serial0
Code:
/usr/bin/qemu-system-aarch64
-id 110
-name DebianARM64
-chardev socket,id=qmp,path=/var/run/qemu-server/110.qmp,server,nowait
-mon chardev=qmp,mode=control
-chardev socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5
-mon chardev=qmp-event,mode=control
-pidfile /var/run/qemu-server/110.pid
-daemonize
-smbios type=1,uuid=ffffffff-ffff-ffff-ffff-ffffffffffff
-drive if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//AAVMF_CODE.fd
-drive if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/pve/vm-110-disk-1
-smp 1,sockets=1,cores=1,maxcpus=1
-nodefaults
-boot menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg
-nographic
-cpu cortex-a57
-m 2048
-device pci-bridge,id=pci.2,chassis_nr=2,bus=pcie.0,addr=0x1f
-device pci-bridge,id=pci.1,chassis_nr=1,bus=pcie.0,addr=0x1e
-device usb-ehci,id=ehci,bus=pcie.0,addr=0x1
-chardev socket,id=serial0,path=/var/run/qemu-server/110.serial0,server,nowait
-serial chardev:serial0
-chardev socket,path=/var/run/qemu-server/110.qga,server,nowait,id=qga0
-device virtio-serial,id=qga0,bus=pcie.0,addr=0x8
-device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0
-device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x3
-iscsi initiator-name=iqn.1993-08.org.debian:01:4a918acfd3ef
-device virtio-scsi-pci,id=scsihw0,bus=pcie.0,addr=0x5
-drive file=/dev/pve/vm-110-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on
-device scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100
-drive file=/var/lib/vz/template/iso/debian-9.6.0-arm64-netinst.iso,if=none,id=drive-scsi1,media=cdrom,aio=threads
-device scsi-cd,bus=scsihw0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,bootindex=200
-netdev type=tap,id=net0,ifname=tap110i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown
-device virtio-net-pci,mac=00:00:00:00:00:00,netdev=net0,bus=pcie.0,addr=0x12,id=net0,bootindex=300
-machine accel=tcg,type=virt
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!