NVME and Win10

Valutin

Member
Sep 28, 2020
10
1
8
42
Hello Everybody (first post here), I have a curious problem. I am new to Proxmox, this is the second computer I am building using it, the first one was a server with multiple Win Server 2008 in a bid to do Physical to Virtual migration.

Now, this is the config:
Ryzen 3700X
32GB RAM
2xWD SSD for proxmox in RAID1
Gigabyte AORUS PRO WIFI
2xNvidia Quadro P620 (GPUA/GPUB)
2xM.2 NVME ADATA 512GB (NVMEA/NVMEB)
2xHDD 4TB (HDDA/HDDB)

The objective is to create a 2 users CAD workstation. VMA and VMB will both run Windows 10 and share 1 of whatever is listed below the motherboard.

Phase 1:
I started by building VMB, it seemed to me that it was the easiest as GPUA is used by proxmox at boot. If I attach NVMEB to VMB, windows installation hit a block at reboot, it copies the file but I got a error message at reboot and it asks to reboot over and over. IF I give VMB a qcow2, it's fine, it goes all the way and I can successfully see Windows on a display attached to GPUB.

Phase 2:
Once I knew VMB was sort of working, I started working on VMA. Spent the whole day digging through forum to realize that I was trying to edit the wrong bootloader file... By the end, I had VMA working, both were running with their own GPU.

Phase 3:
Solve VMB not using NVMEB. this is where I am stuck. I wiped the NVMEB, I attached it to VMB, this is what I have in my conf:

Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: cdn
bootdisk: virtio0
cores: 6
cpu: host,hidden=1,flags=+pcid
efidisk0: local-zfs:vm-102-disk-1,size=1M
hostpci0: 0b:00,pcie=1
ide2: ISO:iso/Windows10.iso,media=cdrom,size=4133824K
machine: q35
memory: 14336
name: GDHK02
net0: virtio=A6:8B:B5:D5:94:65,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=9a0608b1-f657-4ee3-b1bd-6882578f2eb9
sockets: 1
vga: qxl
virtio0: /dev/disk/by-id/nvme-ADATA_SX8200PNPSERIALB,size=500107608K
vmgenid: 2dfc476b-c702-4026-9e63-ecb5f21953f8

VMA:
Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: dcn
bootdisk: virtio0
cores: 6
cpu: host,hidden=1,flags=+pcid
efidisk0: local-zfs:vm-101-disk-1,size=1M
hostpci0: 0a:00,pcie=1,romfile=vbios.bin
machine: q35
memory: 14336
name: GDHK01
net0: virtio=7E:A0:25:9B:02:9C,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=8a6327d4-4773-4ce1-95d2-5e710e10bfa3
sockets: 1
vga: qxl
virtio0: /dev/disk/by-id/nvme-ADATA_SX8200PNPSERIALA,size=500107608K
vmgenid: 292f3f00-7c5c-4322-a3b2-acc93c4a4cd4

When VMB boots, VMA will stop and vice versa, if VMA boots, VMB will stop. Both VMs, have their own NVME. If VMB is alone by itself, it will be stuck at the first reboot after copying win 10 files.
It seems that I isolated it to the NVME. Any ideas? is the attach nvme by id the right way to do it?
Thanks for your help.

EDIT:
I have been looking into it since this morning, recreated a VM from scratch and as soon as I boot it, VMA will stop. I might be onto something... I might have reached maximum memory...
 
Last edited:
Hello Everybody (first post here), I have a curious problem. I am new to Proxmox, this is the second computer I am building using it, the first one was a server with multiple Win Server 2008 in a bid to do Physical to Virtual migration.

Now, this is the config:
Ryzen 3700X
32GB RAM
2xWD SSD for proxmox in RAID1
Gigabyte AORUS PRO WIFI
2xNvidia Quadro P620 (GPUA/GPUB)
2xM.2 NVME ADATA 512GB (NVMEA/NVMEB)
2xHDD 4TB (HDDA/HDDB)

The objective is to create a 2 users CAD workstation. VMA and VMB will both run Windows 10 and share 1 of whatever is listed below the motherboard.

Phase 1:
I started by building VMB, it seemed to me that it was the easiest as GPUA is used by proxmox at boot. If I attach NVMEB to VMB, windows installation hit a block at reboot, it copies the file but I got a error message at reboot and it asks to reboot over and over. IF I give VMB a qcow2, it's fine, it goes all the way and I can successfully see Windows on a display attached to GPUB.

Phase 2:
Once I knew VMB was sort of working, I started working on VMA. Spent the whole day digging through forum to realize that I was trying to edit the wrong bootloader file... By the end, I had VMA working, both were running with their own GPU.

Phase 3:
Solve VMB not using NVMEB. this is where I am stuck. I wiped the NVMEB, I attached it to VMB, this is what I have in my conf:

Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: cdn
bootdisk: virtio0
cores: 6
cpu: host,hidden=1,flags=+pcid
efidisk0: local-zfs:vm-102-disk-1,size=1M
hostpci0: 0b:00,pcie=1
ide2: ISO:iso/Windows10.iso,media=cdrom,size=4133824K
machine: q35
memory: 14336
name: GDHK02
net0: virtio=A6:8B:B5:D5:94:65,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=9a0608b1-f657-4ee3-b1bd-6882578f2eb9
sockets: 1
vga: qxl
virtio0: /dev/disk/by-id/nvme-ADATA_SX8200PNPSERIALB,size=500107608K
vmgenid: 2dfc476b-c702-4026-9e63-ecb5f21953f8

VMA:
Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: dcn
bootdisk: virtio0
cores: 6
cpu: host,hidden=1,flags=+pcid
efidisk0: local-zfs:vm-101-disk-1,size=1M
hostpci0: 0a:00,pcie=1,romfile=vbios.bin
machine: q35
memory: 14336
name: GDHK01
net0: virtio=7E:A0:25:9B:02:9C,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=8a6327d4-4773-4ce1-95d2-5e710e10bfa3
sockets: 1
vga: qxl
virtio0: /dev/disk/by-id/nvme-ADATA_SX8200PNPSERIALA,size=500107608K
vmgenid: 292f3f00-7c5c-4322-a3b2-acc93c4a4cd4

When VMB boots, VMA will stop and vice versa, if VMA boots, VMB will stop. Both VMs, have their own NVME. If VMB is alone by itself, it will be stuck at the first reboot after copying win 10 files.
It seems that I isolated it to the NVME. Any ideas? is the attach nvme by id the right way to do it?
Thanks for your help.

EDIT:
I have been looking into it since this morning, recreated a VM from scratch and as soon as I boot it, VMA will stop. I might be onto something... I might have reached maximum memory...
For some reason, the host is using more RAM than I planned. I was thinking of 4(host)+14(VMA)+14(VMB).
I'll read some stuff on how to reduce Proxmox RAM usage. It's using RAM mostly for ZFS as it's serving only 2 VMs and all other drives are passed through.

Again, Thanks for the people who are putting this massive piece of software together.
Onto the next challenge... splitting USB apart between both VMs.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!