Old Quadra 2000 Card - Windows 10 - Proxmox 6.1-7

[smorvan]

Hello All,

I am struggling to get the aformentionned card running in passthrough mode.

I read and configured as indicated on the relevant page , compiled romparser and dumped the rom to the relevant folder (I am running the VM as UEFI).

I got to run in passthrough mode only once, and I am guessing that Windows ended up reinstalling the driver on its own (I spotted was a warning in device manager indicating that the driver had to be reinstalled shortly before that). This occurred right after I added the rom file for the card.

I can disable to the passthrough by commenting hostpci0 in my vm .conf file, and I did setup the RDP. My issue atm is that I can't see what is going on into the system because neither console, nor RDP or even directly plugged in the board will work. I can not reinstall the correct Nvidia Driver because at the time I can log into the system the Nvidia card is disabled.

I did let the machine sit and reboot a couple of times to let it install its updates, which was completed before I attempted the passthrough.

The host is a Dell Precision 3600 w/ 32 Gb RAM and Xeon 2670v1 - not a monster, but it should work for my purpose. The card is an Nvidia Quadro 2000.

Here is my VM .conf file:

agent: 1
bios: ovmf
bootdisk: scsi0
cores: 8
cpu: host,hidden=1
efidisk0: local-lvm:vm-100-disk-1,size=4M
hostpci0: 04:00,romfile=Quadro2000.rom,rombar=on,x-vga=on,pcie=on
ide0: local:iso/virtio-win-0.1.173.iso,media=cdrom,size=384670K
ide2: local:iso/Win10_1909_English_x64.iso,media=cdrom
machine: q35
memory: 12384
name: Windows10
net0: virtio=2E:B9:CC:23:2E3,bridge=vmbr0
numa: 0
ostype: win10
scsi0: local-lvm:vm-100-disk-0,cache=writeback,size=60G
scsihw: virtio-scsi-pci
smbios1: uuid=fd2ac302-d4bf-40b2-ab67-c02856592c14
sockets: 1
vmgenid: 2a3cda49-0df4-491c-8a0e-b6bbe981cdb5


I also tried the hv_hidden flag, but from what I read, this is now taken care of when the OS Type is set to 'Microsoft Windows10/2019/2019' is specified in the machine profile.

Any hints so what I could be doing wrong? Alternatively, how can I see what is going inside the VM?

TIA!

 

[smorvan]

So I will answer to my own thread.

TL,DR: I was finally able to boot up this machine by creating a new VM (could not keep the old one) and using Seabios. I also had to disable x-vga - on because it was not recognised at launch time.

Additional notes:

1. I did the install without (or with?) the hostpci0 line commented out, and proceeded to activate RDP once I installed the VirtIO utilities.
2. I disabled Windows update from the first boot.
3. Then uncommented the hostpci0 line and rebooted. The graphics card was recognised, and I downloaded the drivers from the Nvidia web site, which installed without a hitch.
4. I then reenabled Windows update and let the machine do its things.
5. I also passed on the Quadra2000.rom bios + rombar. This is what my vm.conf file looks like (it would not boot otherwise)

Performance seems to be ok for this grand daddy of all Nvidia cards.

agent: 1
bootdisk: scsi0
cores: 8
hostpci0: 04:00,pcie=1,romfile=Quadro2000.rom,rombar=0
ide2: local:iso/Win10_1909_English_x64.iso,media=cdrom
machine: q35
memory: 16000
net0: virtio=6A:08:80:71:E4:78,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
sata0: local:iso/virtio-win-0.1.173.iso,media=cdrom,size=384670K
scsi0: local-lvm:vm-100-disk-0,cache=writeback,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=308b6fe7-4457-4b9b-acdb-985c7771faa0
sockets: 1
vmgenid: a6f27f10-01d7-4c70-a587-b178e83b1500

Everything else is pretty much standard according to the Passthrough Wiki. I got a black screen in my RDP that I could not recover from, but reconnecting after closing the first session brought the machine back to life.

I am now going to see how I can make this machine a bit more responsive because it really does feel sluggish.

Edit: Ended up with a black screen after Sysprep'ing the image (while keeping drivers). So not sure if cloning is a viable. As for performance, I moved to an SSD and it helped greatly.

 

[smorvan]

Soooo... Responding to my own thread so that it can be of some use (or when I will eventually mess up by VM beyond repair):

Turning the previous image into a template did not work after additional trial (sysprep and all). Basically, when the clone boots up the C: drive is not seen. With the Windows ISO, I can get to a shell and manually load the VirtioSCSI Driver, which mounts the C: drive but it is empty.

So, best option is to clone the existing image and then change computer name.

I was able to install a SolidWorks Pro 2018 SP5 with a legit online license + PDM Proffessional. Everything is so much faster than on my former VM (MacBook Pro Retina / Parallels Desktop - I know).

I was also able to install my 3D Mouse (SpaceMouse Wireless) directly from the Proxmox GUI. The USB Dongle was immediately recognised. I will start using the system and see how well it does, but so far I am a happy camper.

I turned on the following settings in this WindowsVM (from this post), not sure it has any effect with such an old card:

Computer Configuration -> Administrative Templates -> Windows Components -> Remote Desktop Services -> Remote Desktop Session Host -> Remote Session Environment

Set the following policies to Enable, to enable the use of the AVC/H.264 codec :

• Configure H.264/AVC hardware encoding for Remote Desktop connections
• Prioritize H.264/AVC 444 Graphics mode for Remote Desktop connections
• Use the hardware default graphics adapter for all Remote Desktop Services sessions

 

[smorvan]

Initially, I was not too familiar with zfs, so I went ahead and formatted the hdd as an ext4 filesystem. Turns out zfs is much much more resilient, so I decided to go for zfs. Only sensible way to go from ext4 to zfs, after asking on reddit, is to backup VMs and reinstall Proxmox from scratch. The process went like this:

• Backed up all VMs to a spare USB Hard Drive,
• Backed up all the files that I had tweaked (tweaked according to this guide),
• Reinstalled Proxmox from scratch and chose zfs standard settings during install,
• Restored VMs via qmrestore (I had to change the storage disk, since local-lvm has been replaced by local-zfs with --storage local-zfs),
• Only had to disable the CDRoms in the VMs (I was still mounting Windows CD and VirtIO Drivers, but the images had not been uploaded to Proxmox)

Then I bought two 100 Gb Intel SSDs S3700 on eBay to use as ZIL and L2ARC Cache and:

• Formatted as two partitions (via fdisk):
  • /dev/sdb1 (25 Gb) for ZIL
  • /dev/sdb2 (remainder) for L2ARC
• Added ZIL with zpool add rpool log /dev/sdb1
• Added L2ARC cache with zpool add rpool cache /dev/sdb2

Speed went (measured in a shell, while one VM was running from hdd) from 60 Mb/s to 170 Mb/s.

Also added an old Cyberpower UPS connected via USB (installed apcupsd to trigger shutdown). Thanks APC for supporting other brands with this package, guess you know the brand of my next UPS.

So far, so good. I will migrate the VM to the other SSDs (I have 3 total) to get a bit more speed once I received the SATA Cables I ordered from Amazon. Next step is to add 2 more Quadro Gfx Cards to let the two VMs to Run concurrently (evidently, you can only passthrough one card at a time so this is not possible with my existing setup).

 

[smorvan]

So, after a while, I realised that the Quadro driver would no longer load in Windows, after the first few reboots. Upon closer examination in the Device Manager, there was an IRQ Conflict. After looking for a while, I was almost ready to give up and install bare metal when someone suggested in a related topic to unplug the monitor (so nothing in any of the card's DVI / Displayport / HDMI port), and voila... Super fast machine.

Also changing cpu from 'host' to 'qemu64' sped things up substantially.

 

[smorvan]

Heads-up for anyone using a Quadro K620: It will not work in Bios (at least it did not on my Precision T7810 or T3600). You need to use UEFI.

I tried every possible combination of options on the cmd line (x-vga was a crapshoot, rombar did not change a thing neither did passing the rom that I had dumped). I would basically get a code 12 within Windows (resource conflict), regardless of the PCI Slot used when booting with SeaBIOS.

A telltale sign that something was wrong is that I could never get Kernel driver in use: vfio-pci for the K620. The 2000 and M4000 registered just fine.

I had to reconfigure a VM with OVMF (UEFI) boot, then install the Nvidia Drivers. Then I had a code 43. Added x-vga=1 to the vm .conf file did the trick... The options looks (quite deceivingly) like this:

hostpci0: 04:00,pcie=1,x-vga=1

While the option for the Quadro 2000 looked like this (VM is booting from SeaBIOS):

hostpci0: 04:00,pcie=1,rombar=0,romfile=Quadro2000.rom

The same line worked for a Quadro M4000 (VM is booting fine from SeaBIOS):

hostpci0: 04:00,pcie=1,rombar=0,romfile=QuadroM4000.rom

This kernel booting option also helped (host is booting in UEFI in this case):

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on video=vesafbiff video=efifbff

FYI, Solidworks + 3DMouse runs like a charm. Much better than on my MacOS X VM. No tweaks on the Windows side were required.

 

[Karbadan_Barbadan]

can you pass me your Vbios for M2000, I am going through the same path with having the card in vWin10 running, but not loading itself, not encoding, and generally not wanting to shine out if there no other GPU added, so I added a standard video adapter, and tried to give all the load through nvidia manager, but again nothing in SolidWorks, youtube etc. Then suddenly I made a test with AutoCAD 2024 and!!! It got loaded?!? So, it is halfly working, thus I need the exact working config and scenario