incredibly cheap hardware suggestions for performance boost?

[zenowl77]

does anyone have some incredibly cheap hardware suggestions (perferably under $100-120 or so, cheaper the beter) for performance boosts on proxmox?

currently using a few vms/LXCs, using them for jellyfin, docker, gaming, ai models, video encoding and random other tasks over RDP/sunshine&moonlight

setup:

HP Z240 sff pc (do not recommend it was just a cheap sff with multi full length pcie, multi hdd, nvme on the board and 4x ram slots)
i7-6700
64GB DDR4 2133 (4x16GB)
GT 610 2GB
Tesla P4 8gb
1x 1TB NVME on the board (main/important VMs/LXCs) + 1x 256GB NVME via PCIE X1 (VM/LXC swap files/pagefiles)
2x 10TB 3.5in HGST Helium sata HDD XFS+NTFS (VM storage drives, isos, etc + media)
1x 2TB 2.5in sata HDD ZFS (VM backups)
1x 1TB sata SSD LVM (VMs)
2x 8TB sata HDD NTFS in dual bay external enclosure (media backups, other backups, files, etc)
using single on board 1Gbps nic

i currently have one free x1 pcie slot, looking to probably replace the GT 610 either with another tesla P4 or a intel ARC card for av1 encoding (but i have been really debating the arc card seeing as its not great for gaming, doesn't seem to have SR-IOV according to user reports, doesn't have support on all the AI softwares i use, etc so it would be rather expensive purely for the av1 feature... )

i have been looking into VPU/NPU/FPGA devices (seems like they could be useful esp as some are USB, was thinking it could help with video encoding, ai, maybe compression tasks), intel QAT card (was hoping it could help with zfs backups and other things like game installs for the decompression process, but doesnt seem worth it even if they are cheap because of support)

i want to upgrade to a ryzen cpu based system but that would be an expensive replacement at the moment since i would need mobo+cpu/heatsink+psu+case, i have a dell optiplex sff pc with a modded bios for nvme boot support setting around unused with a i3-4150+8gb ram but not sure how i could use that in combination effectively since the high load processes are usually all on one vm/require the GPU and it is a 54w cpu so is a decent amount of power use for the minimal performance it offers. (about 40% of the 6700/65w)

so what are some really cheap hardware suggestions that could give me a good performance boost? (software recommendations too of course are welcome)
it doesnt matter to me how hacky and unconventional it is as long as it works, (i am thinking about getting pcie extenders or something to use extra gpu, nic, more nvmes, etc )

 

[Kingneutron]

2.5Gbit network. Either an Intel-based pcie card or USB3 adapter (more than likely Realtek chipset.) The switches have gotten really cheap lately and it's basically the last gasp for CAT5E cables. The next step is 10Gbit.

Minimize your swap usage and switch to zram. 1-2GB at most. Set swappiness to 1.

Set noatime on all native filesystems and atime=off everywhere on ZFS.

Max out RAM on the motherboard for ZFS caching, add a Special Mirror vdev to your main pool.

https://forum.level1techs.com/t/zfs-metadata-special-device-z/159954

https://klarasystems.com/articles/openzfs-understanding-zfs-vdev-types/

 

[zenowl77]

i was thinking about a 2.5Gbit Ethernet nic but was unsure about whether it would be a significant boost as all my equipment is gigabit and im mostly connecting via laptops and other devices over 800mbps wifi

so far i have been using 32GB swap on the dedicated NVME with the VM pagefiles and swap files, i usually use a LOT of ram with AI tools, etc so i need all the free ram i can keep free, but usually i do end up using too much swap unless AI tools are running.

i have noatime set, and zfs set to metadata caching only as i is only for on one drive purely for backups for the sake of compression

64GB is currently the max for this motherboard and so far i only have the 1TB nvme as my main drive with LVM

im not really using zfs on any of my in use drives since it seems to not have great performance and i only have single disks to work with in such a small setup, i tried it on my 10tb drive but it went blank on me so i switched to XFS on that one, so now ZFS is only in use on my one 2tb 2.5in drive purely for backups

 

[zzz09700]

This is probably not what op wants but looks like you could use a whole round upgrade. The Skylake CPU and everything around it has been pushed to it's limit and honestly I don't see any room for a sensible upgrade.

You need more cores, more ram, more pcies and those sums up to, a new sever built with all technology advancements made in all these years.

At my place we just retired the last few of Haswell/Broadwell based server which came out roughly the same time as Skylake. It's really been a while and it's time to move on man.

 

[zzz09700]

Maybe I should put it this way: if you don't have enough budget for a while round upgrade then it's best to stick with what you have. Any investment on that Skylake machine is likely to be a waste.

 

[zenowl77]

yeah, that is what i am thinking too, i have my eye on switching to a ryzen setup as i said, i think i have pushed it to the limit and beyond in many ways and i wouldn't invest in anything that couldnt be moved to the new setup since it is planned, that would absolutely be a waste of money, but usb devices, pcie cards, etc, all are compatible with a new ryzen setup.

i am just interested in seeing if anyone can point me towards something i haven't thought of or seen yet that could be helpful as i am always looking to go the extra mile and see what i can get out of my setups.

 

[zzz09700]

From my perspective RAM is the primary limiting factor here but 64G is as far as you can go with that MB and Skylake does not support Optane in any so... Basically it's end of road.

Oh the pcie based Optane should work on Skylake if you have pcie lanes to spare.
Our testing results was, Optane was no where close to real memory, but if you don't have any real memory left then it's the best you can have.

And a few years later the "if you don't have any real memory left" scenario became obsolete too as servers with TB sized real memory started flooding the market.

Now a single socket EPYC can have 8 channels and 16 DIMMs with a total of a few TB of real memory, it's like we can ditch the swap stuff for life. If you plan to buy some Optane then take this into account. Our new servers got like 256GB of memory for each CPU socket and that only fills half of the DIMMs.

 

[zenowl77]

yes ram is definitely a limiting factor here, that s also why i am using the nvme for a 32GB swap on top of the 64GB ram, it is reporting 3500MB/s not as good as ram for sure but that isnt too bad at least, optane would probably be better.

cpu is also a huge limit here which is also why i was looking into encoding via the ARC card or Vpu/NPU/FPGA for possibly using them for AI / video encoding as that is the biggest load on the cpu besides games which it is okay at, not great but decent.

i am definitely not in the market for something crazy, although i would love to get something like that but i cant afford much, usually go for old tech when its cheap, but for now i am looking at the ryzen 5600-5800 G or X models they would provide a significant boost plus be DDR4 so i can reuse my current 64GB ram and i can get an asus board with 2x nvme slots on the board and more PCIE slots which will free up more room for upgrades like additional tesla p4's, nics, etc but that will be in the future when i can afford it, hoping maybe the new 8000 series ryzens will knock the prices down a little more because of everyone upgrading trying to sell old stuff fast

 

[zzz09700]

Optane beats reguly NVME SSD by a large margin when used as swap deive bacause it's super fast at qd1 random reads. Ordinary SSD sucks when qd is limited to 1, that's the whole point of Optane.

 

[zzz09700]

Also there should be cheap Broadwell based servers on second hand market ATM. It's technically from the same era as your Skylake machine but most of them sports 8 DDR4 DIMM and 40 PCIe 3.0 lanes. Some even boots with non-ECC memory.

Or you can get a E5v3/v4 real cheap and look for hacky mb from ali-express.

If you have some more budget there's 2nd gen EPYC that's also reaching it's EOL in large hosting enterprises, but the MB for thouse CPUs could be a problem.