Hello everyone,
wanted to share a very positive experience finally getting a HX370 with 64GB of RAM working as I hoped when building it 6 months ago. Major kudos to all the devs involved.
The result is a controlled multi-agentic tool (OpenKIWI) running in an LXC on Proxmox using a v7 customized pve-kernel connected to a Lemonade server also running in an LXC container with NPU and everyting onboard … all headless !!!
The breakthrough was, of course, Lemonade and the Lemonade team for integrating llama.cpp with rocm/vulkan). FastFlowLM added the cherry on the Lemonade cake! http://github.com/lemonade-sdk/lemonade/ and https://github.com/FastFlowLM/FastFlowLM/
However nothing was possible, for my scenario on running it on Proxmox, without jaminmc from https://github.com/jaminmc/pve-kernel that has a custom v7 kernel (now on RC4) that allows you to be ahead of the pack on running clean LXCs on AMDs with NPU activated. One thing, the difference between having an AMD in a container and a NV is insane, only simple passing through is enough !
And finally, not wanting a *claw that runs on its own, a web based agent orchestrator from https://github.com/chrispyers/openkiwi gives you the final touch (including tools, skills and workflows).
As you can see, clean, simple, efficient and it is something no other platform can do by its own. You can spin as many LXCs you want. Finally, it's absurdly low powered, the Lemonade LXC adds 15W idle, OpenKIWI another 10W at idle, and the full stack gets around 80-90W under use.
wanted to share a very positive experience finally getting a HX370 with 64GB of RAM working as I hoped when building it 6 months ago. Major kudos to all the devs involved.
The result is a controlled multi-agentic tool (OpenKIWI) running in an LXC on Proxmox using a v7 customized pve-kernel connected to a Lemonade server also running in an LXC container with NPU and everyting onboard … all headless !!!
The breakthrough was, of course, Lemonade and the Lemonade team for integrating llama.cpp with rocm/vulkan). FastFlowLM added the cherry on the Lemonade cake! http://github.com/lemonade-sdk/lemonade/ and https://github.com/FastFlowLM/FastFlowLM/
However nothing was possible, for my scenario on running it on Proxmox, without jaminmc from https://github.com/jaminmc/pve-kernel that has a custom v7 kernel (now on RC4) that allows you to be ahead of the pack on running clean LXCs on AMDs with NPU activated. One thing, the difference between having an AMD in a container and a NV is insane, only simple passing through is enough !
And finally, not wanting a *claw that runs on its own, a web based agent orchestrator from https://github.com/chrispyers/openkiwi gives you the final touch (including tools, skills and workflows).
As you can see, clean, simple, efficient and it is something no other platform can do by its own. You can spin as many LXCs you want. Finally, it's absurdly low powered, the Lemonade LXC adds 15W idle, OpenKIWI another 10W at idle, and the full stack gets around 80-90W under use.
Attachments
Last edited: