Well how much ram does windows show using in the task manager?could it be baloon driver related?
Then it is down to what your running within the Windows VM.it is same as PVE shows. I am sure if I give more ram, vm will use it very interesting.