[SOLVED] Dump guest memory

[floh]

Hello guys!

I'm currently working on a paper for hypervisor-based malware detection.
Therefore I need to dump the guests memory.

My first intention was to try using the libvmi python package - but it failed. As I've read proxmox isn't using libvmi.

I've already dumped memory <=4 GB using qm monitor (qmemsave & memsave). The problem: you can't dump bigger memories (32-bit).
Does anyone know how to proper dump the guests memory?

best regards

 

[Chris]

Hi,
just to be clear straight away, I never tried this myself. A quick search resulted in this https://github.com/qemu/qemu/blob/master/scripts/dump-guest-memory.py
Might this be useful for you?

 

[floh]

Hi Chris!

I know the dump-guest-memory [filename] - command in "qm monitor".
Do you possibly know if this script does the same as the built-in 'tool'?

I've found out that the dump-guest-memory command in qm monitor does fail with the following error:
"
ERROR: VM 150 qmp command 'human-monitor-command' failed - got timeout
"

 

[Chris]

Hi Chris!

I know the dump-guest-memory [filename] - command in "qm monitor".
Do you possibly know if this script does the same as the built-in 'tool'?

I've found out that the dump-guest-memory command in qm monitor does fail with the following error:
"
ERROR: VM 150 qmp command 'human-monitor-command' failed - got timeout
"

Using qm monitor will most likely not work, as you see we have timeout ecc. wrapped around commands.
You can try to use the monitor by using a unix socket:
* Extract the qemu command line with all parameters for the VMID qm showcmd VMID
* Append -monitor unix:qemu-monitor-socket,server,nowait and execute it
* Attach to the socket using socat socat -,echo=0,icanon=0 unix-connect:qemu-monitor-socket

 

[floh]

Hi!
I've managed to extract guest-memory.
The problem is that I assume that the file size is x GB (in my case 8GB). BUT it doesn't contain valid data.
The volatility framework (both v2 and v3) doesn't even detect a OS version.

any ideas?

PS: the volatility framework is a framework to analyse memory dumps

 

[Chris]

Hi!
I've managed to extract guest-memory.
The problem is that I assume that the file size is x GB (in my case 8GB). BUT it doesn't contain valid data.
The volatility framework (both v2 and v3) doesn't even detect a OS version.

any ideas?

PS: the volatility framework is a framework to analyse memory dumps

I just tried to dump memory from a guest with dump-guest-memory path-to-dump.img and file path-to-dump.img tells me ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style. So it should work in principle?
How did you dump the memory? Did you maybe compress the dump file via a flag? see: https://qemu.weilnetz.de/doc/qemu-doc.html#pcsys_005fmonitor

 

[floh]

I'm having the same file informations.
When I execute strings <img-file> I can see even many Strings with the word "Windows" in it.
The main problem I have right now is why one of the top memory-analyzing frameworks doesn't work with this dumps.

So I'm currenty testing various things I can find online to check if the memory-dump isn't corrupted or incomplete.
Do you have any further ideas what piece could be missing here?
Have you or your team ever debugged vm-memory?

best wishes
Floh

 

[floh]

I've managed to analyze the memory dumps. The problem was that I hadn't installed the latest profiles to detect the up-to-date win10 version.

So With this method dumping memory works fine and without any problems:
You can try to use the monitor by using a unix socket:

* Extract the qemu command line with all parameters for the VMID qm showcmd VMID
* Append -monitor unix:qemu-monitor-socket,server,nowait and execute it
* Attach to the socket using socat socat -,echo=0,icanon=0 unix-connect:qemu-monitor-socket

One question on top of the possible solution:
Is there a way to add -monitor unix:qemu-monitor-socket,server,nowait to the process (maybe by triggering a migration and appand this snippet to the start-command?

 

[Chris]

I've managed to analyze the memory dumps. The problem was that I hadn't installed the latest profiles to detect the up-to-date win10 version.

So With this method dumping memory works fine and without any problems:
You can try to use the monitor by using a unix socket:



One question on top of the possible solution:
Is there a way to add -monitor unix:qemu-monitor-socket,server,nowait to the process (maybe by triggering a migration and appand this snippet to the start-command?

Glad to hear that it works!
indeed you can do this also by adding and args property to the config of the VM located at /etc/pve/qemu-server/.
Adding something like this should work:

args: -monitor 'unix:/var/run/qemu-server/VMID.monitor,server,nowait'

start the VM and then connect via

socat -,echo=0,icanon=0 unix-connect:/var/run/qemu-server/VMID.monitor

 

[floh]

Thanks Chris for your quick response!

Is there a way/syntax to add a generic line to create a unique socket for every vm so I don't have to hardcode the vmid?

As a quick explanation:
If I understand your line in the config correctly I have to add the following line to the VM 100:
args: -monitor 'unix:/var/run/qemu-server/100.monitor,server,nowait'

And for the VM 123:
args: -monitor 'unix:/var/run/qemu-server/123.monitor,server,nowait'

Is there a generic line for every vmid like:
args: -monitor 'unix:/var/run/qemu-server/VMID.monitor,server,nowait'

And if I have added this one generic line above I can connect to the VM 100 with:

socat -,echo=0,icanon=0 unix-connect:/var/run/qemu-server/100.monitor

same for vm 123:

socat -,echo=0,icanon=0 unix-connect:/var/run/qemu-server/123.monitor

and so on...

best wishes
Flo

 

[Chris]

Well this could in principle be any filename. But by using the VMID you make sure it is unique. A simple shellscript which takes the basename of the config file and places it as VMID should be straight forward.