Privacy of multiple users on one Proxmox machine

[proxenjoy]

Dear Proxmox community,

I have created a separate pool of resources on my proxmox box that I would like to offer to one of my friends, s.t they can create their own VM's on my proxmox box. Now, what I did to create that is the following

1. Create a pool of resources i.e storage and ISO images
2. Create a user account for my friend
3. Give my friend access to that pool with the role set to PVE admin

Now, my question is, is there a possibility that the owner of the proxmox box i.e I do not have access to the data my friend is storing on his VM's. I realize that I can see that he created VM's but I cannot access his VM's for instance since I don't have access to the login credentials of these VM's. However, the data that my friend stores on his VM's is still stored on my hardware. Now, the question is, is it in theory possible for me as the owner to have access to that data ? or I can only have access to that data if I know the login credentials of the VM's of my friend. If that is possible, is there then a way that I can turn off this functionality such that I can proof to my friend that I don't have access to his data ?

Thank you guysss!!!

 

[meyergru]

Of course you have access to the storage of those VMs. The virtual disks are just files or volumes on your hardware and you can transfer them to any other VM (including such that you own). Thus, you can access these disks on VMs that you know the credentials for.

In theory, you friend could encrypt his volumes, which would make accessing them harder but not 100% secure.

 

[gfngfn256]

Your situation is exactly the same as if your friend's bare metal computer is in YOUR possession. So even if the main OS that boots on his computer requires a login at boot, this does not stop you from booting your own OS & then directly reading the data from his drives. The drives could be encrypted to offer some form of protection.

TLDR: Yes you have complete access to his data.

 

[proxenjoy]

Of course you have access to the storage of those VMs. The virtual disks are just files or volumes on your hardware and you can transfer them to any other VM (including such that you own). Thus, you can access these disks on VMs that you know the credentials for.

In theory, you friend could encrypt his volumes, which would make accessing them harder but not 100% secure.

Thank you for your fast reply, but I said in the post that I don't have the credentials to his VM's. If my friend starts an Ubuntu VM for instance then I cannot access his Ubuntu VM since I don't have his login credentials of his Ubuntu VM

 

[meyergru]

Wrong: As I said, you can always mount his disks to your VMs - to which you obviously have the credentials.

In other words: Create a Ubuntu VM yourself, assign a copy of his disks to that VM - et voilà! No need to know his credentials...

 

[proxenjoy]

Your situation is exactly the same as if your friend's bare metal computer is in YOUR possession. So even if the main OS that boots on his computer requires a login at boot, this does not stop you from booting your own OS & then directly reading the data from his drives. The drives could be encrypted to offer some form of protection.

TLDR: Yes you have complete access to his

Wrong: As I said, you can always mount his disks to your VMs - to which you obviously have the credentials.

In other words: Create a Ubuntu VM yourself, assign a copy of his disks to that VM - et voilà! No need to know his credentials...

Thank you!! I will see if I can reproduce this scenario by myself. Do you maybe know a solution another solution such that a friend of mine has a server in my house bare metal or virtually on my machine to store his VMN' s.t I won't have access to them ? thank you...

 

[proxenjoy]

Thank you for your reply

 

[meyergru]

As I said, you cannot secure this 100%, because you can control the VM "hardware".

It also depends a bit on the application: If the machine is being used as a backup target, your friend could encrypt the data - preferably on the source. This is much like encrypted cloud storage where the storage provider cannot look into the encrypted data.

If you want to use something like rsync or dirvish, this is not possible, because you have to process the data on the VM itself. Your friend would have to encrypt the whole data disk (or partition), e.g. with LUKS in this case.

He could then mount the disk (or partition) via ssh from his side, such that the encryption password is not stored on the VM. In that case, the whole data disk (or partition) would be encrypted by a key that you do not know and cannot extract directly. However, I sill could imagine different ways to intercept that encryption key (like manipulating the VM, for instance).

 

[gfngfn256]

What do you mean here ? If he installs a a bare metal in my house and he installs an OS like proxmox on it, then I cannot access his data right ? If I would install another OS on it then I don't have access to these folders right ?

You fail to understand what is being quite simply explained to you. I'll try again:

Normal bare-metal installations refer to a dedicated OS being installed/run directly on/to a PC's hardware. A hypervisor (like PVE) is simply an environment that runs other virtual machines within it. So for example a person could choose to install a Debian OS directly bare-metal on his PC. Or he could install the same Debian OS in a VM within a hypervisor environment (like PVE). In BOTH cases his data which is stored (usually) on local disks, is readily accessible to anyone reading these disks. Take for example the above bare-metal Debian install, even if the OS natively when booting it up requires a login/password, if you boot that PC up with for example, your own Debian USB stick, you'll be able to directly read all of his data on those local disks. The same goes for a VM running within PVE; all the virtual "local disks" of that VM are readily available to anyone with access to that PVE server. I often recover data for clients from their non functional PCs, by simply imaging the disks & reading them on some other HW. I've often gone further, & created a VM with a similar config to their original (bare-metal) HW, and then attach those images to the VM, which I can then run as if it were their old PC.

In short if the data isn't inherently encrypted on the disk - it is most certainly EASILY readable to anyone with physical access to that disk, be it a physical disk or a virtual one.

 

[proxenjoy]

Thank you both meyergru and gfngfn256 for your explanations. I was both able to reproduce your arguments in practice and now also understand how to do this. I first needed to see how it is done, for me to understand it. Luckily enough I found that many applications such as Nextcloud, Joplin or a matrix server all have end to end encryption capacities. To my understanding then, I won't be able to read my friends data. Like you guys said you need to have applications running in VM's that encrypt on disk. In that way a person accessing your drives or VM drives can't do anything with it.

 

[meyergru]

No, not "applications in the VM". That is the less desirable approach, because with a manipulated VM, you as the hardware owner can still intercept the data before it is encrypted or even grab the key itself.

The only secure way is if the client encrypts the data before it even enters your realm. Only is it that way that your friend keeps full control over his data without any need of trusting you.

Of course, that leaves such "applications" as mere flow heaters for data - they do not encrypt anything themselves. This will also keep them from doing anything intelligent, like deduplicating data on a logical level (e.g. rsync, dircish). You could only deduplicate on a block level, depending on how the encryption is done on the client.

 

[proxenjoy]

No, not "applications in the VM". That is the less desirable approach, because with a manipulated VM, you as the hardware owner can still intercept the data before it is encrypted or even grab the key itself.

The only secure way is if the client encrypts the data before it even enters your realm. Only is it that way that your friend keeps full control over his data without any need of trusting you.

Of course, that leaves such "applications" as mere flow heaters for data - they do not encrypt anything themselves. This will also keep them from doing anything intelligent, like deduplicating data on a logical level (e.g. rsync, dircish). You could only deduplicate on a block level, depending on how the encryption is done on the client.

Yess, encrypting the data before sending it to the server is done on the client side with Matrix, Joplin and Nextcloud for instance. That's why I mentioned these.

 

[meyergru]

For example, with NextCloud, that is not the default. It usually uses server-side encryption.

Only since 3.0, you can get a desktop client which can encrypt per folder on the client. Same story with Joplin and Matrix - they call it "end-to-end encryption".

 

[proxenjoy]

For example, with NextCloud, that is not the default. It usually uses server-side encryption.

Only since 3.0, you can get a desktop client which can encrypt per folder on the client. Same story with Joplin and Matrix - they call it "end-to-end encryption".

Yeahhh thank you, so if your application has end to end encryption you are good to go!! That's it basically

 

[gfngfn256]

The closest you'll get is if the client fully encrypts his disks. This also isn't really secure for 3 reasons:

1. You are still in "possession" of his encrypted disks with the possibility of "cracking" that encryption.
2. The client's VM will use a decryption key when booting up - so technically you can read his RAM dumps & maybe able to find the keys used.
3. You will also be able to monitor the RAM dumps during his system operation/usage - so you'll be able to grab "activity" of his there.

In short, with a friend running a VM on your hardware - you can absolutely monitor what is going on.

 

[proxenjoy]

The closest you'll get is if the client fully encrypts his disks. This also isn't really secure for 3 reasons:

1. You are still in "possession" of his encrypted disks with the possibility of "cracking" that encryption.
2. The client's VM will use a decryption key when booting up - so technically you can read his RAM dumps & maybe able to find the keys used.
3. You will also be able to monitor the RAM dumps during his system operation/usage - so you'll be able to grab "activity" of his there.

In short, with a friend running a VM on your hardware - you can absolutely monitor what is going on.

Than you for your insights gfngfn256. So the only thing you as the owner can still see is the metadata then right. But from the metadata and RAM dumps you cannot reconstruct the original files I believe right ? And with the encryption key, this is stored on the client side and with it content is encrypted on the client side. The server only has the public key but not the private one. The private one is then never send to the server right ? Or am I understanding this wrong ?

 

[meyergru]

There is no asymmetric encryption involved, since there is so separation between sender and receiver: The client itself needs to read and write the data, so the server sees only encrypted data and has no key at all.

And since there is no asymmetric encryption, there will be no private and public keys - just one encryption key.

You should really read a thing or two about encryption and how that works before trying to design a "secure" solution like this.

What gfngfn256 was refering to is that when you have the encrypted data, you can always try to decrypt it. Depending on the type of encryption and the key strength, it may be feasible.

 

[proxenjoy]

There is no asymmetric encryption involved, since there is so separation between sender and receiver: The client itself needs to read and write the data, so the server sees only encrypted data and has no key at all.

And since there is no asymmetric encryption, there will be no private and public keys - just one encryption key.

You should really read a thing or two about encryption and how that works before trying to design a "secure" solution like this.

What gfngfn256 was refering to is that when you have the encrypted data, you can always try to decrypt it. Depending on the type of encryption and the key strength, it may be feasible.

Thank you!, for me it is just hard to imagine that you can intercept the keys that are used

 

[gfngfn256]

But from the metadata and RAM dumps you cannot reconstruct the original files I believe right ? And with the encryption key, this is stored on the client side and with it content is encrypted on the client side. The server only has the public key but not the private one. The private one is then never send to the server right ? Or am I understanding this wrong ?

I believe all of this is a gross misconception.

Let's analyze this. Firstly, who is "the client" ? This is the guest VM running on YOUR server.
Where is the RAM of this "client"? On YOUR server.

So yes if you take a RAM dump of that memory being used for that guest VM - you can pretty much see everything that is going on - including, but not limited to, encryption & decryption.

Yes, if you limit the guest VM to ONLY be a server receptable for a client application running on a different PC, & all data is already encrypted on the local PC of your friend - then technically the server only receives & stores already encrypted data.

So for example if you password zip a file on your PC & then send it by Gmail, Google only has the encrypted file, but no passwords or RAM dumps. But I find it hard to believe that this is all your friend is going to be doing with his VM. Add to this - that the filenames will be readily visible to Google. So in your friends case he may not want that you should even see a list of his filenames, & to stop this he'll have to take care to name all of his files differently, or encrypt them under different names/aliases.

All of this is only concerning my points 2 & 3 above, however point 1 stands as it is. So in the Gmail example above; Google could always try & crack that zipped file, it is not easy - but sometimes possible.

In short - if I were your friend I would want more protection than you are offering. Look at it this way - would you agree to the other way round; by putting your PC in his house, and you will access it remotely from your home? I think not.

 

[meyergru]

Thank you!, for me it is just hard to imagine that you can intercept the keys that are used

Probably, you do not even have to: There are two different scenarios:

1. Server-side encryption. In this case, you obviously must transfer the encryption key to the server, thus it can be intercepted, since you control the virtual hardware on which the encryption is done. As I said, the simplest form would be to manipulate the OS/application that is being used on the VM to extract the keys and store them somewhere. Another approach would be to snapshot the VM including RAM and extracting the key from the RAM dump.

2. Client-side or end-to-end encryption. The key stays in the realm of your friend, so it cannot be "intercepted" - however, as I said, if the encryption or even its implementation is weak, you may still be able to break the encryption, like when too short passwords are used, they are not being hashed (think of PBKDF2) or if you have a known plaintext and no salt (likely if you do full-disk encryption because certain areas are know to contain special signatures). And believe me, there have been many examples of so-called "secure" solutions for data encryption in the past, which turned out to be so badly implemented that they essentially gave everything away.

Not to mention quantum computing, which may render many current encryptions useless.