[SOLVED] cannot dd within lxc container

[lethargos]

Why is not possible to use dd to write some junk file with zeros within an lxc container?
For instance, if I do:

dd if=/dev/zero of=/some_file bs=1G

it doesn't work. After a few seconds it says 'killed' and that's that. What is the explanation for this?

 

[Alwin]

I guess, the 1G is bigger then your available memory inside the container?
https://bugzilla.proxmox.com/show_bug.cgi?id=1658

 

[lethargos]

I've also thought about that, but it actually wouldn't make any sense whatsoever. It would mean that in order to write a 10GB-file, you'd need at least 10GB of ram. That's obviously not true. Moreover, I've also tried using a much lower value, such as 30M. Still not working.

After reading your link, I see that it is somehow connected to ram usage. But it's really abnormal. On the host level, the only limit for writing a file would be storage space.

 

[Alwin]

LXC is somehow caching writes, at least if you look at the iostats. This probably would be a reason, why the OOM could happen.

 

[lethargos]

It looks like a terrible limitation...

 

[lethargos]

I've got 20GB on this container with 512MB of RAM. I did dd if=/dev/zero of=crap bs=30M and I ended up with a 20GB file named 'crap'.
dd: error writing 'crap': No space left on device

This is really weird

Ok, I think I really needed to specify the count option. It seems to be working fine when it's included. On the other hand, I remember doing it in other contexts without count and it worked fine.

I'm guessing that bs should be less than RAM, but the whole file doesn't need to.
bs=1G count=1 isn't working.
bs=100M count=10 does work

 

[wbumiller]

`bs` is the block size, in order to read and write blocks of 1G at a time, dd needs to prepare 1G of data in ram, otherwise there's no way to pass that much to a single write() call (okay, for writing zeroes in particular there would be, but dd doesn't care what the input file is).
Apart from that, there have been bugs with I/O caching with some file systems in the past, but if the smaller bs with a count works for you than that's unlikely to be the issue.

 

[lethargos]

What do you mean exactly by 'I/O caching'? If we're talking about a HDD, are you referring to the disk buffer? The one that's 32, 64MB on a desktop hdd or 128MB or more on a server HDD?

 

[wbumiller]

The system's cache in RAM (what you (are supposed to) see as buffer/cache in the output of `free`).