Tool/library to compress/decompress on the fly I am looking for library or linux command line tool that allows to compress and decompress on the fly. By 'on the fly' I mean something that will work with following sh script: > while:; do; echo "foobar"; sleep 1; done; | <tool to compress> > file & > sleep 4 > <tool to decompress> file foobar foobar foobar foobar As far as I understand `gzip` and zlib will not work here. My "test" produces following output: > while:; do; echo "foobar"; sleep 1; done; | gzip > file & > sleep 4 > zcat file gzip: file: unexpected end of file Are there any options?

`gzip` does work, but it only writes 16KiB compressed blocks (in my tests), and since your input compresses _really_ well, it takes a while to get there. If you run


while :; do echo "foobar"; sleep 1; done | gzip > file &


wait a little and then kill only the first part, you’ll get a valid compressed `file`.

If you want to test without killing anything, drop the `sleep`:


yes foobar | gzip > file &


After a few seconds you’ll see `file` increase to 16KiB.