Skip to content

LZMADecompressor.decompress(b'', max_length=0) raises LZMAError on the second consecutive call #160000

Open

Description

@pganssle

Bug description

Calling decompress with no new input and max_length=0 works once, but a second consecutive such call raises:

>>> import lzma
>>> d = lzma.LZMADecompressor()
>>> d.decompress(b"", max_length=0)
b''
>>> d.decompress(b"", max_length=0)
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    d.decompress(b"", max_length=0)
_lzma.LZMAError: Insufficient buffer space

BZ2Decompressor, zlib._ZlibDecompressor and ZstdDecompressor all return b'' for any number of such calls, and the decompressor stays usable. The same happens after real input, e.g. decompress(stream, 1) followed by two decompress(b'', 0) calls.

max_length=0 is a legitimate request ("just buffer this, don't produce output yet"), so I would expect b'' and no error.

The mechanism, as far as I can tell: liblzma returns LZMA_BUF_ERROR when two consecutive lzma_code calls can make no progress (no input consumed, no output produced). decompress_buf already treats that as benign, but only when there is output space available: Modules/_lzmamodule.c lines 962-964. With max_length=0 the output buffer has zero bytes, so avail_out > 0 is false and the error goes to catch_lzma_error, which maps it to "Insufficient buffer space" (Modules/_lzmamodule.c lines 169-171). Dropping the avail_out > 0 condition when avail_in == 0 is probably enough; zlibmodule.c tolerates Z_BUF_ERROR unconditionally in the equivalent loop.

Found by the Hypothesis tests in #159960, which record it as LZMADecompressorTests.test_repeated_max_length_zero under expectedFailure. Those tests also leave max_length=0 out of the broader lzma strategies for now so that the other properties keep running; that carve-out can go once this is fixed.

CPython versions tested on

CPython main (4e3d5ec), liblzma 5.8.4

Operating systems tested on

Linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions