Skip to content

zlib._ZlibDecompressor(zdict=...) raises SystemError on a zlib-wrapped stream #160001

Open

Description

@pganssle

Bug description

zlib._ZlibDecompressor accepts a zdict argument, but for a zlib-wrapped stream (positive wbits) compressed with that dictionary, decompress fails with a SystemError instead of decompressing. zlib.decompressobj handles the same input:

>>> import zlib
>>> zd = b"hello world " * 10
>>> c = zlib.compressobj(zdict=zd)
>>> s = c.compress(b"hello world hello") + c.flush()
>>> zlib.decompressobj(zdict=zd).decompress(s)
b'hello world hello'
>>> zlib._ZlibDecompressor(zdict=zd).decompress(s)
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    zlib._ZlibDecompressor(zdict=zd).decompress(s)
SystemError: <method 'decompress' of 'zlib._ZlibDecompressor' objects> returned NULL without setting an exception

Raw deflate works, so this is specific to the wrapped format:

>>> c = zlib.compressobj(wbits=-15, zdict=zd)
>>> s = c.compress(b"hello world hello") + c.flush()
>>> zlib._ZlibDecompressor(-15, zdict=zd).decompress(s)
b'hello world hello'

Two things combine here. The constructor only applies the dictionary up front for raw deflate (wbits < 0): Modules/zlibmodule.c lines 1775-1780. For a wrapped stream zlib instead signals Z_NEED_DICT after reading the header, and decompress_buf handles that by jumping to error without setting an exception: Modules/zlibmodule.c lines 1535-1541. zlib.Decompress handles the same return by calling inflateSetDictionary and continuing: Modules/zlibmodule.c lines 932-938, which is presumably what _ZlibDecompressor should do as well (and raise zlib.error when no zdict was given, as Decompress does).

_ZlibDecompressor is private and gzip only uses it with wbits=-15, so the impact is low; but a SystemError from a C method is always a bug, and the argument is accepted for this case, so we may as well fix it.

Found by the Hypothesis tests in #159960, recorded there as ZlibDecompressorZdictTests.test_zlib_wrapper under expectedFailure.

CPython versions tested on

CPython main (4e3d5ec), zlib 1.3.2

Operating systems tested on

Linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions