Skip to content

ZLIB failing to inflate entire .gz file that contains an extra field in the header #6116

Closed

Description

@chmille4
  • 4.4.2:
  • OSX:
  • 10.10.5:

The Node ZLIB library only inflates the first 64k of a gzipped file (attached). The file has an extra field in the header, which may be causing the problem. GUNZIP on mac and linux correctly inflate it to it's full size (~33M). I've also attached a simple test script which takes the gzip file on stdin and outputs the inflated file to stdout. If you run the test script with the attached gzip file you should get 33M but you only get 64K of data.
zlib_test.js.zip

IDb.part1.gz

Note: the attached gzip file has been sliced to make it smaller so that I could upload it here. Therefore gunzip will complain and say "unexpected end of input", however this file is displaying the same behavior as the original larger gzip file.

Activity

  1. yorkie commented on Apr 8, 2016

    @yorkie
    Contributor

    @chmille4 Do you have tried with file stream though?

  2. chmille4 commented on Apr 8, 2016

    @chmille4
    Author

    @yorkie I don't quite understand. Can you give me an example of what you mean? Thanks

  3. yorkie commented on Apr 8, 2016

    @yorkie
    Contributor

    I mean you should select single variable when evaluating a result, you are using stdin/stdout to take inputs and to make sure what's the result, but it might not be working with Node.js, so could you debug it by the following steps:

    • read the gzip file by fs.readFile and make sure the contents length is equal to the expect size.
    • inflate the above buffer and put the result to a buffer object.
    • check that buffer's length if is also expected

    If the above steps are working correctly, this issue might not be caused by zlib.

  4. added
    zlibIssues and PRs related to the zlib module and its compression dependencies.
    on Apr 8, 2016
  5. mscdex commented on Apr 8, 2016

    @mscdex
    Contributor

    Here's the script:

    var zlib = require('zlib');
    
    process.stdin.pipe(zlib.createGunzip()).pipe(process.stdout);
  6. mscdex commented on Apr 8, 2016

    @mscdex
    Contributor

    @chmille4 Can you try with node v5.10.1 and the master branch if that doesn't work? I know there were some zlib-related changes recently, some of which are in v5.x and some that are not yet.

  7. addaleax commented on Apr 8, 2016

    @addaleax
    Member

    I’m 99 % sure this is a duplicate of #4306, which has been fixed in #5120. The script works as expected with Node.js v5.9.0+.

  8. chmille4 commented on Apr 8, 2016

    @chmille4
    Author

    Yep, upgrading to v5.10.1. Fixed the issue. Awesome, thanks for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    zlibIssues and PRs related to the zlib module and its compression dependencies.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions