Skip to content

bz2: Fix truncation of total output size causing erroneous errors - #20807

Closed
ndossche wants to merge 2 commits into
php:PHP-8.4from
ndossche:bz2-sucks
Closed

ndossche wants to merge 2 commits into
php:PHP-8.4from
ndossche:bz2-sucks

Conversation

@ndossche

Copy link
Copy Markdown
Member

Also switch to uint64_t as that's used on master and makes the code simpler to fix.

@ndossche
ndossche marked this pull request as ready for review December 31, 2025 11:01

@arnaud-lb arnaud-lb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that this fixes the scaling of total_out_hi32, which was broken because the operation has unsigned int result type. This also backports #20807.

This looks right to me.

It seems worth it to add a test:

$compressed_file = 'test.bz2';
$sum = 'd4b5e52ed34a774fa645f94369b0c61375436d30';

/*
    $fd = popen("bzip2 > $compressed_file", "w");
    for ($i = 0; $i < 8; $i++) {
        fwrite($fd, str_repeat((string)$i, 2**30));
    }
    fclose($fd);
*/

var_dump(sha1(bzdecompress(file_get_contents($compressed_file))) === $sum);

test.bz2 is 8KiB, so we can embed it to save some time.

@ndossche

ndossche commented Jan 2, 2026

Copy link
Copy Markdown
Member Author

Thanks. The test will consume a lot of memory though so perhaps we should postpone merging this until #20762 & co are resolved

@arnaud-lb

Copy link
Copy Markdown
Member

Yes this seems reasonable

Also switch to uint64_t as that's used on master and makes the code
simpler to fix.

Co-authored-by: Arnaud Le Blanc <arnaud.lb@gmail.com>
@ndossche ndossche closed this in 4ee95fc Feb 27, 2026
@iluuu1994

Copy link
Copy Markdown
Member

@ndossche

Copy link
Copy Markdown
Member Author

ASAN could simply be skipped, it's not too surprising it performs worse.

Agreed

Though it also failed for a macOS release job. I presume the test can't be meaningfully made cheaper?

Not really, we need to cross the 4GiB boundary to reproduce the issue. I don't know why macOS fails.

@iluuu1994

Copy link
Copy Markdown
Member

How quickly does the test run on your machine? Given this was limited to macOS apart from ASAN, maybe it's ok to just skip it there as well, assuming the fix is not platform-dependent.

@ndossche

ndossche commented Feb 28, 2026 •

Copy link
Copy Markdown
Member Author

Fix isn't platform dependent. I pushed some extra changes to the SKIPIF.
Takes around 41s on my i7-4790, on a release build

@iluuu1994

Copy link
Copy Markdown
Member

@ndossche Thanks! The timeout doesn't seem too surprising then. The run-tests.php timeout can be controlled through the TEST_TIMEOUT env var, but only for all tests. Another option might be to allow increasing this on a test-by-test basis. I'm happy with the SKIPIF though.

@ndossche

Copy link
Copy Markdown
Member Author

We'll see if the issue reoccurs and can take action then

bukka added a commit that referenced this pull request Sep 22, 2026
The decompressed output of this test peaks at more than 12 GiB of RSS.
On smaller machines, such as the 7 GB GitHub-hosted runners used for
private repositories, this exhausts the whole VM and the OOM killer takes
the test runner down with it, so the job dies with a runner shutdown
signal instead of a test failure.

Gate the test on MemAvailable from /proc/meminfo like the other
resource-heavy tests gate on memory, so it keeps running on the 16 GB
public runners and skips itself elsewhere.
bukka added a commit that referenced this pull request Sep 22, 2026
* PHP-8.4:
  Skip bz2 GH-20807 test when less than 13 GiB of memory is available
  ext/soap: make GHSA-cj93-vc83-wgqv test lean and reliable
  Add NEWS entries
  ext/standard: Fix 1-char relative Location redirects after GH-23467
  [http] Fix out-of-bounds read on empty Location header
  Fix GHSA-ch8v-r6jh-4vvr: encode 0xFF in FILTER_SANITIZE_ENCODED
  Fix GHSA-9f67-6fw4-hpfp
  Fix GHSA-j3wh-g957-2m85: phar tar entry injection
  Fix GHSA-cj93-vc83-wgqv
  Fix GHSA-rgrp-mwpx-f6rm: unbounded recursion in ext/soap XML parsing and decoding
  Fix GHSA-fpwc-w8rq-cr92: strip credentials from user headers on cross-origin redirects
  Fix GHSA-r6x9-5r99-36j7: Various packet overreads in mysqlnd wireprotocol
  Fix GHSA-xr7j-rvgx-xq5p: OOB read in php_openssl_matches_wildcard_name()
  Fix GHSA-vvx9-73fr-5jjx: do not fall back to CN if the cert has a service ID
  Fix GHSA-62xp-839h-2637: FastCGI allowed_clients compared only 96 bits of IPv6 addresses
  Fix heap-buffer-overflow in convert stream filters with NUL in line-break-chars

# Conflicts:
#	NEWS
#	ext/phar/tar.c
#	ext/soap/php_http.c
#	ext/soap/php_xml.c
#	ext/standard/http_fopen_wrapper.c
#	win32/ioutil.c
bukka added a commit that referenced this pull request Sep 22, 2026
* PHP-8.5:
  Skip bz2 GH-20807 test when less than 13 GiB of memory is available
  ext/soap: make GHSA-cj93-vc83-wgqv test lean and reliable
  Add NEWS entries
  ext/standard: Fix 1-char relative Location redirects after GH-23467
  [http] Fix out-of-bounds read on empty Location header
  Fix GHSA-ch8v-r6jh-4vvr: encode 0xFF in FILTER_SANITIZE_ENCODED
  Fix GHSA-9f67-6fw4-hpfp
  Fix GHSA-j3wh-g957-2m85: phar tar entry injection
  Fix GHSA-cj93-vc83-wgqv
  Fix GHSA-rgrp-mwpx-f6rm: unbounded recursion in ext/soap XML parsing and decoding
  Fix GHSA-fpwc-w8rq-cr92: strip credentials from user headers on cross-origin redirects
  Fix GHSA-r6x9-5r99-36j7: Various packet overreads in mysqlnd wireprotocol
  Fix GHSA-xr7j-rvgx-xq5p: OOB read in php_openssl_matches_wildcard_name()
  Fix GHSA-vvx9-73fr-5jjx: do not fall back to CN if the cert has a service ID
  Fix GHSA-62xp-839h-2637: FastCGI allowed_clients compared only 96 bits of IPv6 addresses
  Fix heap-buffer-overflow in convert stream filters with NUL in line-break-chars

# Conflicts:
#	NEWS
#	ext/openssl/xp_ssl.c
#	ext/phar/tar.c
#	ext/soap/php_http.c
#	ext/standard/http_fopen_wrapper.c
edorian pushed a commit that referenced this pull request Sep 22, 2026
The decompressed output of this test peaks at more than 12 GiB of RSS.
On smaller machines, such as the 7 GB GitHub-hosted runners used for
private repositories, this exhausts the whole VM and the OOM killer takes
the test runner down with it, so the job dies with a runner shutdown
signal instead of a test failure.

Gate the test on MemAvailable from /proc/meminfo like the other
resource-heavy tests gate on memory, so it keeps running on the 16 GB
public runners and skips itself elsewhere.
SakiTakamachi pushed a commit that referenced this pull request Sep 22, 2026
The decompressed output of this test peaks at more than 12 GiB of RSS.
On smaller machines, such as the 7 GB GitHub-hosted runners used for
private repositories, this exhausts the whole VM and the OOM killer takes
the test runner down with it, so the job dies with a runner shutdown
signal instead of a test failure.

Gate the test on MemAvailable from /proc/meminfo like the other
resource-heavy tests gate on memory, so it keeps running on the 16 GB
public runners and skips itself elsewhere.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants