Repository navigation
buffer.toString fails for buffers larger than 192MB #4266
Description
buffer.toString with base64 encoding fails for any buffer larger than or equal to 201326581 bytes (~192 MB).
See the following piece of code:
var buf1 = new Buffer(201326580), //biggest buffer that works
buf2 = new Buffer(201326581); //~192 MB
buf1.toString('base64'); //works
buf2.toString('base64'); //fails!
For older node versions (like v0.12.2) this cause the process to crash with this error:
#
# Fatal error in d:\jenkins\workspace\nodejs-msi-julien\d8c2e2bb\deps\v8\src/handles.h, line 48
# CHECK(location_ != NULL) failed
#
Segmentation fault
For newer versions (like v5.2.0), it gives the following error:
buffer.js:401
throw new Error('toString failed');
^
Activity
ChALkeR commentedon Dec 14, 2015
Probably a dupe of #3175
bnoordhuis commentedon Dec 15, 2015
Slightly more representative tests:
Nominally a V8 bug due to the use of
.ToHandleChecked()indeps/v8/src/api.cc. We could work around it by checking thatlength < 268435441before callingv8::String::NewExternal()insrc/string_bytes.cc.I'm not going to work on it but if someone wants to pick it up, I'll review the patches.
skilledDeveloper commentedon Dec 15, 2015
@bnoordhuis
I'll work on it.
fix: workaround for possible V8 bug fixes nodejs#4266
silverwind commentedon Dec 15, 2015
Is v4 also affected?
skilledDeveloper commentedon Dec 15, 2015
Yes. but it doesn't crash the process.
RohanGada commentedon Dec 27, 2015
Unable to read a 40MB xlsx due to this.
mongod 3.0.6
Node 0.12.7
trevnorris commentedon Dec 28, 2015
@RohanGada Mind opening another issue with test case we can look at? Your issue will probably go unnoticed within an already closed issue.
EDIT: This isn't a closed issue yet. Don't know why I thought so.
RohanGada commentedon Dec 29, 2015
Okay
kingroryg commentedon Jan 11, 2016
Is this fixed ?
silverwind commentedon Jan 11, 2016
#4394 has yet to land.
wmbutler commentedon May 31, 2016
Curious to know if there's been any progress on this yet? I'm running v5.6.0 and still seeing this behavior.
trevnorris commentedon Jun 1, 2016
Best I can figure this is a v8 bug. Unfortunately my v8 foo isn't good enough to track this down. /cc @indutny want to track down a v8 crash? :)
wmbutler commentedon Aug 19, 2016
I had to bypass node and use bash with
jqto parse large base64 objects throughbash.davidrapin commentedon Oct 6, 2016
This issue is a duplicate of #3175
skilledDeveloper commentedon Oct 6, 2016
@davidrapin It's not duplicate.
This is about an unhanded exception which breaks the process.
davidrapin commentedon Oct 6, 2016
Ok got it, just found out what the
v0.12tag meant ;)Message: Workaround Buffer.toString failure
jasnell commentedon Dec 29, 2016
Closing this given that support for v0.12 is coming to a close in two days and there will not be any further updates to the v0.12 branch.
Message: Again workaround Buffer.toString failure