Repository navigation
Buffer created using new Buffer(buffer) has a different ArrayBuffer from the original one #4341
Description
Hope the title describes the issue correctly:)
Codes:
var typed_array = new Uint8Array(6);
for(var i = 0; i < typed_array.length; i++){
typed_array[i] = i;
}
var array_buffer = typed_array.buffer;
console.log("byteLength of ArrayBuffer : " + array_buffer.byteLength);
var node_buffer = new Buffer(array_buffer);
console.log("Node Buffer : ");
console.log(node_buffer);
console.log("byteLength of ArrayBuffer in Node Buffer : ", node_buffer.buffer.byteLength);
var another_node_buffer = new Buffer(node_buffer);
console.log("Another Node Buffer : ");
console.log(another_node_buffer);
console.log("byteLength of ArrayBuffer in Another Node Buffer : ", another_node_buffer.buffer.byteLength);node_buffer.buffer is different from another_node_buffer.buffer !
Activity
ChALkeR commentedon Dec 18, 2015
This is expected behaviour.
var node_buffer = new Buffer(array_buffer);does not copy your typed array, it creates aBufferrepresentation that uses the sameArrayBufferas the storage:Edit: example fixed, there was a mistake in the initial version.
var another_node_buffer = new Buffer(node_buffer);copies your buffer to a new one and uses its own storage, which is not guaranteed to be of the same length as your data for performance reasons.See https://nodejs.org/api/buffer.html#buffer_new_buffer_buffer and https://nodejs.org/api/buffer.html#buffer_class_slowbuffer.
ChALkeR commentedon Dec 19, 2015
Btw, is
new Buffer(array_buffer)documented?jasnell commentedon Dec 21, 2015
@ChALkeR ... not actually seeing that behavior at all:
See: https://nodejs.org/dist/latest-v5.x/docs/api/buffer.html#buffer_buffer
Edit... hmm ok, nevermind... went back and re-read the example above... it's a bit inconsistent..
ChALkeR commentedon Dec 21, 2015
@jasnell Sorry, there was a copy-paste error in my example (when I tried to fit it into two repl lines). Fixed.
Fishrock123 commentedon Dec 21, 2015
cc @trevnorris
trevnorris commentedon Dec 21, 2015
So the confusion seems to be that
new Buffer(node_buffer)should essentially perform anode_buffer.slice()operation instead of copying the data into a new Buffer instance.How this works isn't likely to change. Do we need better explanation in our documentation?
ChALkeR commentedon Dec 21, 2015
@trevnorris That and the missing documentation for
new Buffer(array_buffer).jasnell commentedon Mar 22, 2016
I believe this is addressed with the recent commits. Closing
jbergknoff commentedon Jun 8, 2016
I came across this issue myself after investigating a memory corruption issue in https://github.com/mozilla/pdf.js/.
Here's behavior that I consider very surprising:
@jasnell can you elaborate about which commits address this and/or how it's been addressed. Thank you.
trevnorris commentedon Jun 8, 2016
@jbergknoff That is expected.
Buffer.poolSizeis the number of bytes to allocate to a shared pool. Any allocation that's<= Buffer.poolSize / 2in size will be allocated from this pool. In your example doconsole.log(x.byteOffset)and you'll see how many bytes into the pool that allocation starts. node has been doing this since v0.3.jbergknoff commentedon Jun 8, 2016
@trevnorris thank you for the clarification. With respect, it's only expected if one is intimately familiar with the implementation details of Node's
Buffer. I've usedBuffersomewhat extensively, but I was unaware it even exposed a.bufferproperty until I had to dig into the typed array usage in the library I mentioned above. Perhaps the documentation could be expanded on this point.ChALkeR commentedon Jun 8, 2016
@jbergknoff https://nodejs.org/api/buffer.html#buffer_buffers_and_typedarray