Skip to content

Executing unoptimized wasm file causes OOM #45480

New issue
New issue
Closed as not planned
Closed as not planned
Executing unoptimized wasm file causes OOM#45480
Labels
v8 engineIssues and PRs related to the V8 dependency.wasmIssues and PRs related to WebAssembly.

Description

@Nugine
Nugine
opened on Nov 15, 2022

Version

v19.1.0

Platform

Linux Nugine-Ub2 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Dependencies:

git clone https://github.com/Nugine/simd.git -b main
cd simd
git checkout 508bffc65c3b72b889666773fe3e30a2b5a329f4
cd crates/hex-simd
just wasm-test

I uploaded artifacts here for convenience: wbg_tmp.tar.gz

tar xvf wbg_tmp.tar.gz
cd wbg_tmp
ulimit -v 4000000 # 4GB
node run.js
/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:716
const wasmInstance = new WebAssembly.Instance(wasmModule, imports);
                     ^

RangeError: WebAssembly.Instance(): Out of memory: Cannot allocate Wasm memory for new instance
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:716:22)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)
    at Module.load (node:internal/modules/cjs/loader:1068:32)
    at Module._load (node:internal/modules/cjs/loader:909:12)
    at Module.require (node:internal/modules/cjs/loader:1092:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/run.js:3:22)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)

Node.js v19.1.0

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior?

It should work without OOM.

What do you see instead?

Nodejs allocates too much memory until OOM.

Additional information

See Nugine/simd@48bd619#comments

Activity

  1. VoltrexKeyva
    added
    wasmIssues and PRs related to WebAssembly.
    on Nov 16, 2022

  2. bnoordhuis commented on Nov 29, 2022

    @bnoordhuis
    bnoordhuis
    on Nov 29, 2022
    Member

    Guess no one else is going to pick this up? I can't try myself right now but does it work with the --jitless flag? V8's JIT makes fairly large reservations for WASM code. It's possible you're hitting a resource limit.

  3. Nugine commented on Nov 29, 2022

    @Nugine
    Nugine
    on Nov 29, 2022
    Author
    node --jitless run.js
    Warning: disabling flag --expose_wasm due to conflicting flags
/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:715
const wasmModule = new WebAssembly.Module(bytes);
                   ^

ReferenceError: WebAssembly is not defined
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:715:20)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)
    at Module.load (node:internal/modules/cjs/loader:1068:32)
    at Module._load (node:internal/modules/cjs/loader:909:12)
    at Module.require (node:internal/modules/cjs/loader:1092:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/run.js:3:22)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)

Node.js v19.1.0

    ulimit -v 12000000 # 12GB
node run.js

    running 3 tests

test hex_simd::tests::test_safety ... ok
test hex_simd::tests::test_alloc ... ok
test hex_simd::tests::test_str ... ok

test result: ok. 3 passed; 0 failed; 0 ignored



#
# Fatal error in , line 0
# Fatal process out of memory: Zone
#
#
#
#FailureMessage Object: 0x7fb5f0ac6fb0


#
# Fatal error in , line 0
# Fatal process out of memory: Zone
#
#
#
#FailureMessage Object: 0x7fb5eaff3010


#
# Fatal error in , line 0
# Fatal process out of memory: Zone
#
#
#
#FailureMessage Object: 0x7fb5ebff4fc0
 1: 0xc2abf1  [node]
 1: 0xc2abf1  [node]
 1: 0xc2abf1  [node]
 2: 0x1f231d4 V8_Fatal(char const*, ...) [node]
 2: 0x1f231d4 V8_Fatal(char const*, ...) [node]
 3: 0xda71ed  [node]
 3: 0xda71ed  [node]
 4: 0x140d53e  [node]
 5: 0x157353d  [node]
 6: 0x158ed18  [node]
 7: 0x15986b7 v8::internal::wasm::BuildTFGraph(v8::internal::AccountingAllocator*, v8::internal::wasm::WasmFeatures const&, v8::internal::wasm::WasmModule const*, v8::internal::compiler::WasmGraphBuilder*, v8::internal::wasm::WasmFeatures*, v8::internal::wasm::FunctionBody const&, std::vector<v8::internal::compiler::WasmLoopInfo, std::allocator<v8::internal::compiler::WasmLoopInfo> >*, v8::internal::compiler::NodeOriginTable*, int, v8::internal::wasm::InlinedStatus) [node]
 2: 0x1f231d4 V8_Fatal(char const*, ...) [node]
 3: 0xda71ed  [node]
 4: 0x140d53e  [node]
 5: 0x1570b65  [node]
 6: 0x158ceaf  [node]
 7: 0x15986b7 v8::internal::wasm::BuildTFGraph(v8::internal::AccountingAllocator*, v8::internal::wasm::WasmFeatures const&, v8::internal::wasm::WasmModule const*, v8::internal::compiler::WasmGraphBuilder*, v8::internal::wasm::WasmFeatures*, v8::internal::wasm::FunctionBody const&, std::vector<v8::internal::compiler::WasmLoopInfo, std::allocator<v8::internal::compiler::WasmLoopInfo> >*, v8::internal::compiler::NodeOriginTable*, int, v8::internal::wasm::InlinedStatus) [node]
 8: 0x209dc64 v8::internal::compiler::ExecuteTurbofanWasmCompilation(v8::internal::wasm::CompilationEnv*, v8::internal::wasm::WireBytesStorage const*, v8::internal::wasm::FunctionBody const&, int, v8::internal::Counters*, v8::internal::wasm::AssemblerBufferCache*, v8::internal::wasm::WasmFeatures*) [node]
 9: 0x156eac3 v8::internal::wasm::WasmCompilationUnit::ExecuteFunctionCompilation(v8::internal::wasm::CompilationEnv*, v8::internal::wasm::WireBytesStorage const*, v8::internal::Counters*, v8::internal::wasm::AssemblerBufferCache*, v8::internal::wasm::WasmFeatures*) [node]
10: 0x156f5dc v8::internal::wasm::WasmCompilationUnit::ExecuteCompilation(v8::internal::wasm::CompilationEnv*, v8::internal::wasm::WireBytesStorage const*, v8::internal::Counters*, v8::internal::wasm::AssemblerBufferCache*, v8::internal::wasm::WasmFeatures*) [node]
11: 0x15ae590  [node]
12: 0x15af7ab  [node]
13: 0x1beafe8 v8::platform::DefaultJobWorker::Run() [node]
14: 0xc2aea5  [node]
15: 0x7fb5f136cb43  [/lib/x86_64-linux-gnu/libc.so.6]
16: 0x7fb5f13fea00  [/lib/x86_64-linux-gnu/libc.so.6]
Trace/breakpoint trap (core dumped)

  4. bnoordhuis commented on Nov 29, 2022

    @bnoordhuis
    bnoordhuis
    on Nov 29, 2022
    Member

    ulimit -v 12000000 # 12GB

    Failing with a 12 GB virtual memory limit isn't surprising. V8 allocates really large guard pages for performance reasons. See #45557 for a similar bug report.

  5. Nugine commented on Nov 29, 2022

    @Nugine
    Nugine
    on Nov 29, 2022
    Author

    I look into the process status before it gets killed. VmRSS is huge. Is it the expected behavior?

    node run.js

    cat /proc/14376/status

    ...
VmPeak:	28405548 kB
VmSize:	28340012 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	14549680 kB
VmRSS:	14549680 kB
RssAnon:	14545316 kB
RssFile:	    4364 kB
RssShmem:	       0 kB
VmData:	17059148 kB
VmStk:	     280 kB
VmExe:	   77716 kB
VmLib:	    8700 kB
VmPTE:	   33848 kB
VmSwap:	 1056128 kB
...

    It's quite weird that nodejs allocates too much memory after the wasm code is finished. I still think this hehavior may be a bug.

  6. bnoordhuis commented on Nov 29, 2022

    @bnoordhuis
    bnoordhuis
    on Nov 29, 2022
    Member

    That's indeed quite high. It could be the JS heap (not WASM), it maxes out at 1.4 GB unless you raise the limit.

    As a bit of expectation management: WASM is handled by V8, not node, so if there is a bug, you're going to have to report it upstream.

    With that out of the way, it'd be good to test WASM-related flags. See node --v8-options. For example, --liftoff-only lets you check if the problem is in TurboFan, the optimizing compiler (visible in the stack trace above.)

  7. Nugine commented on Nov 30, 2022

    @Nugine
    Nugine
    on Nov 30, 2022
    Author
    node --liftoff-only run.js

    running 3 tests

test hex_simd::tests::test_safety ... ok
test hex_simd::tests::test_alloc ... ok
test hex_simd::tests::test_str ... ok

test result: ok. 3 passed; 0 failed; 0 ignored

    So the problem is in TurboFan.

    I have submitted a issue to v8: https://bugs.chromium.org/p/v8/issues/detail?id=13543

  8. bnoordhuis
    added
    v8 engineIssues and PRs related to the V8 dependency.
    on Nov 30, 2022

  9. bnoordhuis commented on Nov 30, 2022

    @bnoordhuis
    bnoordhuis
    on Nov 30, 2022
    Member

    I'd include more details in the bug report if I were you, makes it more likely to get triaged quickly.

    I'll close out this bug report as there isn't anything for node to do at this time.

  10. bnoordhuis
    closed this as not plannedon Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    v8 engineIssues and PRs related to the V8 dependency.wasmIssues and PRs related to WebAssembly.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions