Node runs out of Memory? #3
Description
Seems I have too many notes, I've tried setting my Node memory stack to 20GB with:
export NODE_OPTIONS="--max-old-space-size=20096"
But it still crashes,
I've got about 2500 notes.
Starting chunk processing for note: HomeBridge Notes / Hacking Homebridge
Using the compiled version as my Mac (M3) has an issue where it wants to use the x86_64 version if I build from scratch.
Thanks for putting this together!
Ant
========= Stack Trace =====
<--- Last few GCs --->
[30166:0x13800510000] 632935 ms: Scavenge 3959.2 (4087.1) -> 3959.1 (4087.9) MB, pooled: 0 MB, 8.50 / 0.00 ms (average mu = 0.425, current mu = 0.245) allocation failure;
[30166:0x13800510000] 637218 ms: Mark-Compact 3960.1 (4087.9) -> 3960.1 (4087.9) MB, pooled: 0 MB, 4277.96 / 0.00 ms (average mu = 0.292, current mu = 0.123) allocation failure; scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0x1121c9254 node::OnFatalError(char const*, char const*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
2: 0x10c68e6a4 _$LT$font_types..tag..Tag$u20$as$u20$core..cmp..PartialEq$LT$$u5b$u8$u3b$$u20$4$u5d$$GT$$GT$::eq::h1cf26754fa042eb7 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
3: 0x10c68e638 _$LT$font_types..tag..Tag$u20$as$u20$core..cmp..PartialEq$LT$$u5b$u8$u3b$$u20$4$u5d$$GT$$GT$::eq::h1cf26754fa042eb7 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
4: 0x10c84f4c0 v8::CppHeap::wrapper_descriptor() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
5: 0x10c8834fc v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
6: 0x10c882e68 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
7: 0x10c882bd0 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
8: 0x10c8937f0 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
9: 0x112237f88 node::OnFatalError(char const*, char const*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
10: 0x1122382d4 node::OnFatalError(char const*, char const*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
11: 0x10c8848b8 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
12: 0x10c875e38 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
13: 0x10c872b3c v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
14: 0x10c852048 v8::CppHeap::wrapper_descriptor() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
15: 0x10c851874 v8::CppHeap::wrapper_descriptor() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
16: 0x10c864cd8 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
17: 0x10c8649d0 v8::internal::StrongRootAllocatorBase::deallocate_impl(unsigned long*, unsigned long) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
18: 0x10cf725d0 cppgc::internal::AgeTable::ResetForTesting() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
19: 0x10c84db60 v8::CppHeap::wrapper_descriptor() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
20: 0x10c843f3c v8::CppHeap::wrapper_descriptor() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
21: 0x10c826c54 v8::CppHeap::wrapper_descriptor() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
22: 0x10cc14778 v8::internal::TickSample::print() const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
23: 0x147ea9614
24: 0x147f49f48
25: 0x1402a8d30
26: 0x14020adbc
27: 0x147e4c8d4
28: 0x147f1f578
29: 0x147e3c184
30: 0x147e0b598
31: 0x10c7a330c v8::Unwinder::PCIsInV8(unsigned long, v8::MemoryRange const*, void*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
32: 0x10c7a3fdc v8::Unwinder::PCIsInV8(unsigned long, v8::MemoryRange const*, void*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
33: 0x10c7a4130 v8::Unwinder::PCIsInV8(unsigned long, v8::MemoryRange const*, void*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
34: 0x10c7cdd28 v8::Unwinder::PCIsInV8(unsigned long, v8::MemoryRange const*, void*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
35: 0x147e10674
36: 0x1403114ec
37: 0x147e0b6a8
38: 0x147e0b398
39: 0x10c7a2484 v8::Unwinder::PCIsInV8(unsigned long, v8::MemoryRange const*, void*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
40: 0x10c6a4568 v8::Function::Call(v8::Localv8::Context, v8::Localv8::Value, int, v8::Localv8::Value) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
41: 0x1121098e8 node::CallbackScope::~CallbackScope() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
42: 0x112109b90 node::CallbackScope::~CallbackScope() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
43: 0x11211e4cc node::EmitAsyncDestroy(node::Environment, node::async_context) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
44: 0x1122c4d24 node::PromiseRejectCallback(v8::PromiseRejectMessage) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
45: 0x1122c6aa0 node::PromiseRejectCallback(v8::PromiseRejectMessage) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
46: 0x1122cb0f4 node::PromiseRejectCallback(v8::PromiseRejectMessage) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
47: 0x1122cb808 node::PromiseRejectCallback(v8::PromiseRejectMessage) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
48: 0x10b488b80 uv_signal_stop [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
49: 0x10b48fe5c uv_free_interface_addresses [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
50: 0x10b47f7d0 uv_run [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
51: 0x10b6259a0 node::FreeArrayBufferAllocator(node::ArrayBufferAllocator*) [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
52: 0x10e786a7c std::sys::pal::unix::os::errno::h12fdde1c2329795a [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
53: 0x10e7a0f5c std::sys::pal::unix::os::errno::h12fdde1c2329795a [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
54: 0x10e7a156c std::sys::pal::unix::os::errno::h12fdde1c2329795a [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
55: 0x10e7eb494 std::process::id::h056ebf195753a258 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
56: 0x10b3f5600 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
57: 0x10e7ea65c std::process::id::h056ebf195753a258 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
58: 0x1820b48a4 CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION [/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation]
59: 0x1820b4838 __CFRunLoopDoSource0 [/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation]
60: 0x1820b459c __CFRunLoopDoSources0 [/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation]
61: 0x1820b3138 __CFRunLoopRun [/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation]
62: 0x1820b2734 CFRunLoopRunSpecific [/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation]
63: 0x18d621530 RunCurrentEventLoopInMode [/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox]
64: 0x18d627348 ReceiveNextEventCommon [/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox]
65: 0x18d627508 _BlockUntilNextEventMatchingListInModeWithFilter [/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox]
66: 0x185c2a848 _DPSNextEvent [/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit]
67: 0x186590c24 -[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] [/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit]
68: 0x185c1d874 -[NSApplication run] [/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit]
69: 0x10e7ec0b0 std::process::id::h056ebf195753a258 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
70: 0x10e7ea020 std::process::id::h056ebf195753a258 [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
71: 0x10e7a1bd4 std::sys::pal::unix::os::errno::h12fdde1c2329795a [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
72: 0x10e76cf04 std::sys::pal::unix::os::errno::h12fdde1c2329795a [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
73: 0x10d93c818 v8::internal::compiler::CompilationDependencies::FieldTypeDependencyOffTheRecord(v8::internal::compiler::MapRef, v8::internal::compiler::MapRef, v8::internal::InternalIndex, v8::internal::compiler::ObjectRef) const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
74: 0x10d93e298 v8::internal::compiler::CompilationDependencies::FieldTypeDependencyOffTheRecord(v8::internal::compiler::MapRef, v8::internal::compiler::MapRef, v8::internal::InternalIndex, v8::internal::compiler::ObjectRef) const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
75: 0x10d93a14c v8::internal::compiler::CompilationDependencies::FieldTypeDependencyOffTheRecord(v8::internal::compiler::MapRef, v8::internal::compiler::MapRef, v8::internal::InternalIndex, v8::internal::compiler::ObjectRef) const [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
76: 0x10b7b766c v8::CodeEvent::GetScriptLine() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
77: 0x10b7b86cc v8::CodeEvent::GetScriptLine() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
78: 0x10b7b8544 v8::CodeEvent::GetScriptLine() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
79: 0x10b7b6e84 v8::CodeEvent::GetScriptLine() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
80: 0x10b7b7048 v8::CodeEvent::GetScriptLine() [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
81: 0x10b49052c ElectronMain [/Applications/NoteChat.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
82: 0x181c4c274 start [/usr/lib/dyld]