core icon indicating copy to clipboard operation
core copied to clipboard

Published 20 hours ago • verified publisher icon opensumi

[BUG] 打开文件导致后端进程退出

Open sleepyyj opened this issue 2 years ago • 6 comments

描述你的问题（Describe the bug） 使用 ide-startup 测试，前端打开一个文件后，后端进程由于 OOM 退出。

<--- Last few GCs --->

[30947:0x130008000]   536403 ms: Mark-sweep 394.3 (434.6) -> 370.8 (410.2) MB, 71.1 / 0.0 ms  (+ 0.0 ms in 3 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 396 ms) (average mu = 0.999, current mu = 0.853) finalize incre[30947:0x130008000]   539844 ms: Mark-sweep 1714.8 (1754.3) -> 1546.8 (1586.2) MB, 4.0 / 0.0 ms  (+ 0.0 ms in 3 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 3118 ms) (average mu = 0.999, current mu = 0.999) finalize i

<--- JS stacktrace --->

 3: 0x1024e0000 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x1024dff94 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x1026636ac v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/usr/local/bin/node]
 6: 0x10286ffb8 v8::internal::Handle<v8::internal::NumberDictionary> v8::internal::HashTable<v8::internal::NumberDictionary, v8::internal::NumberDictionaryShape>::EnsureCapacity<v8::internal::LocalIsolate>(v8::internal::LocalIsolate*, v8::internal::Handle<v8::internal::NumberDictionary>, int, v8::internal::AllocationType) [/usr/local/bin/node]
 7: 0x102870830 v8::internal::Handle<v8::internal::NumberDictionary> v8::internal::Dictionary<v8::internal::NumberDictionary, v8::internal::NumberDictionaryShape>::Add<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::NumberDictionary>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, v8::internal::InternalIndex*) [/usr/local/bin/node]
 8: 0x1027b4908 v8::internal::(anonymous namespace)::DictionaryElementsAccessor::AddImpl(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, unsigned int) [/usr/local/bin/node]
 9: 0x1027b37dc v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::DictionaryElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)12>>::Add(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, unsigned int) [/usr/local/bin/node]
10: 0x10283d850 v8::internal::JSObject::AddDataElement(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [/usr/local/bin/node]
11: 0x102880a04 v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::Maybe<v8::internal::ShouldThrow>, v8::internal::StoreOrigin) [/usr/local/bin/node]
12: 0x10287f5c4 v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::StoreOrigin, v8::Maybe<v8::internal::ShouldThrow>) [/usr/local/bin/node]
13: 0x10297f044 v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::StoreOrigin, v8::Maybe<v8::internal::ShouldThrow>) [/usr/local/bin/node]
14: 0x1026fd498 v8::internal::Runtime_KeyedStoreIC_Slow(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
15: 0x102c8cbcc Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/usr/local/bin/node]
16: 0x1071cb2dc 
17: 0x102c1e64c Builtins_JSEntryTrampoline [/usr/local/bin/node]
18: 0x102c1e2e4 Builtins_JSEntry [/usr/local/bin/node]
19: 0x1025efe94 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/usr/local/bin/node]
20: 0x1025ef528 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/usr/local/bin/node]
21: 0x1027554c4 v8::internal::JsonStringifier::ApplyToJsonFunction(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
22: 0x10275c030 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
23: 0x10275e74c v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
24: 0x10275adf8 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
25: 0x1027549bc v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
26: 0x102569f40 v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
27: 0x102c8cd0c Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/usr/local/bin/node]
28: 0x1073a9974 
29: 0x107394f78 
30: 0x10733a4c8 
31: 0x102cd4cf8 Builtins_PromiseFulfillReactionJob [/usr/local/bin/node]
32: 0x102c42234 Builtins_RunMicrotasks [/usr/local/bin/node]
33: 0x102c1e524 Builtins_JSRunMicrotasksEntry [/usr/local/bin/node]
34: 0x1025efe5c v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/usr/local/bin/node]
35: 0x1025f0290 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/usr/local/bin/node]
36: 0x1025f037c v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*, v8::internal::MaybeHandle<v8::internal::Object>*) [/usr/local/bin/node]
37: 0x102612fb8 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/usr/local/bin/node]
38: 0x10261384c v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [/usr/local/bin/node]
39: 0x1022dddb4 node::InternalCallbackScope::Close() [/usr/local/bin/node]
40: 0x1022de48c node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/usr/local/bin/node]
41: 0x1022f34c8 node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/usr/local/bin/node]
42: 0x1023949d0 node::fs::FSReqCallback::Resolve(v8::Local<v8::Value>) [/usr/local/bin/node]
43: 0x102395128 node::fs::AfterNoArgs(uv_fs_s*) [/usr/local/bin/node]
44: 0x102bfebd4 uv__work_done [/usr/local/bin/node]
45: 0x102c02370 uv__async_io [/usr/local/bin/node]
46: 0x102c140e8 uv__io_poll [/usr/local/bin/node]
47: 0x102c02800 uv_run [/usr/local/bin/node]
48: 0x1022deccc node::SpinEventLoop(node::Environment*) [/usr/local/bin/node]
49: 0x1023c9aec node::NodeMainInstance::Run(int*, node::Environment*) [/usr/local/bin/node]
50: 0x1023c97b8 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/usr/local/bin/node]
51: 0x1023635c8 node::Start(int, char**) [/usr/local/bin/node]
52: 0x19bbfbf28 start [/usr/lib/dyld]

复现路径（To Reproduce） 1、启动 ide-startup 2、添加一个文件到 workspace，文件大小根据机器配置 20M~200M 不等 3、前端打开这个文件

预期表现（Expected behavior） 能正常打开文件

环境信息（Environment）

  • OS: macOS 13.5.1
  • Browser: Chrome
  • OpenSumi Version: 2.23.2

sleepyyj avatar Oct 16 '23 03:10 sleepyyj

@sleepyyj 是否测试过如 2.26，2.27 版本可不可以正常使用？看了一下目前集成的版本是比较旧的

erha19 avatar Oct 17 '23 03:10 erha19

@sleepyyj 是否测试过如 2.26，2.27 版本可不可以正常使用？看了一下目前集成的版本是比较旧的

我测一下

sleepyyj avatar Oct 17 '23 08:10 sleepyyj

@erha19 2.26.8 和 2.27.0 都是一样的报错

sleepyyj avatar Oct 17 '23 09:10 sleepyyj

@sleepyyj 这里应该存在大文件打开时的内存泄漏问题，（即部分服务端环境下的机器确实无法承载大文件的语法高亮、缓存等）暂时你可以先通过设置 editor.largeFile 小一些来控制其展示，同时关闭大文件内的语法高亮来减少 OOM 问题，配置项如：

  'editor.languageFeatureEnabledMaxSize': {
    type: 'number',
    default: 4 * 1024 * 1024 * 1024, // 4096 MB
    description: '%editor.configuration.languageFeatureEnabledMaxSize%',
  },
  // 会同步到插件进程的最大文件尺寸, 必须大于等于 languageFeatureEnabledMaxSize
  'editor.docExtHostSyncMaxSize': {
    type: 'number',
    default: 4 * 1024 * 1024 * 1024, // 4096 MB
    description: '%editor.configuration.docExtHostSyncMaxSize%',
  },

相关配置项：

https://github.com/opensumi/core/blob/d10597dd539a36aa31012f091b16c71ed80ed252/packages/editor/src/browser/preference/schema.ts#L1474-L1484

降低这类配置会让宿主机器的表现会好一些

erha19 avatar Oct 18 '23 03:10 erha19

@erha19 把配置大小改为 20MB 后，打开文件提示 The file is too large, continuing to open it may cause it to jam or crash.，点击 still open it。146MB 的文件无法打开，没有出现OOM，但是 readFile 消息没有响应。263MB 的文件依然 OOM。

sleepyyj avatar Oct 18 '23 06:10 sleepyyj

@sleepyyj 在 opensumi/core 中也复现了这个问题，本质原因是目前框架使用了 jsonRPC 这一通信结构，好处是在 WS 中能方便的进行问题调试，坏处就是像这里这种情况（或者说文件读取情况），读取文件获得的 Buffer 数据在经过 JSON.stringify 及 JSON.parse 过程会耗费大量的内存，导致内存泄漏问题。

该问题要彻底解决，需要通过对通信层协议进行改造来解决这一问题，在 架构改进 中有这块规划，但目前应该没有进度。

相关问题代码见：

https://github.com/opensumi/core/blob/73a869a50919a03768c1c707011ec28408950129/packages/connection/src/common/message.ts#L75

cc @bytemain

erha19 avatar Oct 23 '23 10:10 erha19