dat share crashes with a very large file

[tiagoantao]

I have a large directory to transfer, composed of 1.25M files. This currently does not work because dat has problems with lots of files. But it also does not work if I just transfer a single zip file, because it crashes. What I want to transfer is the following:

$ ls -lh
total 240G
-rw-rw-r-- 1 tiago_antao tiago_antao  30G Mar 20 16:46 home.zip
-rw-rw-r-- 1 tiago_antao tiago_antao 210G Mar 20 22:42 scratch.zip

I end up with this:

$ dat share
dat v13.10.0
Created new dat in /core/galaxy/xylemsap/resl/.dat
dat://082b0e4d517996373c4da7e9f92c1d39fd0eb31801e36c8b4578dce540d36262
Sharing dat: 1 files (31 GB)

0 connections | Download 0 B/s Upload 0 B/s

Creating metadata for 2 files (249 KB/s)
[=====================---------------------] 50%
ADD: scratch.zip (98 GB / 225 GB)


Ctrl+C to Exit

<--- Last few GCs --->

[445067:0x30df8b0] 13676704 ms: Mark-sweep 1399.6 (1437.5) -> 1399.6 (1437.5) MB, 1165.4 / 0.0 ms  allocation failure GC in old space requested
[445067:0x30df8b0] 13678523 ms: Mark-sweep 1399.6 (1437.5) -> 1399.6 (1429.0) MB, 1818.8 / 0.0 ms  last resort GC in old space requested
[445067:0x30df8b0] 13679787 ms: Mark-sweep 1399.6 (1429.0) -> 1399.6 (1429.0) MB, 1263.6 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2084efa254d9 <JSObject>
    1: _onTimeout [/usr/lib/node_modules/dat/node_modules/multicast-dns/index.js:~122] [pc=0x153d63b0ca5c](this=0x989a9fc2671 <Timeout map = 0x3fb9ab49e031>)
    2: listOnTimeout [timers.js:~214] [pc=0x153d63b46fa0](this=0x989a9fc2729 <Timer map = 0x3fb9ab4b66b1>)

==== Details ================================================

[1]: _onTimeout [/usr/lib/node_modules/dat/node_modules/multicast-dns...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [dat]
 2: 0x8c807c [dat]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [dat]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [dat]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [dat]
 6: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::NewInternal(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [dat]
 7: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [dat]
 8: v8::internal::HashTable<v8::internal::StringTable, v8::internal::StringTableShape>::EnsureCapacity(v8::internal::Handle<v8::internal::StringTable>, int, v8::internal::PretenureFlag) [dat]
 9: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [dat]
10: v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::LookupIterator::Configuration) [dat]
11: v8::Object::Has(v8::Local<v8::Context>, v8::Local<v8::Value>) [dat]
12: 0x91ee15 [dat]
13: 0x153d63b79b27
Aborted (core dumped)