dat-node

dat-node copied to clipboard

I am getting this error when trying to close a dat that i had just created and am the owner of. Is this something that I am doing wrong - or a problem within dat-node?

    at /Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/recursive-watch/index.js:23:5
    at EventEmitter.destroy (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/mirror-folder/index.js:257:21)
    at closeFileWatch (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/dat-node/dat.js:220:19)
    at Dat.close (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/dat-node/dat.js:203:3)
    at _bluebird2.default (/Users/b/documents/84.DatLibrary/dat-cardcat/src/dat.js:272:54)
    at Promise._execute (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/bluebird/js/release/debuggability.js:300:9)
    at Promise._resolveFromExecutor (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/bluebird/js/release/promise.js:483:18)
    at new Promise (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/bluebird/js/release/promise.js:79:10)
    at DatWrapper.close (/Users/b/documents/84.DatLibrary/dat-cardcat/src/dat.js:272:12)
    at Context.done (/Users/b/documents/84.DatLibrary/dat-cardcat/test/dat.test.js:118:18)
    at callFnAsync (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/mocha/lib/runnable.js:371:21)
    at Hook.Runnable.run (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/mocha/lib/runnable.js:318:7)
    at next (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/mocha/lib/runner.js:309:10)
    at Immediate.<anonymous> (/Users/b/documents/84.DatLibrary/dat-cardcat/node_modules/mocha/lib/runner.js:339:5)
    at runCallback (timers.js:637:20)
    at tryOnImmediate (timers.js:610:5)
    at processImmediate [as _immediateCallback] (timers.js:582:5)```

@e-e-e looks like a bug in recursive-watch. Can you paste the code that is causing this?

The fs.lstat is failing on the directory/file you're sharing. The clear function should get set. We should probably catch that error but it'll be helpful to know what your case is.

There may be an option in dat-node we should be requiring but aren't enforcing.

@joehand here is a snipet that i used to replicate it - it has to do with the watch options, as the original tests which were causing me issues was fixed as a side effect of removing it.

const createDatAsync = Promise.promisify(createDat);
const temporaryDir = './temp';

describe.only('replicate', () => {
  let close;
  let externalLibraryKey;

  before((done) => {
    temp.track();
    shareLibraryDat((err, shareKey, closeShare) => {
      close = closeShare;
      externalLibraryKey = shareKey;
      done(err);
    });
  });
  after((done) => {
    temp.cleanupSync();
    close(done);
  });

  it('breaks', () => {
    const opts = {
      latest: true,
      indexing: true,
    };
    const tmpPath = temp.mkdirSync(temporaryDir);
    return createDatAsync(tmpPath, opts)
      .then((dat) => {
        const importOpts = {
          key: externalLibraryKey,
          watch: true,
          count: true,
          dereference: true,
          indexing: true,
        };
        dat.importFiles(importOpts, () => {
          console.log('finished importing');
        });
        return new Promise(resolve => dat.close(resolve));
      });
  });
});

I thought it may also be a result of temp cleaning up the directory before the dat is properly closed, but it is even caused by us waiting until close is successful before cleaning up.

I thought it may also be a result of temp cleaning up the directory before the dat is properly closed, but it is even caused by us waiting until close is successful before cleaning up.

Ya that would have been my guess too, since close relies on fs.lstat(temp) here. I'll see if I can reproduce.

cool if its temp - it could be that the dat close callback is firing before the the mirroring process is closed properly.

I misread before, clear should be set when dat.importFiles is called. Can you see what happens here if you run lstat before import:

 return createDatAsync(tmpPath, opts)
      .then((dat) => {
        const importOpts = {
          key: externalLibraryKey,
          watch: true,
          count: true,
          dereference: true,
          indexing: true,
        };
        fs.lstat(temp, (err, stat) {
          console.log(err, stat) // what happens here?
          dat.importFiles(importOpts, () => {
            console.log('finished importing');
          });
        })
        return new Promise(resolve => dat.close(resolve));
      });

If nothing looks wrong there, can you add a console log for the lstat values in recurisve-watch. Sorry, still kind of searching in the dark here, I haven't been able to reproduce this.

Sorry for the slow reply - I ran fs.lstat on tempDir, and it returns fine

null { dev: 16777217,
  mode: 16832,
  nlink: 3,
  uid: 501,
  gid: 20,
  rdev: 0,
  blksize: 4096,
  ino: 37143144,
  size: 102,
  blocks: 0,
  atime: 2017-06-21T08:10:51.000Z,
  mtime: 2017-06-21T08:10:51.000Z,
  ctime: 2017-06-21T08:10:51.000Z,
  birthtime: 2017-06-21T08:10:51.000Z }

but the clear error is still raised.

@joehand perhpas you can see its replicate for you by running the test here https://github.com/sdockray/dat-cardcat/blob/master/test/issue.test.js It could also just be our code causing it in some weird way.