memfs icon indicating copy to clipboard operation
memfs copied to clipboard

Published 20 hours ago • verified publisher icon streamich

`readdir` on a symlinked dir fails with `ENOTDIR: not a directory, scandir '/foo'`

Open vjpr opened this issue 7 years ago • 14 comments
trafficstars 

const {fs} = require('memfs')
const realFs = require('fs')

realFs.readdirSync('/')
realFs.symlinkSync('/', '/foo')
realFs.readdirSync('/')
realFs.readdirSync('/foo') // works

console.log('works')

fs.readdirSync('/')
fs.symlinkSync('/', '/foo')
fs.readdirSync('/')
fs.readdirSync('/foo') // fails

https://runkit.com/vjpr/5a87383e1ff7740012733bd1

vjpr avatar Feb 16 '18 20:02 vjpr

Please see: https://github.com/streamich/memfs/pull/101

I think I fixed it with this line: https://github.com/streamich/memfs/pull/101/commits/0cf18dd3784f7079a8647d930edb5cf512bbcf21#diff-237b96688267edc5b8448d6322f36d8dL1477

streamich avatar Feb 16 '18 21:02 streamich

@streamich BTW master version is 2.5.10. Latest published is 2.7.0.

vjpr avatar Feb 16 '18 21:02 vjpr

Also, could you publish this version? Or add a post-install build step so that I can install from git.

vjpr avatar Feb 16 '18 21:02 vjpr

When using globby when I run:

import glob from 'globby'
vol.writeFileSync('/app/foo.json', 'foo')
glob.sync('/app')
vol.symlinkSync('/app', '/app/foo')
glob.sync('/app') // This is empty array.

The second glob returns an empty array.

However, fs.readdirSync now works when reading the symlinked dir.

vjpr avatar Feb 16 '18 21:02 vjpr

semantic-release should publish it automatically, hmmm, but it looks like it does not, I'll take a look.

streamich avatar Feb 16 '18 21:02 streamich

with regards to globby:

How do you tell it to use memfs?

streamich avatar Feb 16 '18 21:02 streamich

patcher.js

import {Volume} from 'memfs'
import {patchFs} from 'fs-monkey'
import {ufs} from 'unionfs'
import * as originalFs from 'fs'

let vol

function start() {
  vol = new Volume()
  ufs.use(vol).use(originalFs)
  patchFs(ufs)
}

start()

export function reset() {
  vol.reset()
}

export {vol}

import {vol} from './patcher'

// do stuff with vol

vol.reset()

NOTE: globby uses node-glob

vjpr avatar Feb 16 '18 21:02 vjpr

Published manually under 2.7.1, will take a look at globby.

streamich avatar Feb 16 '18 22:02 streamich

fast-glob internally uses readdir-enhanced and that seems to work:

const glob = require('globby');
const {Volume} = require('../lib/volume');
const {patchFs} = require('fs-monkey');
const {ufs} = require('unionfs');
const fs = require('fs');
var readdir = require('readdir-enhanced');

let vol

function start() {
  vol = new Volume()
  ufs.use(fs).use(vol);
  patchFs(ufs)
}

start()

function reset() {
  vol.reset()
}

// console.log('vol', vol);

vol.mkdirpSync('/app2');
vol.writeFileSync('/app2/foo.json', 'foo')
var files = readdir.sync('/app2');
console.log(files);

So, it must be something in globby or fast-glob, or glob.

Also globby has both, fast-glob and glob in its dependencies.

streamich avatar Feb 16 '18 22:02 streamich

fast-glob does not seem to work with memfs, don't know why, giving it up for now.

streamich avatar Feb 16 '18 23:02 streamich

fast-glob is swallowing errors in node_modules/fast-glob/out/providers/reader-sync.js:38.

Here is the error it was swallowing:

   { Error: ENOENT: no such file or directory, lstat '/dev/fd/12'
    at createError (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/memfs/2.7.1/node_modules/memfs/lib/volume.js:105:17)
    at throwError (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/memfs/2.7.1/node_modules/memfs/lib/volume.js:114:11)
    at Volume.lstatBase (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/memfs/2.7.1/node_modules/memfs/lib/volume.js:1116:13)
    at Volume.lstatSync (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/memfs/2.7.1/node_modules/memfs/lib/volume.js:1120:21)
    at Union.syncMethod (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/unionfs/3.0.2/node_modules/unionfs/lib/union.js:61:35)
    at Union.this_1.(anonymous function) [as lstatSync] (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/unionfs/3.0.2/node_modules/unionfs/lib/union.js:17:30)
    at exports.lstat (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/fs.js:58:20)
    at Object.safeCall [as safe] (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:24:8)
    at stat (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/stat.js:19:8)
    at DirectoryReader.processItem (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:178:5)
    at array.forEach.item (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/for-each.js:14:5)
    at Array.forEach (<anonymous>)
    at Object.syncForEach [as forEach] (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/for-each.js:13:9)
    at call.safe (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:87:16)
    at onceWrapper (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:45:17)
    at onceWrapper (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:45:17)
    at exports.readdir (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/fs.js:19:5)
    at Object.safeCall [as safe] (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:24:8)
    at DirectoryReader.readNextDirectory (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:78:10)
    at Readable.DirectoryReader.stream._read (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:57:18)
    at Readable.read (_stream_readable.js:442:10)
    at readdirSync (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/index.js:27:21)
    at Function.readdirSyncStat (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/index.js:34:10)
    at ReaderSync.api (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/providers/reader-sync.js:24:24)
    at ReaderSync.read (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/providers/reader-sync.js:33:32)
    at Array.map (<anonymous>)
    at getWorks (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/index.js:18:18)
    at Function.sync (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/index.js:24:17)
    at Test.fn (xxx/xxx/index.test.js:45:18)
    at Test.callFn (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/test.js:305:18)
    at Test.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/test.js:318:23)
    at runNext (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at Concurrent.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/concurrent.js:41:37)
    at runNext (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at runNext (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at Bluebird.try (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/runner.js:224:48)
    at tryCatcher (xxx/xxx/node_modules/.registry.npmjs.org/bluebird/3.5.1/node_modules/bluebird/js/release/util.js:16:23)
    at Function.Promise.attempt.Promise.try (xxx/xxx/node_modules/.registry.npmjs.org/bluebird/3.5.1/node_modules/bluebird/js/release/method.js:39:29)
    at Runner.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/runner.js:224:22)
    at process.on.options (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/main.js:84:10)
    at emitOne (events.js:116:13)
    at process.emit (events.js:211:7)
    at process.on.message (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/process-adapter.js:14:10)
    at emitTwo (events.js:126:13)
    at process.emit (events.js:214:7)
    at emit (internal/child_process.js:772:12)
    at _combinedTickCallback (internal/process/next_tick.js:141:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
     code: 'ENOENT',
     prev:
      { Error: EBADF: bad file descriptor, lstat '/dev/fd/12'
    at Object.fs.lstatSync (fs.js:941:11)
    at Union.syncMethod (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/unionfs/3.0.2/node_modules/unionfs/lib/union.js:61:35)
    at Union.this_1.(anonymous function) [as lstatSync] (xxx/mock-fs-with-memfs/node_modules/.registry.npmjs.org/unionfs/3.0.2/node_modules/unionfs/lib/union.js:17:30)
    at exports.lstat (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/fs.js:58:20)
    at Object.safeCall [as safe] (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:24:8)
    at stat (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/stat.js:19:8)
    at DirectoryReader.processItem (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:178:5)
    at array.forEach.item (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/for-each.js:14:5)
    at Array.forEach (<anonymous>)
    at Object.syncForEach [as forEach] (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/for-each.js:13:9)
    at call.safe (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:87:16)
    at onceWrapper (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:45:17)
    at onceWrapper (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:45:17)
    at exports.readdir (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/fs.js:19:5)
    at Object.safeCall [as safe] (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:24:8)
    at DirectoryReader.readNextDirectory (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:78:10)
    at Readable.DirectoryReader.stream._read (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:57:18)
    at Readable.read (_stream_readable.js:442:10)
    at readdirSync (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/index.js:27:21)
    at Function.readdirSyncStat (xxx/xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/index.js:34:10)
    at ReaderSync.api (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/providers/reader-sync.js:24:24)
    at ReaderSync.read (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/providers/reader-sync.js:33:32)
    at Array.map (<anonymous>)
    at getWorks (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/index.js:18:18)
    at Function.sync (xxx/xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/index.js:24:17)
    at Test.fn (xxx/xxx/index.test.js:45:18)
    at Test.callFn (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/test.js:305:18)
    at Test.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/test.js:318:23)
    at runNext (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at Concurrent.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/concurrent.js:41:37)
    at runNext (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at runNext (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at Bluebird.try (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/runner.js:224:48)
    at tryCatcher (xxx/xxx/node_modules/.registry.npmjs.org/bluebird/3.5.1/node_modules/bluebird/js/release/util.js:16:23)
    at Function.Promise.attempt.Promise.try (xxx/xxx/node_modules/.registry.npmjs.org/bluebird/3.5.1/node_modules/bluebird/js/release/method.js:39:29)
    at Runner.run (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/runner.js:224:22)
    at process.on.options (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/main.js:84:10)
    at emitOne (events.js:116:13)
    at process.emit (events.js:211:7)
    at process.on.message (xxx/xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/process-adapter.js:14:10)
    at emitTwo (events.js:126:13)
    at process.emit (events.js:214:7)
    at emit (internal/child_process.js:772:12)
    at _combinedTickCallback (internal/process/next_tick.js:141:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
        errno: -9,
        code: 'EBADF',
        syscall: 'lstat',
        path: '/dev/fd/12',
        prev: null } } }

vjpr avatar Feb 17 '18 10:02 vjpr

Looks like its a fast-glob issue: https://github.com/mrmlnc/fast-glob/issues/62

vjpr avatar Feb 17 '18 11:02 vjpr

@streamich https://github.com/mrmlnc/fast-glob/issues/62#issuecomment-366525852

vjpr avatar Feb 18 '18 17:02 vjpr

@vjpr is there a minimal reproducible example? Because if I do this

const { Volume } = require('../src/volume');

const vol = new Volume;

vol.mkdirpSync('./app');
vol.writeFileSync('./app/foo.json', 'foo')
vol.symlinkSync('./app', './app/foo')

console.log('fs', vol.readdirSync('./app'));
console.log('fs', vol.readdirSync('./app/foo'));

it works fine

fs [ 'foo', 'foo.json' ]
fs [ 'foo', 'foo.json' ]

streamich avatar Feb 19 '18 16:02 streamich