copy is not following symlinks correctly, only creates empty directory

[tobyroworth]

When copying a symlink, I've usually expected the contents of the link's target directory to be copied. Since updating to Grunt 1.0 an empty directory with the name of the symlink is created, with no contents.

Sample gruntfile.js:

module.exports = function(grunt) {
    require('jit-grunt')(grunt);
    // Project configuration.
    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),
        copy: {
            dist: {
                files: [
                    {
                        expand: true,
                        src: 'test3',
                        dest: 'test1'
                    }
                ]
            }
        }
    });
    // Default task(s).
    grunt.registerTask('default', ['copy:dist']);

};

link created with ln -s test2 test3 test2 contains file1 and file2

Folder structure before running grunt (ls -lR test*):

lrwxrwxrwx  1 ubuntu ubuntu    5 Apr 21 15:50 test3 -> test2/
lrwxrwxrwx 1 ubuntu ubuntu    5 Apr 21 15:50 test3 -> test2/

test1:
total 0

test2:
total 0
-rw-r--r-- 1 ubuntu ubuntu 0 Apr 21 15:47 file1
-rw-r--r-- 1 ubuntu ubuntu 0 Apr 21 15:47 file2

Folder structure after running grunt (ls -lR test*):

lrwxrwxrwx 1 ubuntu ubuntu    5 Apr 21 15:50 test3 -> test2/

test1:
total 4
drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 16:03 test3/

test1/test3:
total 0

test2:
total 0
-rw-r--r-- 1 ubuntu ubuntu 0 Apr 21 15:47 file1
-rw-r--r-- 1 ubuntu ubuntu 0 Apr 21 15:47 file2

test1/test3 should contain file1 and file2, but is empty. It's also not a symlink to test2, which would be kind of understandable.

[tobyroworth]

In https://github.com/gruntjs/grunt/blob/master/lib/grunt/file.js file.copy runs file.isDir on line 297, but doesn't do antyhing special with symlinks

[hrj]

I am seeing the same behaviour after upgrading grunt to 1.0.1. However, should this be reported to grunt, instead of this package?

[karllhughes]

@hrj same here. Did you file an issue with the Grunt package? If not, I'll go ahead and do it. I don't know much about the inner workings of Grunt, but I'm happy to help with suggesting a fix.

[tobyroworth]

@karllhughes I never reported it in the grunt package - that line I mentioned could be the culprit, but I didn't have time to do any proper testing on it.

I wasn't 100% sure it wasn't a quirk of the copy package, so thought it best to start here, although copy seems to mostly just call file.copy.

[karllhughes]

@tobyroworth I actually think it is a deficiency in this package (possibly based on changes in the primary grunt package).

The grunt package includes methods for isDir and isLink, so I'm thinking we could add a conditional in this project to check if the file is a symlink, and if so to copy it over anyway.

I'm working on a fix to propose. I've never worked with grunt internals before though, so it'll take me a little time to figure out how exactly everything works.

[tobyroworth]

@karllhughes sound like a good start. You might need to recurse manually though, to get around the check in the grunt package.

I think there's a file.recurse, but I might be thinking of a Go library!

https://github.com/gruntjs/grunt/issues/371 is worth a look, as it covers the opposite of this issue (and explicitly states file.copy should follow symlinks)

[karllhughes]

@tobyroworth just out of curiosity, were you testing this on Mac or Linux?

I pulled the project down and wrote a symlink test on my Mac and it seems to be working. Then I tried it on my Linux box and it didn't work. Could be an issue with the way Linux handles symlinks? Also could be a Node thing (I'm using Node 6 on my Mac, but Node 4 on the Ubuntu server).

[tobyroworth]

@karllhughes tested on Linux (on cloud9 - have a test workspace I used to verify the bug report if you're interested: (https://c9.io/tobyroworth/grunt-copy-test)

I updated grunt (to 1) and node (to 5, I think - whatever "stable" was at the time) at the same time, so not sure if it's a node thing or a grunt thing, but the problem happened in fresh container, with only what was necessary installed/updated, and an old one I was developing in.

cp, ls etc work as expected, but I guess that doesn't mean it doesn't upset grunt somehow.

I'll try updating node and see what happens

[tobyroworth]

Nope, node 6.1.0 doesn't fix it on Linux

[CodySchaaf]

I'm also having the issue on my Mac with both node 5 and 6. It only breaks when updating grunt from 0.4.x to 1.0.1. Did anyone report in that project?

[mwsam]

This is actually caused by a breaking change in node-glob which is the underlying module used by grunt, see isaacs/node-glob#139.

So to get back the old behavior of following symlinks simply specify follow:true as part of file matching option, for which grunt will pass along to node-glob according to http://gruntjs.com/configuring-tasks#files.

[tobyroworth]

@mwsam just tried it, and it didn't seem to make a difference:

       copy: {
            dist: {
                files: [
                    {
                        follow: true,
                        expand: true,
                        src: 'test3',
                        dest: 'test1'
                    }
                ]
            }
        }

[mwsam]

It seems follow:true is effective only for ** matching. I faced the same empty directory problem as you but my config is a bit different:

        files: [{
          expand: true,
          follow: true,
          cwd: 'client/',
          src: '**',
          dest: 'build/client/'
        }]

A symlinked directory under client/ will just be copied as empty directory if follow:true is not used.

[Flolagale]

Adding follow: truedid the trick for us, indeed with a **style matching rule.