.jscpd.json does not ignore file(s)

[aregtech]

Describe the bug Created .jscpd.json file in the project root as it is described here. In the section "ignore" left only "**/*.md" Still getting failure with error message ERROR: jscpd found too many duplicates (35.55%) over threshold (0%)

To Reproduce Steps to reproduce the behavior:

1. Go to branch https://github.com/aregtech/areg-sdk/tree/20211009-readme
2. Check the content of file .jscpd.json
3. Open to edit file README.md. Type anything is some place and commit.
4. Wait a little bit, go to Action menu and see the result.

Expected behavior Regardless it is absolutely the root cause of the failure, but my expectation is that mega-linter ignores all *.md files.

Screenshots

Desktop (please complete the following information):

• OS: Ubuntu
• OS Version: 20.04.3, LTS
• NodeJS Version: I have no idea, i don't use JS
• jscpd version: super-linter:v4.8.1

Run github/super-linter@v4
/usr/bin/docker run --name ghcriogithubsuperlinterv481_686139 --label fa4e14 --workdir /github/workspace --rm -e VALIDATE_ALL_CODEBASE -e DEFAULT_BRANCH -e GITHUB_TOKEN -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/areg-sdk/areg-sdk":"/github/workspace" ghcr.io/github/super-linter:v4.8.1
--------------------------------------------------------------------------------

Additional context

[kucherenko]

will check and fix the issue, thank you for the report

[paulalex]

Could you confirm if this issue is affecting me? I started using this action today and I want to exclude directories from being validated:

{
    "threshold": 0,
    "ignore": [
      "**/target/**",
      "**/resources/**"
    ]
}

It seems regardless of the pattern I use I am unable to exclude files, I am not sure if this is related to this, or if I am simply doing something wrong.

[Kurt-von-Laven]

I suspect these matters as well as #466 may all be addressed in one shot by replacing the buggy gitignore-to-glob with git check-ignore and not bothering to support the gitignore feature when Git is not installed. Alternatively, it might be a matter of switching to [email protected]?

[kucherenko]

@Kurt-von-Laven will try to use the lib

[Kurt-von-Laven]

Apologies, I just realized this issue pertains to .jscpd.json, not .gitignore. In any case, it looks like there is also @gerhobbelt/gitignore-parser, which unlike parse-gitignore appears to still suit your purposes in the latest release, not only the previous major version. Just saw the note on your README. Best wishes to you and your family.

[wulfland]

So right now there is no way to ignore files? This makes superlinter unusable for me. Or am I missing something?

[wulfland]

Never mind. I found how I can disable jcpd in superlinter:

      - name: Lint Code Base
        uses: github/super-linter@v4
        env:
          VALIDATE_JSCPD: false

[matthew-dean]

I'm also seeing an issue where ignore file patterns are not ignored.

[matthew-dean]

Aha! I've figured out the issue. If you run jscpd like jscpd --pattern 'src/**/*' and then you have an ignore pattern like src/app/data/**/*, jscpd will incorrectly resolve src/app as relative to (I believe) the source pattern, vs the root. But if you change the pattern to **/app/data/**/* it works.

I believe this is a bug in jscpd.

[soullivaneuh]

Have the issue using v3.5.0. Seems to works fine with v3.4.5.

[kucherenko]

@soullivaneuh could you please provide steps to reproduce, I didn't touch this part of jscpd

[soullivaneuh]

My config file:

{
  "ignore": ["src/services/**/*.service.ts"]
}

The command with the not expected behavior:

❯ npx [email protected] src
Need to install the following packages:
  [email protected]
Ok to proceed? (y) 
Clone found (typescript):
 - src/services/children/children.service.ts [6:19 - 18:12] (12 lines, 88 tokens)
   src/services/credit-cards/credit-cards.service.ts [6:23 - 18:16]

┌────────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format     │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ typescript │ 100            │ 4923        │ 40146        │ 1            │ 12 (0.24%)       │ 88 (0.22%)        │
├────────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total:     │ 100            │ 4923        │ 40146        │ 1            │ 12 (0.24%)       │ 88 (0.22%)        │
└────────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 1 clones.
Detection time:: 516.723ms

But I have a weird behavior with v3.4.5 using just npx jscpd without specifing the version works and return the same version. But specifying it reproduce also the bug. Please see the following:

❯ npx jscpd src
┌────────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format     │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ typescript │ 84             │ 4641        │ 37741        │ 0            │ 0 (0%)           │ 0 (0%)            │
├────────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total:     │ 84             │ 4641        │ 37741        │ 0            │ 0 (0%)           │ 0 (0%)            │
└────────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 0 clones.
Detection time:: 482.751ms
❯ npx jscpd --version
3.4.5
❯ npx [email protected] src
Clone found (typescript):
 - src/services/children/children.service.ts [6:19 - 18:12] (12 lines, 88 tokens)
   src/services/credit-cards/credit-cards.service.ts [6:23 - 18:16]

┌────────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format     │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ typescript │ 100            │ 4923        │ 40146        │ 1            │ 12 (0.24%)       │ 88 (0.22%)        │
├────────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total:     │ 100            │ 4923        │ 40146        │ 1            │ 12 (0.24%)       │ 88 (0.22%)        │
└────────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 1 clones.
Detection time:: 503.91ms

[soullivaneuh]

I also confirm the workaround of @matthew-dean changing the configuration like so removes the report:

diff --git a/.jscpd.json b/.jscpd.json
index b75b178..54e242f 100644
--- a/.jscpd.json
+++ b/.jscpd.json
@@ -1,3 +1,3 @@
 {
-  "ignore": ["src/services/**/*.service.ts"]
+  "ignore": ["**/services/**/*.service.ts"]
 }

However, I assume this is not exactly what we need, as it will match any services base path, not only the one from src.

[kucherenko]

Thank you, will investigate and fix

[rafaelmaeuer]

Hello this still seem to be an issue, I am using super-linter and getting several Clone founds errors in a yaml file. All attempts to ignore the file with a .jscpd.json failed, is there any solution/workaround to successfully ignore a .yaml-file???

First try:

 {
  "ignore": ["**/filename.yaml"]
 }

Second try:

{
  "ignore": ["**/*.yaml"]
}