nx icon indicating copy to clipboard operation
nx copied to clipboard

fix(core): hash git-ignored files when explicitly listed as inputs

Open ZackDeRose opened this issue 3 weeks ago • 3 comments

Current Behavior

When a file is listed as an input for a task but is git-ignored (e.g., .env, secrets.json), its contents are not included in the hash computation. This causes incorrect cache hits when these git-ignored input files change.

Example:

{
  "targets": {
    "build": {
      "inputs": ["{projectRoot}/.env"],
      "cache": true
    }
  }
}

If .env is in .gitignore, changes to .env do NOT invalidate the cache (bug).

Expected Behavior

Files explicitly listed as inputs should be hashed regardless of git-ignore status. When a git-ignored input file changes, the cache should be invalidated.

Changes Made

Core Fix (Rust)

  • Modified hash_project_files() in packages/nx/src/native/tasks/hashers/hash_project_files.rs

    • Added hash_missing_files() helper that walks the filesystem without git-ignore
    • Finds files matching input patterns that were filtered out by git-ignore
    • Hashes these files directly from the filesystem
  • Modified hash_workspace_files() in packages/nx/src/native/tasks/hashers/hash_workspace_files.rs

    • Same approach for workspace-level inputs

Tests

  • Added Rust unit test: should_hash_git_ignored_files_when_explicitly_specified
  • Added e2e test in cache.test.ts: should invalidate cache when git-ignored input file changes

Important Note for Reviewers ⚠️

This implementation hashes all git-ignored files that match the input patterns, including broad wildcards like {projectRoot}/**/*.

Potential concerns:

  1. Performance: Walks the directory twice (once with gitignore, once without)
  2. Broad patterns: If someone uses {projectRoot}/**/* (the default), ALL git-ignored files will now be hashed

Should we refine this to:

  • Only hash git-ignored files for explicit patterns (no wildcards)?
  • Add caching to avoid double directory walking?
  • Optimize walk scope to specific subdirectories?

Would appreciate team feedback on whether the current approach is acceptable or if we should implement the refinements mentioned above.

Test Plan

  • [x] Rust unit test passes
  • [x] E2e test added (demonstrates the fix)
  • [ ] Run full e2e test suite
  • [ ] Performance testing with large monorepos

🤖 Generated with Claude Code

ZackDeRose avatar Nov 28 '25 21:11 ZackDeRose

Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
Latest commit 16821753b6bf87f4a19c80325fd54abe1565b7f9

netlify[bot] avatar Nov 28 '25 21:11 netlify[bot]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nx-dev Ready Ready Preview Nov 28, 2025 9:33pm

vercel[bot] avatar Nov 28 '25 21:11 vercel[bot]

View your CI Pipeline Execution ↗ for commit 16821753b6bf87f4a19c80325fd54abe1565b7f9

Command Status Duration Result
nx affected --targets=lint,test,test-kt,build,e... ❌ Failed 38m 37s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 2m 42s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 11s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 2s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2025-11-28 22:16:51 UTC

nx-cloud[bot] avatar Nov 28 '25 21:11 nx-cloud[bot]