Scoop icon indicating copy to clipboard operation
Scoop copied to clipboard
verified publisher icon ScoopInstaller

test(static-analysis): Add dot-sourcing dependency check

Open z-Fng opened this issue 1 month ago • 2 comments

Description

This PR makes the following changes:

  • test(static-analysis): Add dot-sourcing dependency check.
  • fix(dependency): Fix missing dot-sourcing dependency.

Static Analysis -- Dot-sourcing dependency check

A Function-Level Dot-Sourced Dependency Check is a static analysis mechanism that scans PowerShell scripts to determine which external scripts (modules or libraries) are dot-sourced (i.e., loaded via the . operator) and which functions depend on them.

How It Works

A typical implementation proceeds as follows:

  1. Parse the script into an AST (Abstract Syntax Tree)

    • Get-ScriptAst: The script is parsed into an AST structure, enabling static inspection of function definitions and calls.

  2. Extract function definitions and invocations

    • Get-ScriptFunction: Collect all functions defined in the current script.
    • Get-ScriptFunctionCall: Collect all functions called in the current script.
    • Get-ScriptFunctionCallMap: Collect all function calls inside each function body.
    • ~~Skip system-level cmdlets and built-in commands (retrieved via $cmdlet, $function, $alias, or Get-Command -CommandType Cmdlet).~~ This step was removed because executing Get-Command in a GitHub workflow takes too long.

  3. Resolve dot-sourced dependencies

    • Get-ImportedDependency: Detect all . expressions (dot-sourcing operators). Resolve these file paths (handling $PSScriptRoot, using: prefixes, etc.). Record each dot-sourced file as a potential provider of external functions.

  4. Cross-link function calls to definitions, Traverse dependencies (BFS)

    • Get-DotsourcingDependency: For each function call, determine whether it refers to a local function or one defined in a dot-sourced file. Use breadth-first traversal to resolve transitive dependencies between functions and scripts. Recursively check the function calls within each external function call. Build a dependency map that associates each script with the files and functions it depends on.
    • Check for dot-sourcing dependencies that are used but not imported.

Advantage

Helps to detect missing dot-sourced dependencies automatically.

Limitations

  • Static limitations
    • Since the check is purely static, it may miss dynamically constructed imports (e.g., if a path is computed at runtime) or conditional dot-sourcing based on environment state.
    • This check may produce false positives. For instance, if the caller only executes one branch (e.g., if-else branch) of the callee, the dependencies used in other branches will still be marked as dependencies. (Personally, I think this is an acceptable limitation.)

Motivation and Context

  • Relates to #6385
  • Closes #6514
  • Closes #6527
  • Closes https://github.com/ScoopInstaller/GithubActions/issues/37

How Has This Been Tested?

Tested in GitHub workflow CI.

Checklist

  • [x] I have read the Contributing Guide.
  • [x] I have ensured that I am targeting the develop branch.
  • [ ] I have updated the documentation accordingly.
  • [x] I have updated the tests accordingly.
  • [x] I have added an entry in the CHANGELOG.

Summary by CodeRabbit

  • Tests

    • Added a static code analysis test suite that checks script dependencies and verifies imports and dot-sourced files are resolved.

  • Chores

    • Added a reusable static-analysis utility library to parse scripts, map function calls, and determine inter-file dependencies for automated validation.

z-Fng avatar Nov 05 '25 15:11 z-Fng

Walkthrough

Adds a PowerShell static-analysis Pester test and a supporting library that parse script ASTs, extract functions and call sites, resolve imported and dot-sourced dependencies, and perform BFS traversal to detect missing dot-sourced file dependencies; also adds multiple dot-sourced imports across several bin/libexec scripts.

Changes

Cohort / File(s) Summary
Static analysis test & library
test/Scoop-00StaticAnalysis.Tests.ps1, test/Scoop-StaticAnalysisLib.ps1		 Adds a Pester test suite and a comprehensive static-analysis library providing AST parsing/caching, function extraction, call-site mapping, imported-dependency resolution, and BFS dot-sourcing dependency computation; test validates dependencies per script and fails on missing deps.
CLI/init scripts — added dot-sources
bin/auto-pr.ps1, bin/refresh.ps1, bin/scoop.ps1, bin/uninstall.ps1		 Adds additional dot-sourced library imports (e.g., core.ps1, lib/system.ps1, versions.ps1, manifest.ps1, system.ps1, database.ps1, download.ps1) to expand initialization surface prior to command handling.
libexec scripts — added dot-sources
libexec/scoop-import.ps1, libexec/scoop-uninstall.ps1		 Adds dot-sourcing of lib/download.ps1 to ensure download-related functions are available during import/uninstall flows.
Changelog
CHANGELOG.md		 Updates Unreleased section: replaces/fixes previous item with "Fix missing dot-sourcing dependency" and adds Tests entry for static-analysis dot-sourcing check.

Sequence Diagram(s)

sequenceDiagram
    participant Test as Pester Test Suite
    participant Lib as StaticAnalysisLib
    participant PS as PowerShell Parser

    Test->>Lib: Get-ScriptAst(path)
    Lib->>PS: Parse file (cache)
    PS-->>Lib: AST / error
    Lib-->>Test: AST

    Test->>Lib: Get-ScriptFunction(path)
    Lib->>Lib: Scan AST -> Function list
    Lib-->>Test: Functions

    Test->>Lib: Get-ScriptFunctionCallMap(path)
    Lib->>Lib: Find invocations per function
    Lib-->>Test: Call map

    Test->>Lib: Get-ImportedDependency(path)
    Lib->>Lib: Resolve dot-sources / using: / relative paths
    Lib-->>Test: Resolved imports (or warnings)

    Test->>Lib: Get-DotsourcingDependency(libs, entries)
    rect rgb(230,250,230)
    note right of Lib: BFS over functions and files\nuse call maps + lib functions\naccumulate per-path deps
    Lib-->>Test: AllDeps
    end

    Test->>Test: Validate AllDeps exist and are imported/pre-imported
    alt no missing deps
        Test-->>Test: Pass
    else
        Test-->>Test: Fail (messages)
    end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30–40 minutes

  • Pay special attention to:
    • Get-DotsourcingDependency BFS correctness (cycle detection/termination).
    • Path resolution semantics (PSScriptRoot, relative vs absolute, normalization).
    • Caching behavior and error handling in Get-ScriptAst and Get-ImportedDependency.
    • The new dot-sourced imports in bin/ and libexec/ for ordering or unintended side effects.

Suggested reviewers

  • niheaven

Poem

🐇 I nibble AST leaves in moonlit code,

I chase each dot-sourced path along the road.
Functions mapped and dependencies found,
No missing file will make me bound.
Hooray — the tests keep builds in hoard-mode!

Pre-merge checks and finishing touches

❌ Failed checks (2 inconclusive)
Check name Status Explanation Resolution
Linked Issues check ❓ Inconclusive The PR adds the requested static-analysis test framework (#6527) but does not fix the root causes of missing dependencies (#6514, #37) in the codebase. Verify whether the PR is intended to implement only the test infrastructure or also fix the identified missing dot-sourced dependencies across bin/ and libexec/ scripts.
Out of Scope Changes check ❓ Inconclusive Changes to bin/auto-pr.ps1, bin/refresh.ps1, bin/scoop.ps1, bin/uninstall.ps1, libexec/scoop-import.ps1, and libexec/scoop-uninstall.ps1 add dot-sourced dependencies, which partially address missing dependencies but appear inconsistent with the test infrastructure objective. Clarify whether these dependency additions are partial fixes to identified issues or if they are incomplete and require additional changes across all affected scripts.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding a static-analysis test for dot-sourcing dependency checks.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • [ ] 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • [ ] Create PR with unit tests
  • [ ] Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Nov 05 '25 15:11 coderabbitai[bot] 

Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-alias.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function add_alias called at D:\a\Scoop\Scoop\libexec\scoop-alias.ps1:54
	Function set_config called at D:\a\Scoop\Scoop\lib\commands.ps1:74
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-alias.ps1 : D:\a\Scoop\Scoop\lib\manifest.ps1
	Function add_alias called at D:\a\Scoop\Scoop\libexec\scoop-alias.ps1:54
	Function set_config called at D:\a\Scoop\Scoop\lib\commands.ps1:74
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215
	Function arch_specific called at D:\a\Scoop\Scoop\lib\database.ps1:185
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-alias.ps1 : D:\a\Scoop\Scoop\lib\database.ps1
	Function add_alias called at D:\a\Scoop\Scoop\libexec\scoop-alias.ps1:54
	Function set_config called at D:\a\Scoop\Scoop\lib\commands.ps1:74
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-checkup.ps1 : D:\a\Scoop\Scoop\lib\versions.ps1
	Function Test-HelperInstalled called at D:\a\Scoop\Scoop\libexec\scoop-checkup.ps1:22
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:543
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-checkup.ps1 : D:\a\Scoop\Scoop\lib\manifest.ps1
	Function Test-HelperInstalled called at D:\a\Scoop\Scoop\libexec\scoop-checkup.ps1:22
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:543
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
	Function parse_json called at D:\a\Scoop\Scoop\lib\versions.ps1:54
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-config.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-config.ps1:162
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-config.ps1 : D:\a\Scoop\Scoop\lib\manifest.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-config.ps1:162
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215
	Function arch_specific called at D:\a\Scoop\Scoop\lib\database.ps1:185
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-config.ps1 : D:\a\Scoop\Scoop\lib\database.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-config.ps1:162
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-download.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function is_scoop_outdated called at D:\a\Scoop\Scoop\libexec\scoop-download.ps1:47
	Function set_config called at D:\a\Scoop\Scoop\lib\core.ps1:1182
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-export.ps1 : D:\a\Scoop\Scoop\lib\versions.ps1
	Function list_buckets called at D:\a\Scoop\Scoop\libexec\scoop-export.ps1:18
	Function Invoke-Git called at D:\a\Scoop\Scoop\lib\buckets.ps1:110
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:235
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-export.ps1 : D:\a\Scoop\Scoop\lib\manifest.ps1
	Function list_buckets called at D:\a\Scoop\Scoop\libexec\scoop-export.ps1:18
	Function Invoke-Git called at D:\a\Scoop\Scoop\lib\buckets.ps1:110
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:235
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
	Function parse_json called at D:\a\Scoop\Scoop\lib\versions.ps1:54
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-hold.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-hold.ps1:36
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-hold.ps1 : D:\a\Scoop\Scoop\lib\database.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-hold.ps1:36
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-import.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-import.ps1:27
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-import.ps1 : D:\a\Scoop\Scoop\lib\versions.ps1
	Function add_bucket called at D:\a\Scoop\Scoop\libexec\scoop-import.ps1:32
	Function Test-GitAvailable called at D:\a\Scoop\Scoop\lib\buckets.ps1:124
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:452
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-import.ps1 : D:\a\Scoop\Scoop\lib\database.ps1
	Function add_bucket called at D:\a\Scoop\Scoop\libexec\scoop-import.ps1:32
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\buckets.ps1:166
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-import.ps1 : D:\a\Scoop\Scoop\lib\download.ps1
	Function url_manifest called at D:\a\Scoop\Scoop\libexec\scoop-import.ps1:21
	Function Get-UserAgent called at D:\a\Scoop\Scoop\lib\manifest.ps1:18
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-prefix.ps1 : D:\a\Scoop\Scoop\lib\manifest.ps1
	Function currentdir called at D:\a\Scoop\Scoop\libexec\scoop-prefix.ps1:12
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
	Function parse_json called at D:\a\Scoop\Scoop\lib\versions.ps1:54
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-unhold.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-unhold.ps1:36
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-unhold.ps1 : D:\a\Scoop\Scoop\lib\database.ps1
	Function set_config called at D:\a\Scoop\Scoop\libexec\scoop-unhold.ps1:36
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-uninstall.ps1 : D:\a\Scoop\Scoop\lib\download.ps1
	Function Invoke-Installer called at D:\a\Scoop\Scoop\libexec\scoop-uninstall.ps1:78
	Function url_filename called at D:\a\Scoop\Scoop\lib\install.ps1:113
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-virustotal.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function is_scoop_outdated called at D:\a\Scoop\Scoop\libexec\scoop-virustotal.ps1:44
	Function set_config called at D:\a\Scoop\Scoop\lib\core.ps1:1182
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Get-EnvVar called at D:\a\Scoop\Scoop\lib\core.ps1:164
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\libexec\scoop-virustotal.ps1 : D:\a\Scoop\Scoop\lib\database.ps1
	Function is_scoop_outdated called at D:\a\Scoop\Scoop\libexec\scoop-virustotal.ps1:44
	Function set_config called at D:\a\Scoop\Scoop\lib\core.ps1:1182
	Function Complete-ConfigChange called at D:\a\Scoop\Scoop\lib\core.ps1:124
	Function Set-ScoopDB called at D:\a\Scoop\Scoop\lib\core.ps1:215

Missing dot-sourcing dependency in D:\a\Scoop\Scoop\bin\auto-pr.ps1 : D:\a\Scoop\Scoop\lib\core.ps1
	Function abort called at D:\a\Scoop\Scoop\bin\auto-pr.ps1:111
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\bin\refresh.ps1 : D:\a\Scoop\Scoop\lib\system.ps1
	Function shim called at D:\a\Scoop\Scoop\bin\refresh.ps1:16
	Function Add-Path called at D:\a\Scoop\Scoop\lib\core.ps1:897
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\bin\scoop.ps1 : D:\a\Scoop\Scoop\lib\manifest.ps1
	Function Test-GitAvailable called at D:\a\Scoop\Scoop\bin\scoop.ps1:23
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:452
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
	Function parse_json called at D:\a\Scoop\Scoop\lib\versions.ps1:54
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\bin\scoop.ps1 : D:\a\Scoop\Scoop\lib\versions.ps1
	Function Test-GitAvailable called at D:\a\Scoop\Scoop\bin\scoop.ps1:23
	Function Get-HelperPath called at D:\a\Scoop\Scoop\lib\core.ps1:452
	Function Get-AppFilePath called at D:\a\Scoop\Scoop\lib\core.ps1:470
	Function currentdir called at D:\a\Scoop\Scoop\lib\core.ps1:429
	Function Select-CurrentVersion called at D:\a\Scoop\Scoop\lib\core.ps1:357
Missing dot-sourcing dependency in D:\a\Scoop\Scoop\bin\uninstall.ps1 : D:\a\Scoop\Scoop\lib\download.ps1
	Function Invoke-Installer called at D:\a\Scoop\Scoop\bin\uninstall.ps1:45
	Function url_filename called at D:\a\Scoop\Scoop\lib\install.ps1:113

z-Fng avatar Nov 05 '25 15:11 z-Fng