Implement package pinning

[florelis]

This PR builds on top of #2769. I can close that PR in favor of this one, or we can merge that one first to make the changes smaller and easier to review :)

Changes in this PR (not in #2769):

• Modified the CompositePackage to be able to hold more than one available package, each from a different source. The GetAvailableVersionKeys() function now can return versions from multiple sources, so the source field of PackageVersionKey is more relevant now. If the same version is available from multiple sources, they are returned in the order the available packages were added, which matches the previous behavior of just adding the first available package found.
• Added a PinBehavior enum which is passed as an argument to an IPackage's functions for getting available versions. Only the CompositePackage uses it (as it needs both an installed and an available package) and it is used to determine whether we ignore pinned versions when listing the available versions or not. For example, for list we don't care about pins, for upgrade we do consider them, and for upgrade --include-pinned we care about them unless they have Pinning type.
  • Added this new argument all through the codebase. I'm open to making it have a default value, but I wanted to highlight all the places it is used to be sure I made the appropriate choice on each place.
• Updated existing tests to new behaviors. For example, CompositeSource tests now need to ensure that the test packages have a reference to their source as it is used to determine if they are pinned.
• There was already some work for pinning done for the RequiresExplicitUpgrade manifest field. This change is mostly independent of that one, as in my mind the pinning from RequiresExplicitUpgrade is a property of the installed package, whereas user-defined pins are properties of the available packages from each source, so they are handled at different places.

Closes #476 Related to #2611

Microsoft Reviewers: Open in CodeFlow

[florelis]

Here's a little screenshot of this thing in action:

[github-actions[bot]]

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view or the :scroll:action log for details.

Unrecognized words (1)

Geted

Previously acknowledged words that are now absent

PWSTR :arrow_right:

To accept :heavy_check_mark: these unrecognized words as correct and remove the previously acknowledged and now absent words, run the following commands

... in a clone of the [email protected]:florelis/winget-cli.git repository on the pinningLogic branch (:information_source: how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/v0.0.21/apply.pl' |
perl - 'https://github.com/microsoft/winget-cli/actions/runs/3833163240/attempts/1'

Available :books: dictionaries could cover words not in the :blue_book: dictionary

This includes both expected items (399) from .github/actions/spelling/expect.txt and unrecognized words (1)

Dictionary	Entries	Covers
cspell:cpp/src/cpp.txt	30216	26
cspell:win32/src/win32.txt	53509	18
cspell:python/src/python/python-lib.txt	3873	7
cspell:php/php.txt	2597	6
cspell:java/java.txt	7642	5
cspell:python/src/python/python.txt	453	3
cspell:python/src/common/extra.txt	741	3
cspell:django/django.txt	859	3
cspell:typescript/typescript.txt	1211	2
cspell:npm/npm.txt	288	2

Consider adding them using (in .github/workflows/spelling3.yml):

      with:
        extra_dictionaries:
          cspell:cpp/src/cpp.txt
          cspell:win32/src/win32.txt
          cspell:python/src/python/python-lib.txt
          cspell:php/php.txt
          cspell:java/java.txt
          cspell:python/src/python/python.txt
          cspell:python/src/common/extra.txt
          cspell:django/django.txt
          cspell:typescript/typescript.txt
          cspell:npm/npm.txt

To stop checking additional dictionaries, add:

      with:
        check_extra_dictionaries: ''

If the flagged items are :exploding_head: false positives

If items relate to a ...

• binary file (or some other file you wouldn't want to check at all).

  Please add a file path to the excludes.txt file matching the containing file.

  File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

  ^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

• well-formed pattern.

  If you can write a pattern that would match it, try adding it to the patterns.txt file.

  Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

  Note that patterns can't match multiline strings.

[github-actions[bot]]

@check-spelling-bot Report

:red_circle: Please review

See the :open_file_folder: files view or the :scroll:action log for details.

Unrecognized words (1)

pinnned

Previously acknowledged words that are now absent

PWSTR :arrow_right:

To accept :heavy_check_mark: these unrecognized words as correct and remove the previously acknowledged and now absent words, run the following commands

... in a clone of the [email protected]:florelis/winget-cli.git repository on the pinningLogic branch (:information_source: how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/v0.0.21/apply.pl' |
perl - 'https://github.com/microsoft/winget-cli/actions/runs/3842897034/attempts/1'

Available :books: dictionaries could cover words not in the :blue_book: dictionary

This includes both expected items (399) from .github/actions/spelling/expect.txt and unrecognized words (1)

Dictionary	Entries	Covers
cspell:cpp/src/cpp.txt	30216	26
cspell:win32/src/win32.txt	53509	18
cspell:python/src/python/python-lib.txt	3873	7
cspell:php/php.txt	2597	6
cspell:java/java.txt	7642	5
cspell:python/src/python/python.txt	453	3
cspell:python/src/common/extra.txt	741	3
cspell:django/django.txt	859	3
cspell:typescript/typescript.txt	1211	2
cspell:npm/npm.txt	288	2

Consider adding them using (in .github/workflows/spelling3.yml):

      with:
        extra_dictionaries:
          cspell:cpp/src/cpp.txt
          cspell:win32/src/win32.txt
          cspell:python/src/python/python-lib.txt
          cspell:php/php.txt
          cspell:java/java.txt
          cspell:python/src/python/python.txt
          cspell:python/src/common/extra.txt
          cspell:django/django.txt
          cspell:typescript/typescript.txt
          cspell:npm/npm.txt

To stop checking additional dictionaries, add:

      with:
        check_extra_dictionaries: ''

If the flagged items are :exploding_head: false positives

If items relate to a ...

• binary file (or some other file you wouldn't want to check at all).

  Please add a file path to the excludes.txt file matching the containing file.

  File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

  ^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

• well-formed pattern.

  If you can write a pattern that would match it, try adding it to the patterns.txt file.

  Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

  Note that patterns can't match multiline strings.

[yao-msft]

                    // TODO: Needs a whole separate change to fix the fact that we don't support multiple available packages and what the different search behaviors mean

Is this still relevant?

---

Refers to: src/AppInstallerRepositoryCore/CompositeSource.cpp:1293 in 395a9f3. [](commit_id = 395a9f39c1337469ceea6e9bba41833f3805cc11, deletion_comment = False)

[yao-msft]

By the way, I did not see changes related to --force override behavior for pinning, I thought with user passing in --force, PinBehavior will be IgnorePins, is that not needed in the whole flow?

[florelis]

I moved the logic to filter out pinned versions out of CompositePackage's GetAvailableVersionKeys() and GetAvailableVersion(). Instead, now the version keys returned from GetAvailableVersionKeys() include the pinned state for that version. Versions from different sources may have different pin types, and versions that match a gating pin are not marked as pinned. Callers can the use that info to decide what to do.

I added another function like GetAvailableVersion() that also returns the pinned state.

I kept the PinBehavior argument for GetLatestAvailableVersion() because the logic there is more complicated and so it is harder to have each caller do it.

I'm still missing using the --force argument, and the messages that I'm exposing could probably be more specific.

[florelis]

/azp run

[azure-pipelines[bot]]

Pull request contains merge conflicts.

[felipecrs]

That's great news! I can't wait to test it :)

[denelon]

We're checking to see if any other PRs are super close to done, and we'll get a preview build out quickly™️.