Add search read-through caching

[loic-sharma]

Is your feature request related to a problem? Please describe.

Search results do not include packages from the mirrored package source. See https://github.com/loic-sharma/BaGet/issues/400#issuecomment-545658107

Describe the solution you'd like

Search results should include both local packages and upstream packages on the mirrored package source. Things to figure out:

1. How do we order results from separate sources?
2. How do we merge results from different sources on the same packages?
3. How do we handle features that work on one source but not another? (Filter by package type or by target framework)

[tzographos]

Some ideas:

1. We could order by what's in the read-though cache first, then by source (e.g. nuget.org).
2. Bring source results first, make cache results "sit on-top" of the source results.
3. Bring from any source that can handle the features, behaving like described in 1 and 2.

[Svisstack]

Maybe it will make sense to add just a read-trough at the first step and at the next add caching.

1. Downloads descending
2. Local first, highest downloads next
3. Cache the data and implement these features locally?

[fourpastmidnight]

If I'm understanding the problem correctly, it would seem to me that Baget ought to present a unified source of information. Let's take a well-known package: EntityFramework (not Core...vanilla). Let's say Baget has locally stored EntityFramework 6.3.0. Upstream NuGet has (at least) EntityFramework 6.4.0.

Baget ought to show a single entry for EntityFramework in the search results. The packages (local 6.3.0 and remote 6.4.0) have the same package ID, so we'll be able to "link" the two disparate sources of package information. (Ideally, you would not create your own personal NuGet package with the same ID as a publicly available package. If you do, well, that's on you. And the behavior from Baget's point of view should be "undefined".) Optionally, Baget could cache package metadata from the remote package source to cut down on latency. (But that may need more thought.) You may want to include in the top-level search results some iconography that details that there are locally cached versions as well as remote versions available. And perhaps even iconography detailing that a newer version exists on the remote source.

Now, when clicking in to view package details, you would see the latest version is EntityFramework 6.4.0. There should be additional metadata available (an Icon, with hover tooltip?) that says this will be sourced from NuGet. Additionally, you would see v6.3.0 also listed in the "available versions" table. And it, too, should have an icon (or some other visual clue) that this package is sourced locally from Baget. Furthermore, it might be nice to enable someone to "click a button" to cache the latest version of EntityFramework locally in Baget apart from performing an actual package restore/upgrade in some project or other.

Just my $0.02.

[reduckted]

This feature is desperately needed. If you're using BaGet as the single package source, then searching for packages, both in Visual Studio's UI and in BaGet's UI, is completely useless unless you've already installed the package and it's been cached by BaGet.

~~Such a shame that this repo appears to no longer be active.~~

Edit: I should add that although I don't know much about the inner-workings of NuGet, I'd be happy to try and assist with this implementing this.

[VAllens]

Any progress on this? I believe people desperately need it.

These are two things. One is caching and the other is mirror indexing.

I propose to retrieve the index cache from local when people initiate a query, and if there is no index cache locally, the reverse proxy initiates the query request and caches the query results. Note that only the json document indexes are cached here, not the nuget package files.

[Mo-GH]

Any new progress?

[jkone27]

this would be a very useful feature