ChocolateyGUI Loading Xth result page performs 2X queries, making each page slower and slower to load

Loading Xth result page performs 2X queries, making each page slower and slower to load

Open samhocevar opened this issue 3 years ago • 7 comments

When showing page X of an online gallery and clicking “next”, instead of performing one HTTP query with e.g. $skip=Z&$top=30 (with Z = 30×(X-1)), the GUI also does 2X-1 more queries with $skip=0, 30, 60, etc. It does nothing with the results of these additional queries.

For instance, showing page 2 will perform 4 queries on the website ($skip=0, 30, 60, 90). Showing page 10 will perform 20 queries.

The worst case is when clicking “last”. On my machine it hangs the application for more than 3 minutes, cannot be interrupted, performs 316 queries, and downloads almost 60 MiB of data, just so that it can display a list of 30 packages.

This is probably a major cause for #777. Fixing it may also make #552 less relevant.

edit: rewrote some figures to better reflect reality

Sep 17 '20 08:09 samhocevar

Another way to see the problem is that in order to browse the first 50 result pages, ChocolateyGUI makes 5000 server queries instead of the expected 50.

Sep 17 '20 12:09 samhocevar

@samhocevar so that we can be sure that we are talking about the same thing, can you provide information on exactly how you have been capturing these requests? I would like to be able to fully replicate the issue on my side.

Sep 17 '20 14:09 gep13

@gep13 I have set up an HTTP proxy and I monitor the requests. Here is what I see:

When launching Chocolatey GUI:

[17/Sep/2020:16:22:39 +0200] "GET /api/v2/ HTTP/1.0" 200 1109 "-" "NuGet/0.10.15.0 (Microsoft Windows NT 10.0.19041.0)"
[17/Sep/2020:16:22:39 +0200] "GET /api/v2/$metadata HTTP/1.0" 200 7168 "-" "NuGet/0.10.15.0 (Microsoft Windows NT 10.0.19041.0)"
[17/Sep/2020:16:22:39 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'7zip')%20and%20IsLatestVersion HTTP/1.1" 200 6827 "-" "Chocolatey Core"
[17/Sep/2020:16:22:40 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'7zip.install')%20and%20IsLatestVersion HTTP/1.1" 200 7085 "-" "Chocolatey Core"
[17/Sep/2020:16:22:40 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'chocolatey')%20and%20IsLatestVersion HTTP/1.1" 200 8534 "-" "Chocolatey Core"
[17/Sep/2020:16:22:40 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'chocolatey-core.extension')%20and%20IsLatestVersion HTTP/1.1" 200 6372 "-" "Chocolatey Core"
[17/Sep/2020:16:22:40 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'chocolatey-dotnetfx.extension')%20and%20IsLatestVersion HTTP/1.1" 200 6366 "-" "Chocolatey Core"
[17/Sep/2020:16:22:41 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'chocolatey-visualstudio.extension')%20and%20IsLatestVersion HTTP/1.1" 200 6375 "-" "Chocolatey Core"
[17/Sep/2020:16:22:42 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'chocolatey-windowsupdate.extension')%20and%20IsLatestVersion HTTP/1.1" 200 6321 "-" "Chocolatey Core"
[17/Sep/2020:16:22:42 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'chocolateygui')%20and%20IsLatestVersion HTTP/1.1" 200 7600 "-" "Chocolatey Core"
[17/Sep/2020:16:22:42 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'dependencywalker')%20and%20IsLatestVersion HTTP/1.1" 200 6848 "-" "Chocolatey Core"
[17/Sep/2020:16:22:43 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'directx')%20and%20IsLatestVersion HTTP/1.1" 200 6043 "-" "Chocolatey Core"
[17/Sep/2020:16:22:43 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'discord')%20and%20IsLatestVersion HTTP/1.1" 200 6401 "-" "Chocolatey Core"
[17/Sep/2020:16:22:43 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'discord.install')%20and%20IsLatestVersion HTTP/1.1" 200 6643 "-" "Chocolatey Core"
[17/Sep/2020:16:22:44 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'dotnet4.5.2')%20and%20IsLatestVersion HTTP/1.1" 200 5887 "-" "Chocolatey Core"
[17/Sep/2020:16:22:44 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'dotnet4.6')%20and%20IsLatestVersion HTTP/1.1" 200 6083 "-" "Chocolatey Core"
[17/Sep/2020:16:22:44 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'dotnet4.6-targetpack')%20and%20IsLatestVersion HTTP/1.1" 200 6312 "-" "Chocolatey Core"
[17/Sep/2020:16:22:44 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'dotnet4.6.1')%20and%20IsLatestVersion HTTP/1.1" 200 6253 "-" "Chocolatey Core"
[17/Sep/2020:16:22:44 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'dotnetfx')%20and%20IsLatestVersion HTTP/1.1" 200 7502 "-" "Chocolatey Core"
[17/Sep/2020:16:22:45 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'gimp')%20and%20IsLatestVersion HTTP/1.1" 200 7536 "-" "Chocolatey Core"
[17/Sep/2020:16:22:45 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'git')%20and%20IsLatestVersion HTTP/1.1" 200 8305 "-" "Chocolatey Core"
[17/Sep/2020:16:22:45 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'git-credential-manager-for-windows')%20and%20IsLatestVersion HTTP/1.1" 200 7639 "-" "Chocolatey Core"
[17/Sep/2020:16:22:46 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'git.install')%20and%20IsLatestVersion HTTP/1.1" 200 8422 "-" "Chocolatey Core"
[17/Sep/2020:16:22:46 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'googlechrome')%20and%20IsLatestVersion HTTP/1.1" 200 7008 "-" "Chocolatey Core"
[17/Sep/2020:16:22:46 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'inkscape')%20and%20IsLatestVersion HTTP/1.1" 200 7734 "-" "Chocolatey Core"
[17/Sep/2020:16:22:46 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'innosetup')%20and%20IsLatestVersion HTTP/1.1" 200 8988 "-" "Chocolatey Core"
[17/Sep/2020:16:22:47 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'kb2882822')%20and%20IsLatestVersion HTTP/1.1" 200 6718 "-" "Chocolatey Core"
[17/Sep/2020:16:22:47 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'kb2919355')%20and%20IsLatestVersion HTTP/1.1" 200 6721 "-" "Chocolatey Core"
[17/Sep/2020:16:22:47 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'kb2919442')%20and%20IsLatestVersion HTTP/1.1" 200 6398 "-" "Chocolatey Core"
[17/Sep/2020:16:22:48 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'kb2999226')%20and%20IsLatestVersion HTTP/1.1" 200 7420 "-" "Chocolatey Core"
[17/Sep/2020:16:22:48 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'kb3033929')%20and%20IsLatestVersion HTTP/1.1" 200 6742 "-" "Chocolatey Core"
[17/Sep/2020:16:22:49 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'kb3035131')%20and%20IsLatestVersion HTTP/1.1" 200 7013 "-" "Chocolatey Core"
[17/Sep/2020:16:22:49 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'microsoft-teams')%20and%20IsLatestVersion HTTP/1.1" 200 7102 "-" "Chocolatey Core"
[17/Sep/2020:16:22:50 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'netfx-4.5.1-devpack')%20and%20IsLatestVersion HTTP/1.1" 200 6204 "-" "Chocolatey Core"
[17/Sep/2020:16:22:50 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'netfx-4.5.2-devpack')%20and%20IsLatestVersion HTTP/1.1" 200 6615 "-" "Chocolatey Core"
[17/Sep/2020:16:22:51 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'netfx-4.6.1-devpack')%20and%20IsLatestVersion HTTP/1.1" 200 6616 "-" "Chocolatey Core"
[17/Sep/2020:16:22:51 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'netfx-4.7.2-devpack')%20and%20IsLatestVersion HTTP/1.1" 200 7084 "-" "Chocolatey Core"
[17/Sep/2020:16:22:51 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'nuget.commandline')%20and%20IsLatestVersion HTTP/1.1" 200 6304 "-" "Chocolatey Core"
[17/Sep/2020:16:22:51 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'p4v')%20and%20IsLatestVersion HTTP/1.1" 200 6416 "-" "Chocolatey Core"
[17/Sep/2020:16:22:52 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'procexp')%20and%20IsLatestVersion HTTP/1.1" 200 7080 "-" "Chocolatey Core"
[17/Sep/2020:16:22:52 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'putty.install')%20and%20IsLatestVersion HTTP/1.1" 200 7883 "-" "Chocolatey Core"
[17/Sep/2020:16:22:53 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'python')%20and%20IsLatestVersion HTTP/1.1" 200 6408 "-" "Chocolatey Core"
[17/Sep/2020:16:22:53 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'python3')%20and%20IsLatestVersion HTTP/1.1" 200 7176 "-" "Chocolatey Core"
[17/Sep/2020:16:22:53 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'renderdoc')%20and%20IsLatestVersion HTTP/1.1" 200 6453 "-" "Chocolatey Core"
[17/Sep/2020:16:22:54 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'slack')%20and%20IsLatestVersion HTTP/1.1" 200 6597 "-" "Chocolatey Core"
[17/Sep/2020:16:22:54 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'sysinternals')%20and%20IsLatestVersion HTTP/1.1" 200 7165 "-" "Chocolatey Core"
[17/Sep/2020:16:22:55 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'telegram')%20and%20IsLatestVersion HTTP/1.1" 200 8837 "-" "Chocolatey Core"
[17/Sep/2020:16:22:55 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'telegram.install')%20and%20IsLatestVersion HTTP/1.1" 200 8927 "-" "Chocolatey Core"
[17/Sep/2020:16:22:56 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'tightvnc')%20and%20IsLatestVersion HTTP/1.1" 200 6698 "-" "Chocolatey Core"
[17/Sep/2020:16:22:56 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vcredist140')%20and%20IsLatestVersion HTTP/1.1" 200 7480 "-" "Chocolatey Core"
[17/Sep/2020:16:22:56 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vcredist2010')%20and%20IsLatestVersion HTTP/1.1" 200 6294 "-" "Chocolatey Core"
[17/Sep/2020:16:22:57 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vcredist2012')%20and%20IsLatestVersion HTTP/1.1" 200 6598 "-" "Chocolatey Core"
[17/Sep/2020:16:22:57 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vcredist2013')%20and%20IsLatestVersion HTTP/1.1" 200 6687 "-" "Chocolatey Core"
[17/Sep/2020:16:22:57 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vcredist2015')%20and%20IsLatestVersion HTTP/1.1" 200 7478 "-" "Chocolatey Core"
[17/Sep/2020:16:22:57 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vim')%20and%20IsLatestVersion HTTP/1.1" 200 8061 "-" "Chocolatey Core"
[17/Sep/2020:16:22:57 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio-installer')%20and%20IsLatestVersion HTTP/1.1" 200 6663 "-" "Chocolatey Core"
[17/Sep/2020:16:22:58 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2017-workload-manageddesktop')%20and%20IsLatestVersion HTTP/1.1" 200 9776 "-" "Chocolatey Core"
[17/Sep/2020:16:22:58 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2017-workload-nativedesktop')%20and%20IsLatestVersion HTTP/1.1" 200 9685 "-" "Chocolatey Core"
[17/Sep/2020:16:22:58 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2017-workload-netcoretools')%20and%20IsLatestVersion HTTP/1.1" 200 9800 "-" "Chocolatey Core"
[17/Sep/2020:16:22:58 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2017professional')%20and%20IsLatestVersion HTTP/1.1" 200 10762 "-" "Chocolatey Core"
[17/Sep/2020:16:22:59 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2019-workload-manageddesktop')%20and%20IsLatestVersion HTTP/1.1" 200 9710 "-" "Chocolatey Core"
[17/Sep/2020:16:22:59 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2019-workload-nativedesktop')%20and%20IsLatestVersion HTTP/1.1" 200 9714 "-" "Chocolatey Core"
[17/Sep/2020:16:22:59 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2019-workload-netcoretools')%20and%20IsLatestVersion HTTP/1.1" 200 9826 "-" "Chocolatey Core"
[17/Sep/2020:16:23:00 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'visualstudio2019professional')%20and%20IsLatestVersion HTTP/1.1" 200 10595 "-" "Chocolatey Core"
[17/Sep/2020:16:23:00 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vlc')%20and%20IsLatestVersion HTTP/1.1" 200 6716 "-" "Chocolatey Core"
[17/Sep/2020:16:23:00 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vscode')%20and%20IsLatestVersion HTTP/1.1" 200 7809 "-" "Chocolatey Core"
[17/Sep/2020:16:23:01 +0200] "GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'vscode.install')%20and%20IsLatestVersion HTTP/1.1" 200 8534 "-" "Chocolatey Core"

(This is Chocolatey GUI querying the server for each package I have installed; that step could be improved a lot by using a query I provided here)

When navigating from “This PC” to the “chocolatey” source on the left tab:

[17/Sep/2020:16:27:42 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=0&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 190183 "-" "Chocolatey Core"
[17/Sep/2020:16:27:43 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=30&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 174882 "-" "Chocolatey Core"
[17/Sep/2020:16:27:44 +0200] "GET /api/v2/Search()/$count?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 663 "-" "Chocolatey Core"

(For some reason the whole list of updates is downloaded again but I omitted that part).

When clicking “next” to navigate from page 1 to page 2:

[17/Sep/2020:16:29:07 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=0&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 190127 "-" "Chocolatey Core"
[17/Sep/2020:16:29:08 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=30&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 174882 "-" "Chocolatey Core"
[17/Sep/2020:16:29:08 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=60&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 172354 "-" "Chocolatey Core"
[17/Sep/2020:16:29:09 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=90&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 178058 "-" "Chocolatey Core"
[17/Sep/2020:16:29:10 +0200] "GET /api/v2/Search()/$count?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 663 "-" "Chocolatey Core"

Navigating from page 2 to page 3:

[17/Sep/2020:16:33:15 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=0&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 190183 "-" "Chocolatey Core"
[17/Sep/2020:16:33:16 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=30&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 174882 "-" "Chocolatey Core"
[17/Sep/2020:16:33:17 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=60&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 172354 "-" "Chocolatey Core"
[17/Sep/2020:16:33:17 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=90&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 178058 "-" "Chocolatey Core"
[17/Sep/2020:16:33:18 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=120&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 164856 "-" "Chocolatey Core"
[17/Sep/2020:16:33:19 +0200] "GET /api/v2/Search()/$count?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 663 "-" "Chocolatey Core"

From page 3 to page 4:

[17/Sep/2020:16:34:01 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=0&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 190127 "-" "Chocolatey Core"
[17/Sep/2020:16:34:02 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=30&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 174882 "-" "Chocolatey Core"
[17/Sep/2020:16:34:03 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=60&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 172354 "-" "Chocolatey Core"
[17/Sep/2020:16:34:04 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=90&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 178058 "-" "Chocolatey Core"
[17/Sep/2020:16:34:04 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=120&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 164856 "-" "Chocolatey Core"
[17/Sep/2020:16:34:05 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=150&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 180943 "-" "Chocolatey Core"
[17/Sep/2020:16:34:06 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=180&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 181559 "-" "Chocolatey Core"
[17/Sep/2020:16:34:07 +0200] "GET /api/v2/Search()/$count?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 663 "-" "Chocolatey Core"

From page 4 to page 5:

[17/Sep/2020:16:35:01 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=0&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 190127 "-" "Chocolatey Core"
[17/Sep/2020:16:35:02 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=30&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 174882 "-" "Chocolatey Core"
[17/Sep/2020:16:35:03 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=60&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 172354 "-" "Chocolatey Core"
[17/Sep/2020:16:35:03 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=90&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 178058 "-" "Chocolatey Core"
[17/Sep/2020:16:35:04 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=120&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 164856 "-" "Chocolatey Core"
[17/Sep/2020:16:35:05 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=150&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 180943 "-" "Chocolatey Core"
[17/Sep/2020:16:35:05 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=180&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 181559 "-" "Chocolatey Core"
[17/Sep/2020:16:35:06 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=210&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 173572 "-" "Chocolatey Core"
[17/Sep/2020:16:35:07 +0200] "GET /api/v2/Search()?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&$skip=240&$top=30&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 170966 "-" "Chocolatey Core"
[17/Sep/2020:16:35:08 +0200] "GET /api/v2/Search()/$count?$filter=IsLatestVersion&$orderby=DownloadCount%20desc,Id&searchTerm=''&targetFramework=''&includePrerelease=false HTTP/1.1" 200 663 "-" "Chocolatey Core"

As you can see, each click on “next” takes more and more time than the previous page.

Sep 17 '20 14:09 samhocevar

@samhocevar thank you for the information. Can you confirm exactly which HTTP Proxy you are using? I am trying to replicate the issue that you are seeing with Fiddler, but I am not able to reproduce.

To provide some context about how Chocolatey GUI works, it is essentially a wrapper around choco.exe, via the Chocolatey.Lib NuGet Package. What I mean by this is there are no direct queries to any remote source from Chocolatey GUI (except when fetching the icons to display for the package). Any outgoing queries for searching for packages, checking whether packages are outdated, etc, are all done via Chocolatey itself.

On start up, Chocolatey GUI will run the choco outdated command to establish which packages are outdated on your machine, but I can't reproduce the output you are showing in the first output you are showing. Can you confirm if you see the same amount of requests if you run the choco outdated command directly at the CLI?

I am able to reproduce the second part of the issue that you are bringing up here, with regard to the growing number of requests as you move through the pages when looking at a remote source. I will continue to investigate that issue.

Sep 18 '20 10:09 gep13

I am using Apache as a proxy, with the following simple configuration:

RewriteEngine On
RewriteRule "^(api/.*)$" "https://www.chocolatey.org/$1" [P,L]

Then in Chocolatey I set up my source URL to http://mydomain/api/v2/ instead of https://www.chocolatey.org/api/v2/.

I can confirm that choco outdated performs exactly the same number of requests as the GUI. Adding --trace let me confirm this explicitly.

Sep 18 '20 11:09 samhocevar

@samhocevar thanks for confirming.

Given that choco outdated is performing the same number of requests, any optimisations that can be made here would need to be made in Chocolatey itself. In the pre-release version of Chocolatey GUI, there is a new option:

Which can be enabled to prevent the automated outdated check when the application first runs. This may be useful to you, and I would invite you to take this for a spin.

Regarding the increasing number of requests, this could be something that can be addressed in Chocolatey GUI, but I am still investigating.

Sep 18 '20 11:09 gep13

I've created https://github.com/chocolatey/choco/issues/2120, which we are considering a high priority issue. The culprit for this appears to be NuGet.Core, which is ignoring the requests we are passing to it to limit the result sets. It's also possible we are asking for those results to be limited incorrectly.

Sep 19 '20 15:09 ferventcoder

ChocolateyGUI ChocolateyGUI copied to clipboard

Loading Xth result page performs 2X queries, making each page slower and slower to load

ChocolateyGUI
ChocolateyGUI copied to clipboard