nixos-search
nixos-search copied to clipboard
Hard-coded package and option count on search.nixos.org
The message at the header saying search.nixos.org enables searching of "more than 80 000 packages" is hard-coded. Should be way higher by now, as repology also states.
Search more than 100,000 packages! 👍
Here is a small script that retrieve the number of packages from the latest nixpkgs release
It fetches the packages.json
from the last item of nixpkgs s3 bucket
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p curl jq brotli yq
while [ "$latest" != "null" ]; do
prev="${latest}"
latest=$(curl "https://nix-releases.s3.amazonaws.com/?delimiter=/&prefix=nixpkgs/&marker=${latest}" \
| xq -rMc ".ListBucketResult.CommonPrefixes[-1].Prefix")
echo "-> $latest"
done
target="${prev%/}"
echo "Selecting latest: $target"
count=$(curl https://releases.nixos.org/${target}/packages.json.br \
| brotli -d --stdout | jq ".packages | length")
echo "=> $count"
For nixpkgs/nixpkgs-24.05pre606477.e976fa8f49c3
there are 111246
packages available!
For options, it is almost the same script!
--- count-pkgs.sh 2024-04-04 08:04:14.290391078 +0200
+++ count-options.sh 2024-04-04 08:10:17.967651425 +0200
@@ -3,14 +3,14 @@
while [ "$latest" != "null" ]; do
prev="${latest}"
- latest=$(curl "https://nix-releases.s3.amazonaws.com/?delimiter=/&prefix=nixpkgs/&marker=${latest}" \
+ latest=$(curl "https://nix-releases.s3.amazonaws.com/?delimiter=/&prefix=nixos/unstable/&marker=${latest}" \
| xq -rMc ".ListBucketResult.CommonPrefixes[-1].Prefix")
echo "-> $latest"
done
target="${prev%/}"
echo "Selecting latest: $target"
-count=$(curl https://releases.nixos.org/${target}/packages.json.br \
- | brotli -d --stdout | jq ".packages | length")
+count=$(curl https://releases.nixos.org/${target}/options.json.br \
+ | brotli -d --stdout | jq "length")
echo "=> $count"
Thus, for nixos/unstable/nixos-24.05pre606533.08b9151ed403
there are 18396
options available!
Hej @Sigmanificient, nice work! Danke!
You may want to contact the Documentation Team, in case you want to contribute to the code base of search.nixos.org. I'll highlight @fricklerhandwerk and @infinisil of the Docu Team, in case they missed this issue for now.
Hej @Sigmanificient, nice work! Danke!
You may want to contact the Documentation Team, in case you want to contribute to the code base of search.nixos.org. I'll highlight @fricklerhandwerk and @infinisil of the Docu Team, in case they missed this issue for now.
I would love to contribute to the website, but unfortunately I don't know anything about elm, and I am unsure on what to exactly do.
I shared the script in case they could help open the discussion, and I would be more than happy if they turned out useful
Nah the docs team doesn't have write access to search.nixos.org, that's for the marketing team :)
Ah, the infamous NixOS labyrinth, thanks for the insight @infinisil :)
Small update: I found that https://channels.nixos.org/nixos-unstable
that already redirects to the latest nixpkgs version, which means I can cut off half of the previous script!
Here is a newer version, that makes way fewer requests, and outputs both packages count and options count in a JSON format
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p curl jq brotli
function parse_unstable_json() {
local url="https://channels.nixos.org/nixos-unstable/$1.json.br"
echo $(curl -L "$url" | brotli -d --stdout | jq "$2")
}
pkg_count=$(parse_unstable_json "packages" ".packages | length")
opt_count=$(parse_unstable_json "options" "length")
echo "{ 'packages': $pkg_count, 'options': $opt_count }" | tr "'" '"'
{ "packages": 111413, "options": 18524 }
This issue should be renamed Hard-coded package and option count on search.nixos.org