pkgx
pkgx copied to clipboard
pkgxdev
`curl dist.tea.xyz` should download the latest tea.xyz tarball
We have this in the README it's a neat idea.
Needs a lambda on our cloudfront to do it, and it should only do it for cURL so we can provide the web listing for the whole of dist.tea.xyz like I keep asking @jhheider to do.
Done: https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions/www-redirect-dist/versions/3?tab=code
I'll get the prototype web listing put together so we can argue about webUI stuff.
Takes about 5s to walk the tree, but here's a list of tar.(g|x)z keys.join('\n')ed. So, we have something. I'm thinking the friendly way is to list the packages in a set of collapsed cards that can expand to see the specific individual tarballs.
https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions/www-redirect-dist/versions/7?tab=code
(n.b.: design choice: I went with text/plain instead of application/json we specifically return our source code instead to curl/wget.)
(n.b.2: I know 5s isn't a friendly run time. the alternative is to build/cache the data for easy retrieval (as we did in upload.ts at one point).)
Hm, seems to running much faster (40ms) now, so maybe lambda/cloudfront is caching better.
If we decide on a preferred visual representation, I can make the / redirect to the application files and have the lambda scan the repo at, say, /repo.json, and work with Tom to have it not look like I designed it.
cool, though now I think about it maybe it should be a redirect. That way the filename will be correct if you do -LO.
Also I wanted the binaries not the sources, which ofc I now realize is possibly impossible. I doubt the user agent contains enough information.
I considered the binary, but I doubt curl exposes platform and arch (though it might; I can check.) Seems like redirecting to the installer would be preferred then.
Redirecting to get the filename shouldn't be difficult.
> User-Agent: curl/7.79.1
Yeah, it doesn't look like it wants to expose enough to determine binary platform.
curl -i dist.tea.xyz
HTTP/1.1 302 Moved Temporarily
Content-Length: 114
Connection: keep-alive
Server: CloudFront
Date: Mon, 31 Oct 2022 18:35:05 GMT
Location: https://dist.tea.xyz/tea.xyz/v0.11.3.tar.gz
X-Cache: LambdaGeneratedResponse from cloudfront
Via: 1.1 4c18dd7deeecd61e783c74198943db58.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: PHL50-C1
X-Amz-Cf-Id: XgE4rzaq_4sGOcIV6uqZYO7CyhjDKK-DZ6CUX8uLpxuW6pRZW6RoWw==
Interestingly, that doesn't help with -O. From the man page:
The remote file name to use for saving is extracted from the given URL, nothing else, and if it already exists it will be
overwritten. If you want the server to be able to choose the file name refer to -J, --remote-header-name which can be used
in addition to this option. If the server chooses a file name and that name already exists it will not be overwritten.
And cloudfront doesn't return a filename so -J doesn't help. Possibly if we inject it in the return header.
Oof. I think the trick here is going to be adding a content disposition header in the response step. annoying.
Blech. So, curl -O uses the command line. curl -LJO (barf) will use the content-disposition header, we can use a lambda to jam onto the response (not the request, duh). So we do that:
$ curl -i dist.tea.xyz
HTTP/1.1 302 Moved Temporarily
Content-Length: 114
Connection: keep-alive
Server: CloudFront
Date: Tue, 01 Nov 2022 00:09:40 GMT
Location: https://dist.tea.xyz/tea.xyz/v0.11.3.tar.gz
X-Cache: Hit from cloudfront
Via: 1.1 cf88880413082302757828626cf7b020.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: PHL50-C1
X-Amz-Cf-Id: yw3SPuHeStduqo4ILeyBo9RdwkoaTlcEVuVoQrDLBV4wx49w10ElGQ==
Age: 150
<html><head><meta http-equiv='refresh' content='0; URL=https://dist.tea.xyz/tea.xyz/v0.11.3.tar.gz'></head></html>⏎
$ curl -i dist.tea.xyz -L
HTTP/1.1 302 Moved Temporarily
Content-Length: 114
Connection: keep-alive
Server: CloudFront
Date: Tue, 01 Nov 2022 00:09:40 GMT
Location: https://dist.tea.xyz/tea.xyz/v0.11.3.tar.gz
X-Cache: Hit from cloudfront
Via: 1.1 613a9db04d23967014b7c42269dc1c12.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: PHL50-C1
X-Amz-Cf-Id: 2mPPyym3brCLJIlcZfEHbkEc7NLYL3UAsVpnq-7KO1Z6pGztVUUR2g==
Age: 152
HTTP/2 200
content-type: binary/octet-stream
content-length: 54604
x-amz-id-2: sVN9Jn1OI7tRvdTYqsUPpks29UQ/RPjy7W/1Ttv5au+/F8Dc1z03tllY6LDY9+c7nkKjvwiEtgA=
x-amz-request-id: S8VTR91ZDNXJR4D3
date: Mon, 31 Oct 2022 18:36:00 GMT
last-modified: Sun, 30 Oct 2022 15:55:46 GMT
etag: "1adcaa2549f08b734405bc5f3d42c544"
x-amz-version-id: vJkZWG.2QDcl8crFWuX20T5eF_Byiunm
accept-ranges: bytes
server: AmazonS3
content-disposition: attachment; filename=v0.11.3.tar.gz
x-cache: Hit from cloudfront
via: 1.1 aed3f8ed29085c056c75452d71b07f7e.cloudfront.net (CloudFront)
x-amz-cf-pop: PHL50-C1
x-amz-cf-id: KGPKlFz1j8UV6N6e6IAxqx8GvGHpBx4FVVL99NY2z9wT1Er1FrksiQ==
age: 20173
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
$ curl dist.tea.xyz -LJO
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 114 100 114 0 0 1191 0 --:--:-- --:--:-- --:--:-- 1239
100 54604 100 54604 0 0 184k 0 --:--:-- --:--:-- --:--:-- 184k
$ ls -l
...
-rw-r--r-- 1 jacob staff 54604 Oct 31 20:12 v0.11.3.tar.gz
I appreciate the work but getting the sources is not useful so we may as well stop unless we can figure out how to make cURL send a more useful user-agent or more information we can use at the lambda.
No worries. I'll disable the source sending. Keep the plaintext listing on the bottles/srcs for now?
disabled source and content-disposition rules. left text listing as default root.
@Xercesblu3 I think that's what we can do for now. If we want to alter language in the future, that's simple enough (it's a lambda)