Error 451 with https://ghapi.huchen.dev/

[hedyhli]

[nahtnam]

Same here

I believe Vercel provides free hosting for open source projects, have you applied to that?

[huchenme]

Unfortunately Vercel did not want to sponsor this project as it is API only. I could no longer find a good free host for the project. I suggest you clone the repo and deploy the code somewhere 😞

[omarryhan]

That's sad news :/

@github should definitely be sponsoring this or atleast provide an API of their own.

[omarryhan]

@huchenme How about we add a simple frontend that makes use of the API, that wouldn't be breaching Vercel's TOS, right?

[huchenme]

I can try add frontend for this actually

[sandoche]

@huchenme I can give you the code of GitNews https://git.news if you wish. Let me know!

[huchenme]

@sandoche I do have some code, let me add to the repo during nights and apply Vercel's sponsor again, if everything goes smooth it should be back in a week 🤞 Meanwhile, if you need the API now, I suggest you clone the code and deploy to somewhere like Vercel

[akane10]

I use this api for my personal website. Since it's having an issue, I cloned and hosted it. So in the meantime, feel free to use it, gtrend.yapie.me

[huchenme]

@akane10 that is great, may I know where did you host it?

[akane10]

@huchenme yeah sure, it's on digitalocean. But anyway, isn't there netlify function and it is free?

[huchenme]

@akane10 I tried Netlify function and could not recall what did not work as expected. if you plan to keep your server, do you mind if I point ghapi.huchen.dev to your server temporarily?

[akane10]

@huchenme Yes, sure. I don't mind

[huchenme]

The issue I was facing for Netlify is the caching: s-maxage does not actually work: if I hit https://ghapi.huchen.dev/repositories?language=javascript, all the subsequent requests with different parameters (e.g. https://ghapi.huchen.dev/repositories?language=java) will return the same results (javascript)

If anyone has any luck with Netlify please let me know

[huchenme]

This is a miracle. They just blocked me

I am thinking to deploy to Netlify without caching. It might be slower, and it might hit the limit soon as well 🤞

[huchenme]

It is back now! the only downside is that it does not have browser cache now

[huchenme]

It quickly finished the free quota in Netlify, have to stop the project.

[nahtnam]

Hmm, have you considered instead making a library that we can import and await and it will make the request and parse the request for us? That way you don't have to worry about hosting it, and we are able to use the code you wrote

[akane10]

@nahtnam he already did it actually https://github.com/huchenme/github-trending-api#install. And yeah I agree, that's totally fine if there is no API, only library. We appreciate your work @huchenme.

[pupubird]

Hey hey for anyone that's reading on this, I had been using this library for quite a long time, definitely appreciate your work! @huchenme !

I had also created a self-hosted version of the scraper server ( with AWS lambda function compatible ), https://github.com/pupubird/get-github-trending

And the demo site here https://hackertab.pupubird.com/repositories

All credits go to @huchenme

[pupubird]

Also, by utilizing AWS CloudFront, I am able to cache the response with TTL to hourly, will write up a tutorial about it too!

[waningflow]

Feel free to use. https://github-trending-api.waningflow.com/languages https://github-trending-api.waningflow.com/repositories?language=typescript&since=daily

[Smeet97Kathiria]

@waningflow will this ever go down? is there a limit ?

[waningflow]

@Smeet97Kathiria Deployed on Function Compute of Aliyun. About first 500k requests per month are free.

[Smeet97Kathiria]

@waningflow what happens after it hit the limits?

[waningflow]

@Smeet97Kathiria I suppose it's not likely to reach the limit for normal usage. If it does, I may restrict the apis temporarily. It's also recommended to deploy the service youself. Here is what I use in Function Compute of Aliyun, github-trending-api-node. Just yarn to install dependencies and upload the whole git to the platform.

[hedyhli]

hey @akane10 I tried out your url, and it seems https://gtrend.yapie.me/spoken_languages doesn't work?

[akane10]

@hedythedev yeah, route languages and spoken_languages was I disabled for a reason. But it is now working. In case anyone uses my endpoints, it won't run out of request, as long as I can affort my VM.

[andraz-at]

Your problem is caused by re-computing the same exact data thousands of times for no good reason. In the cloud CPU is expensive, disk is practically free. Use functions to write static data in files, then route users to static files like it's 1995.

Can you workaround this issue by creating an AWS hosted endpoint (trivially built with AWS Amplify -> Hosting)?

Then you just need to add a Lambda function which calculates all required responses and stores the responses into static files to an S3 bucket. From this bucket you can serve static files as a web page by simply enabling public access to this static hosting (amplify hosting does that in the background for you).

tl;dr: Create a trivial "web site" to set up free web hosting + create a Lambda function to update this same "web site's" files in the S3 bucket (fetch, parse, write json...).

[omarryhan]

@andraz-at Instead of making the function store the response on S3, wouldn't it be simpler to cache all the function responses on a CDN e.g. AWS Cloudfront or Cloudflare? Serving from a CDN cache is even cheaper than serving directly from S3.

Since these responses will only change once a day, you can set the TTL to be 86400 seconds (1 day). That way, it's very unlikely you run out of free function calls on most FAAS providers.

If my memory serves me correctly, @huchenme was actually caching the responses when this project was being hosted on Vercel, it's a bummer that they suspended this project and his account as well :/

[cm3z4]

Hey @akane10, I'm currently routing my site/app GitYeti to your server. Is that okay?