https://letterboxd.com/films/popular/

[azumukupoe]

Could you make it work?

[screeny05]

I'm afraid i cannot fetch the whole list, as it contains over half a million movies. That would probably take forever. I could make the list work with a hard pagecount-limit, so you could fetch the first 50 pages or something.

Calculating the runtime for /films/popular is pretty easy:

• 12 * 6 = 72 Movies per Page
• 564.035 Movies = 7834 Pages
• Average Letterboxd response-time: 200ms/Request

Accumulated response-time: ((564035 + 7834) * 200) = 114373800ms ~= 32h

If you can run 10 requests in parallel, that would get down to maybe 5h.

But i'm afraid that something along the way will get broken. The http-request might timeout, your redis might eat all of your memory, radarr will be unable to handle such a large amount of movies. This is not really feasible.

[azumukupoe]

Maybe fetch only the first 5 pages?

[screeny05]

That would be possible. Maybe let the User adjust the limit with an optional parameter.

[nathanbirrell]

Is there a limit parameter, or a perPage parameter like in the API?

I would be happy with just the top 5 results from https://letterboxd.com/films/popular/this/week/

[screeny05]

Can now be done. I've added a limit-parameter so you can fetch only the amount of movies you want to.

But there's a catch: letterboxd.com only allows us to fetch /films/popular/. Nothing else. No filters, no time, no genre, etc. See https://letterboxd.com/robots.txt