Add support for videos

[iMuzz]

React-static-tweets doesn't current support videos. Here's an example tweet:

On Twitter: https://twitter.com/sassal0x/status/1364207094614749184 Rendered Version: https://react-static-tweets.vercel.app/1364207094614749184

Would love any pointers on how/where one might start to implement this! 🙂

[DaniGuardiola]

Working on this already (as agreed with @iMuzz).

Here's an update on my progress:

• The static-tweets ast parser is not finding the video media.
• That's because it's not there in the HTML in the first place (likely Twitter API change).
• The media URI is available through this URL:

syndication.twimg.com/tweet?id=1378452375296442371

Or, alternatively:

cdn.syndication.twimg.com/tweet?id=1378452375296442371

• Here's the relevant part of the JSON returned:

{
  "video": {
    "aspectRatio": [
      249,
      220
    ],
    "contentType": "media_entity",
    "durationMs": 9934,
    "mediaAvailability": {
      "status": "available"
    },
    "poster": "https://pbs.twimg.com/ext_tw_video_thumb/1378452274570264576/pu/img/ytz1zFSu4mFOGGI_.jpg",
    "variants": [
      {
        "type": "video/mp4",
        "src": "https://video.twimg.com/ext_tw_video/1378452274570264576/pu/vid/498x440/s2-MGvS6ilwspHei.mp4?tag=12"
      },
      {
        "type": "application/x-mpegURL",
        "src": "https://video.twimg.com/ext_tw_video/1378452274570264576/pu/pl/bScvTVcujd7bUNjO.m3u8?tag=12"
      },
      {
        "type": "video/mp4",
        "src": "https://video.twimg.com/ext_tw_video/1378452274570264576/pu/vid/304x270/0xP7m72L6iuRzaht.mp4?tag=12"
      }
    ],
    "videoId": {
      "type": "tweet",
      "id": "1378452375296442371"
    },
    "viewCount": 473
  },
}

• Currently debugging the static-tweets fetcher and parser to figure out how to fix it. Looks promising though. I'm only worried about CORS-type issues, but maybe there's still a workaround by using iframes or similar. Will continue this afternoon.

Will keep updating on this issue, and create a draft PR once I start having actual changes to push.

[transitive-bullshit]

That's because the video's not there in the HTML in the first place (likely Twitter API change).

One option would be to switch from using cheerio to puppeteer.. but I would really like to avoid this if at all possible since puppeteer is more heavyweight and introduces difficulties in serverless environments.

but maybe there's still a workaround by using iframes or similar

Let's avoid this if at all possible. What CORS issues are you referring to? I can't imagine there would be any issues playing the video from the client-side if we already have the video URL.

We circumvent all the CORS stuff in general by fetching the tweet info + metadata server-side where CORS doesn't exist 😄

[DaniGuardiola]

I am gonna get the data from the new source. I've found the endpoint that provides that data now (I got it by reverse-engineering the official tweet embeds), so I'll just add it.

We circumvent all the CORS stuff in general by fetching the tweet info + metadata server-side where CORS doesn't exist

The video is precisely the one thing that isn't requested by the server, but it'll probably be fine.

One option would be to switch from using cheerio to puppeteer.. but I would really like to avoid this

Puppeteer is definitely too heavy, I agree. Cheerio is okay. Btw, I've been thinking about revamping the scraper with surgeon, I used to work on scraping and this tool is pretty amazing. But that's for a different time :P

Fortunately, this has a pretty straightforward solution that doesn't involve changing the HTML parsing :D

[DaniGuardiola]

I got it, but doing this I realized that the code is indeed due for a bit of refactoring, especially since the changes in the Twitter API. We could release an experimental version or publish it under something like unstable_fetchTweetAst named export if we want to ship it right away.

I'll push the working code later and create a PR. @iMuzz you should be able to use it directly from GitHub temporarily if it is urgent.

[transitive-bullshit]

@DaniGuardiola try to keep the refactoring and feature additions as separate as possible (ideally in separate PRs).

[kevcodez]

Any updates on this @DaniGuardiola ?

[DaniGuardiola]

@kevcodez I got it working but as mentioned in my last comment, it'd be better imo to do a more global refactor of the AST fetching code, because my current solution duplicates the calls to Twitter's API and I think there might be a way to avoid it.

Maybe we could ship this under an option so that it's opt-in for users who don't mind the cost (extra requests could lead to problems with Twitter's API rate-limits). Sort of like an experimental feature flag. What do you think @transitive-bullshit?

@kevcodez if you need to use this right now, feel free to build the package from my branch yourself, but be aware of the trade-offs. Also, although it works just fine, it hasn't been tested thoroughly so it could break. Keep that in mind.

[remorses]

Gifs aren't rendered too, here is an example https://react-static-tweets.vercel.app/1461738430698860544

[kasem-sm]

any updates?