nitter icon indicating copy to clipboard operation
nitter copied to clipboard

Missing posts in thread in rss

Open highroller opened this issue 2 years ago • 9 comments

I found that there are constantly missing posts in thread with 20+ posts in rss. The web nitter version works fine, just not rss. Sometimes, the rss version missed first 6+ posts. The problem is not rss reader because I tried more than 2 readers, same problem.

The other problem is the posts arrive out of order in rss, make it harder to read.

highroller avatar Feb 14 '23 19:02 highroller

Could you link to an example where you see this happening?

pin-grid-array avatar Feb 18 '23 19:02 pin-grid-array

Could you link to an example where you see this happening?

https://unofficialbird.com/macrodailyco

highroller avatar Feb 23 '23 04:02 highroller

@highroller Thank you. I see what you mean. But I believe what you are describing is not a bug.

There are two different things going on here. First, the RSS feed can only hold 20 tweets at once. So for threads longer than 20 tweets, you will miss some tweets in the feed unless your reader is requesting updates as fast as (or faster than) the tweets are being made.

To explain your second point about out-of-order tweets in the RSS feed, it helps to work with an example.

Let's look at the thread dated Feb 22 that starts with "Moving over to markets, here's some of what we're watching in the latest #StockMarket indexes..." (below, I put the tweets in that thread in bold, and I put tweets outside that thread in italics).

Web nitter threads the tweets together and displays first tweet in that thread (i.e. the oldest tweet) at the top, followed by later tweets below it.

But the RSS version displays everything in reverse chronological order (newest at the top and oldest at the bottom) and does not keep threads together.

In that thread, macrodailyco started by making four tweets in a thread:

Then they posted a new tweet outside of that thread:

Then they continued adding to the above thread with two more tweets:

Then they posted another new tweet outside that thread:

And finally, they returned to the above thread with one last tweet:

So, web nitter keeps all the tweets in the thread (the bold tweets) together to improve ease of reading, even though that results in the tweets becoming chronologically out of order as you scroll down the webpage.

But the RSS version keeps everything in reverse chronological order.

pin-grid-array avatar Feb 24 '23 03:02 pin-grid-array

@highroller Thank you. I see what you mean. But I believe what you are describing is not a bug.

There are two different things going on here. First, the RSS feed can only hold 20 tweets at once. So for threads longer than 20 tweets, you will miss some tweets in the feed unless your reader is requesting updates as fast as (or faster than) the tweets are being made.

To explain your second point about out-of-order tweets in the RSS feed, it helps to work with an example.

Let's look at the thread dated Feb 22 that starts with "Moving over to markets, here's some of what we're watching in the latest #StockMarket indexes..." (below, I put the tweets in that thread in bold, and I put tweets outside that thread in italics).

Web nitter threads the tweets together and displays first tweet in that thread (i.e. the oldest tweet) at the top, followed by later tweets below it.

But the RSS version displays everything in reverse chronological order (newest at the top and oldest at the bottom) and does not keep threads together.

In that thread, macrodailyco started by making four tweets in a thread:

* [**Moving over to markets, here's some of what we're watching in the latest #StockMarket indexes...**](https://unofficialbird.com/macrodailyco/status/1628196451003035650#m)

* [**Zooming out, an indicator that everyone can easily access & utilize is the 'MACD'...**](https://unofficialbird.com/macrodailyco/status/1628199170396880897#m)

* [**Zooming back in, tracking the up-trend since the Oct. lows...**](https://unofficialbird.com/macrodailyco/status/1628201587897245696#m)

* [**Looking beneath the surface of the #StockMarket, there's been quite the change of character in...**](https://unofficialbird.com/macrodailyco/status/1628204903108681730#m)

Then they posted a new tweet outside of that thread:

* [_Remember what @jam_croissant said... 👀_](https://unofficialbird.com/macrodailyco/status/1628205540403736586#m)

Then they continued adding to the above thread with two more tweets:

* [**Going back to what @jam_croissant 🥐 shared in his recent interview...**](https://unofficialbird.com/macrodailyco/status/1628205850153082880#m)

* [**Looking ahead into the end of Feb. for #stocks #StockMarket...**](https://unofficialbird.com/macrodailyco/status/1628208227618881537#m)

Then they posted another new tweet outside that thread:

* [_Hawks 🦅 have been given their talking points, even though Bullard..._](https://unofficialbird.com/macrodailyco/status/1628391204223086597#m)

And finally, they returned to the above thread with one last tweet:

* [**Quite a bit of important supports (moving averages, etc.) that have been building...**](https://unofficialbird.com/macrodailyco/status/1628399533737517056#m)

So, web nitter keeps all the tweets in the thread (the bold tweets) together to improve ease of reading, even though that results in the tweets becoming chronologically out of order as you scroll down the webpage.

But the RSS version keeps everything in reverse chronological order.

Sadly, 20+ tweets thread was just an example.

I have seen thread with 9 tweets missing 5 tweets and were out of order. 1, 8, 7,6. Rss reader is setup to show in chronological order.

I have seen thread with 2 tweets that were out of order with no other outside tweets in between.

highroller avatar Feb 25 '23 23:02 highroller

@highroller I have no explanation for the things you describe. The 2-tweet thread appearing out of order is especially puzzling to me.

The next time you see these things happening, please post a link to them and tag the dev, and hopefully they will be able to see it before it disappears from the timeline.

Even though web nitter is causing problems, it sounds like the RSS feed is still nevertheless displaying everything properly. So at the very least you have that to fall back on, even if the reading experience is not as convenient as having a display of a single long thread.

pin-grid-array avatar Feb 26 '23 04:02 pin-grid-array

@highroller I have no explanation for the things you describe. The 2-tweet thread appearing out of order is especially puzzling to me.

The next time you see these things happening, please post a link to them and tag the dev, and hopefully they will be able to see it before it disappears from the timeline.

Even though web nitter is causing problems, it sounds like the RSS feed is still nevertheless displaying everything properly. So at the very least you have that to fall back on, even if the reading experience is not as convenient as having a display of a single long thread.

No, web nitter works fine. Rss has problem.

highroller avatar Feb 26 '23 06:02 highroller

No, web nitter works fine. Rss has problem.

Right! The RSS feed is sometimes out of order while web nitter's threads are fine. I don't know why I said it the other way. 😅

pin-grid-array avatar Feb 27 '23 00:02 pin-grid-array

Define order? Web nitter reorders tweets in a thread to show them from oldest to newest, while RSS is always chronological.

zedeus avatar Feb 27 '23 00:02 zedeus

Define order? Web nitter reorders tweets in a thread to show them from oldest to newest, while RSS is always chronological.

I just found that the tweets that are out of order in the thread have same timestamp. It confused the rss reader. E.g. 1,2,4,3,5... 3 and 4 had the same timestamp.

This missing tweets problem was most likely caused by what described by pin-grid-array. Is it possible to increase the rss limit from 20 to 100 or even 1000? It should go around the problem.

highroller avatar Mar 01 '23 23:03 highroller

It's possible, but it would hurt big instances with extreme RSS traffic.

zedeus avatar Jun 17 '23 01:06 zedeus