nitter icon indicating copy to clipboard operation
nitter copied to clipboard

18+ Tweets no long available

Open kaijudude opened this issue 1 year ago • 57 comments

Any 18+ Tweet that could be viewed before can't be any longer, did some update age restrict them? If so, then how do I switch it off?

kaijudude avatar Apr 07 '23 21:04 kaijudude

I believe this may not be due to a Nitter change. There's been a change Twitter has made that places 18+ content behind an age verification check. Since there's no way they can verify that users (that consume content from their API) are of appropriate age, they just dont include them.

markcellus avatar Apr 07 '23 22:04 markcellus

I believe this may not be due to a Nitter change. There's been a change Twitter has made that places 18+ content behind an age verification check. Since there's no way they can verify that users (that consume content from their API) are of appropriate age, they just dont include them. Well crap, that sucks. Hope they can find a way around it.

kaijudude avatar Apr 07 '23 22:04 kaijudude

I believe this may not be due to a Nitter change. There's been a change Twitter has made that places 18+ content behind an age verification check. Since there's no way they can verify that users (that consume content from their API) are of appropriate age, they just dont include them.

I can't believe that that's entirely correct, I still see some thumbnailing services, such as vxtwitter, fxtwitter, still functioning in this case. That said, I also note it started occurring on nitter.net itself a few days ago while it only seemingly just started hitting some mirror sites, so I suppose it's possible that it's a slow rollout, but it's still also possible these other sites are doing something that nitter isn't

gentlecolts avatar Apr 08 '23 02:04 gentlecolts

IIRC based on some warnings I've seen when using yt-dlp plus other stuff:

  • The new graphql API won't show 18+ tweets to guest users (aka nitter)
  • the old api that's getting shut down does, but only in some regions

Edit: nitter.net uses more graphql endpoints than most instances because it's running a dev version

AlyoshaVasilieva avatar Apr 08 '23 11:04 AlyoshaVasilieva

Will the workaround be added to the browser instances?

crockpoot avatar Apr 09 '23 00:04 crockpoot

IIRC based on some warnings I've seen when using yt-dlp plus other stuff:

  • The new graphql API won't show 18+ tweets to guest users (aka nitter)
  • the old api that's getting shut down does, but only in some regions

Edit: nitter.net uses more graphql endpoints than most instances because it's running a dev version

Idk,. It seems like Twitter's API has changed

I haven't found one instance that works now. I tried many of those not updated.

ultrajae avatar Apr 09 '23 02:04 ultrajae

IIRC based on some warnings I've seen when using yt-dlp plus other stuff:

  • The new graphql API won't show 18+ tweets to guest users (aka nitter)
  • the old api that's getting shut down does, but only in some regions

Edit: nitter.net uses more graphql endpoints than most instances because it's running a dev version

Idk,. It seems like Twitter's API has changed

I haven't found one instance that works now. I tried many of those not updated.

That is definitely the case. I tried them all and a few unofficial ones. And some of the ones I tested have not been updated in months.

Mine has my workaround already applied and works tho. https://nitter.privacydev.net/

PrivacyDevel avatar Apr 09 '23 08:04 PrivacyDevel

How does the workaround… work? Whose credentials are used?

roryyamm avatar Apr 09 '23 17:04 roryyamm

How does the workaround… work? Whose credentials are used?

The administrator of the respective instance needs to create a Twitter account, sign in and then put their session information into Nitter's configuration file. This session information will then basically be shared for all users. It's the instance's Twitter account, if you will.

As an end user, you shouldn't need to do anything.

PrivacyDevel avatar Apr 09 '23 19:04 PrivacyDevel

Won't that lead to severe rate limiting, though?

roryyamm avatar Apr 09 '23 19:04 roryyamm

Won't that lead to severe rate limiting, though?

It seems like that does indeed happen sometimes. But for the most part it seems like it's actually working fairly well for my instance. But I am not yet aware of any other satisfying workaround for that problem. Forcing a user to login and get tracked is not really a satisfying solution for me.

I see the following general solutions for this problem, personally:

  1. rate limit the end users to make it so that the instance works more reliably overall
  2. create more accounts and make the server juggle between different accounts / sessions as needed
  3. maybe try to cache things more aggressively
  4. a combination of those options that I just listed

PrivacyDevel avatar Apr 09 '23 19:04 PrivacyDevel

Luckily, the workaround works but it's unfortunately very slow.

crockpoot avatar Apr 09 '23 20:04 crockpoot

indeed, slow. probably rate limited already I guess unfortunately.

I am waiting so far 5 minutes for a vid clip to play. gonna have to give up

At least the tweet comes up tho, A+ on effort

TYVM

ultrajae avatar Apr 09 '23 20:04 ultrajae

Forcing a user to login and get tracked is not really a satisfying solution for me.

What about a voluntary system - ask for a login to increase the rate limit, but fall back on a set account if no login is provided?

I use Nitter mostly for its RSS feeds. Since I have a lot of NSFW accounts as feeds, some of whom tweet frequently, the frequency by which I ask for tweets is probably a major reason why the instance is so slow. If I could somehow pass my own credentials when fetching those RSS feeds, even if it meant I was tracked, I'd do it if that meant a faster or more reliable system with less caching for all.

Perhaps users willing to forgo the non-tracking nature could put a query string at the end of the URL for the RSS feed, containing the exact ct0, auth_token and csrf-token, and the server could use those credentials instead of its own if all are present.

roryyamm avatar Apr 09 '23 20:04 roryyamm

indeed, slow. probably rate limited already I guess unfortunately.

I am waiting so far 5 minutes for a vid clip to play. gonna have to give up

At least the tweet comes up tho, A+ on effort

TYVM

I just made it so that videos aren't proxied anymore by default on my server to take some load off of it, make it so that people can actually watch videos again on my instance and to hopefully make Twitter not rate limit my instance as harshly.

And I also made it so that the request of userIds is made unauthenticated. Hopefully that makes it so the instance doesn't get rate limited as fast anymore as well.

PrivacyDevel avatar Apr 09 '23 21:04 PrivacyDevel

Forcing a user to login and get tracked is not really a satisfying solution for me.

What about a voluntary system - ask for a login to increase the rate limit, but fall back on a set account if no login is provided?

I use Nitter mostly for its RSS feeds. Since I have a lot of NSFW accounts as feeds, some of whom tweet frequently, the frequency by which I ask for tweets is probably a major reason why the instance is so slow. If I could somehow pass my own credentials when fetching those RSS feeds, even if it meant I was tracked, I'd do it if that meant a faster or more reliable system with less caching for all.

Perhaps users willing to forgo the non-tracking nature could put a query string at the end of the URL for the RSS feed, containing the exact ct0, auth_token and csrf-token, and the server could use those credentials instead of its own if all are present.

A URL-based solution would work the best for my needs, if possible.

crockpoot avatar Apr 09 '23 21:04 crockpoot

indeed, slow. probably rate limited already I guess unfortunately. I am waiting so far 5 minutes for a vid clip to play. gonna have to give up At least the tweet comes up tho, A+ on effort TYVM

I just made it so that videos aren't proxied anymore by default on my server to take some load of my poor server, make it so that people can actually watch videos again on my instance and to hopefully make Twitter not rate limit my instance as harshly.

And I also made it so that the request of userIds is made without unauthenticated. Hopefully that makes it so the instance doesn't get rate limited as fast anymore as well.

Thanks for that

Now working flawlessly

Hopefully it can last but appreciate what you're doing nonetheless!

ultrajae avatar Apr 09 '23 23:04 ultrajae

The search tab still shows the complete feed. Could fetching this way be a workaround? However, it seems to depend on the region again, e.g. US works, DE does not.

jve-engineering avatar Apr 10 '23 22:04 jve-engineering

The search tab still shows the complete feed. Could fetching this way be a workaround? However, it seems to depend on the region again, e.g. US works, DE does not.

That bizarrely works (not with nitter.net but with other instances at least) and adding the "/rss" at the end cleanly enables rss functionality lol thank you very much for this.

crockpoot avatar Apr 10 '23 23:04 crockpoot

I understand that the solution that is being proposed will probably only be applied by a minority of instances, right?

I see a problem and that is that the +18 tweets seem to disappear completely. I don't know about the web, but in the RSS feed there is not a single reference. This implies that users, far from being able to look for an alternative, won't even be aware of the problem.

Would it be possible for +18 tweets to be included without content (only metadata), even if the instance does not use an account?

ghost avatar Apr 11 '23 06:04 ghost

The search tab still shows the complete feed. Could fetching this way be a workaround? However, it seems to depend on the region again, e.g. US works, DE does not.

That's a great find! The only thing that seems to be missing there is the pinned tweet.

PrivacyDevel avatar Apr 11 '23 09:04 PrivacyDevel

And threads and retweets unfortunately also don't show up properly in the search

PrivacyDevel avatar Apr 11 '23 09:04 PrivacyDevel

Unrelated to the last few posts: It's probably also worth noting that NSFW users don't show up in the search

PrivacyDevel avatar Apr 11 '23 09:04 PrivacyDevel

Search uses a non-GraphQL endpoint, hence why it isn't affected in the same way. I've considered using search for timelines before, but it provides a much worse experience and is less reliable. I'll look into including sensitive tweets in timelines/RSS instead of omitting them.

I looked into this issue to understand it better, and it seems there's no way around it when using the GraphQL APIs, except authenticating with an account (which supposedly stays working even after the account gets suspended).

zedeus avatar Apr 12 '23 15:04 zedeus

IIRC based on some warnings I've seen when using yt-dlp plus other stuff:

  • The new graphql API won't show 18+ tweets to guest users (aka nitter)
  • the old api that's getting shut down does, but only in some regions

Edit: nitter.net uses more graphql endpoints than most instances because it's running a dev version

Idk,. It seems like Twitter's API has changed I haven't found one instance that works now. I tried many of those not updated.

That is definitely the case. I tried them all and a few unofficial ones. And some of the ones I tested have not been updated in months.

Mine has my workaround already applied and works tho. https://nitter.privacydev.net/

image Hmm, doesn't appear like it's working for me. Anything I'm doing wrong or is this an issue on your end?

RepresentativeCry66 avatar Apr 15 '23 04:04 RepresentativeCry66

IIRC based on some warnings I've seen when using yt-dlp plus other stuff:

  • The new graphql API won't show 18+ tweets to guest users (aka nitter)
  • the old api that's getting shut down does, but only in some regions

Edit: nitter.net uses more graphql endpoints than most instances because it's running a dev version

Idk,. It seems like Twitter's API has changed I haven't found one instance that works now. I tried many of those not updated.

That is definitely the case. I tried them all and a few unofficial ones. And some of the ones I tested have not been updated in months. Mine has my workaround already applied and works tho. https://nitter.privacydev.net/

image Hmm, doesn't appear like it's working for me. Anything I'm doing wrong or is this an issue on your end?

The global search doesn't show nsfw results. That didn't even work when I tried doing it with my usual authentication headers for some reason. When you access a profile directly, it should work tho (unless one of my instances / accounts (signified by the number in the parenthesis) is currently rate limited). Just open https://nitter.privacydev.net/NAME_OF_NSFW_USER

PrivacyDevel avatar Apr 15 '23 10:04 PrivacyDevel

Twitter blocks certain searches, including that hashtag. You can check by doing it on Twitter and checking the adaptive.json response in devtools, which should contain an error about the query being invalid.

zedeus avatar Apr 15 '23 12:04 zedeus

For those searches to work, the account owner needs to go to https://twitter.com/settings/search and uncheck "Hide sensitive content".

WaywardHeart avatar Apr 16 '23 05:04 WaywardHeart

For those searches to work, the account owner needs to go to https://twitter.com/settings/search and uncheck "Hide sensitive content".

Thank you! Fixed it.

PrivacyDevel avatar Apr 16 '23 05:04 PrivacyDevel

@PrivacyDevel could you link to your fork or give any info on the changes made allowing for authentication? thanks!

DrSocket avatar Apr 21 '23 00:04 DrSocket