Custom Nitter Instance Not Working

[EthanC]

Nitter, and many of its instances (including nitter.lovinator.space) are currently down due to Twitter changes (https://github.com/zedeus/nitter/issues/983). There are a handful of instances currently using a rewrite branch that are working, but setting the Nitter Instance in this app doesn't appear to function. In the logs, I see the settings are applied, but the app doesn't seem to ever fetch Tweets from the new instance. Restarting the app after applying the setting does not resolve the issue.

2023-08-23 05:09:02.479 | INFO     | discord_twitter_webhooks.reader_settings:get_data_location:36 - Data will be stored in /home/botuser/.local/share/discord_twitter_webhooks
2023-08-23 05:09:02.479 | DEBUG    | discord_twitter_webhooks.reader_settings:get_reader:68 - Data directory is owned by botuser:botuser
INFO:     Started server process [1]
INFO:     Waiting for application startup.
2023-08-23 at 05:09:02 INFO  I will check for new tweets every 1 minutes
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     X:52346 - "GET / HTTP/1.1" 200 OK
INFO:     X:52346 - "GET /settings HTTP/1.1" 200 OK
2023-08-23 at 05:28:13 DEBUG Saved application settings: ApplicationSettings(nitter_instance='https://nitter.cz', deepl_auth_key='', piped_instance='https://piped.video', teddit_instance='https://teddit.net', delay=1)
INFO:     X:52389 - "POST /settings HTTP/1.1" 200 OK

[EthanC]

@TheLovinator1 any chance you have time to look into this? It'd be great to be able to workaround the current Nitter issues!

[0xpr03]

Please do not use public instances for your bots and operate your own. We will probably have to rate limit you, or straight up block bots due to how brittle the current rate limit situation from twitter is. There is a high chance you are already blocked due to cloud providers being the biggest source of scrapers.

[EthanC]

After changing the Nitter instance, I noticed many URLs to the old instance in the database. I assume there's some kind of comparison happening here after the instance change that causes this issue.

Knowing this, I found a less-than-ideal workaround.

1. Delete discord_twitter_webhooks.db (this will reset everything to defaults)
2. Set the Nitter instance in Settings
3. Re-add all of your Feeds

Feeds will then function as expected, using the newly-set, non-default Nitter instance.

[dziban303]

I noticed this too, I thought it just saved the currently selected instance when creating a feed. Does it not use the new instance when you create a new feed? I can't remember if I tested it or not @EthanC

[EthanC]

Hm, I didn't test creating a new feed without first deleting the db. But I can confirm that deleting the db, setting the new instance, and then creating a new feed will in fact use the new instance.

[dziban303]

Ah it's just a SQL database, might be able to just change the instance for the feeds with sed or something

[dziban303]

OK, yes, creating a new feed after changing the default instance will apparently use the new instance. @TheLovinator1 given the present circumstances I think it'd be better if the currently selected default Nitter instance was used for all feeds instead of whatever instance was in use when the feed was originally created, and/or, allow changing the instance used for individual feeds.

Ethan I think we've annoyed him too much, lol.