Gazou run error

[Ulytor65]

Hi,

I successfully linked the API Twitter to Gazouilloire but when I type "gazou run" i get this error :

2023-06-27 15:34:23,771 - MainProcess [13320] - ERROR - Could not open C:\Users\XXX\config.json: <class 'json.decoder.JSONDecodeError'> Expecting ',' delimiter: line 18 column 2 (char 504)

has someone encountered the same issue ?

[boogheta]

The error seems to indicate you formatted your configuration file wrongly. There must be a missing comma somewhere at the end of line 18.

[Ulytor65]

As stupid as that... thank you ! But now I get this " Error while accessing the Twitter API, please retry: Twitter sent status 404 for URL: 1.1/statuses/filter.json using parameters: (filter_level=none&stall_warnings=true&track=%5B%27the%27%5D) " I'm sorry to bother you again, I'm kinda new to this but I wanna learn

[boogheta]

I guess your queries might be wrong, can you copy paste the keywords part of your config file ?

[Ulytor65]

Here it is

I hid my API Twitter key but I wonder if the issue is not there, maybe I need to activate something idk

    "twitter": {
        "key": "XXXXXXXXXXXXXXXXXXXXXXX",
        "secret": "Xxxxxxxxxxxxxxxxxxxxxxxxxx",
        "oauth_token": "XXXXXXXXXXXXXXXXXXXXXXX",
        "oauth_secret": "XXXXXXXXXXXXXXXXXXXXXXXl"
    },
    "database": {
        "host": "localhost",
        "port": 9200,
        "db_name": "datam2",
        "multi_index": false,
	"nb_past_months": 0
    },
    "keywords": [
        "gephi",
        "digital humanities",
	"@NEH_ODH",
	"@NEHgov"
    ],
    "url_pieces": [],
    "time_limited_keywords": {},
    "language": null,
    "geolocation": "",
    "resolve_redirected_links": true,
    "resolving_delay": 30,
    "grab_conversations": false,
    "catchup_past_week": true,
    "download_media": {
        "photos": false,
        "videos": false,
        "animated_gifs": false,
        "media_directory": "media"
    },
    "timezone": "Europe/Paris",
    "verbose": false
}

[boogheta]

There is something weird, the keywords you're giving do not correspond to the query that seems to be returned by twitter as an error, it should be the case if you had put the following instead:

    "keywords": [
        "['the']"
    ],

Have you maybe changed your config file in between, and in which case can you give us the latest error returned?

[Yomguithereal]

Also @Ulytor65, just to be sure you are not wasting your time here, do you happen to have a Twitter key for the v1 API (so quite an old one in fact, not one you created recently)? Because the v2 of the API is basically dead because of elno the turdlord.

[Ulytor65]

Hi, First, thank you both for your answers. I have two Twitter key, an old one (anterior to the date indicated in the tutorial and a new one that I've created recently. Problem is, both don't work. I think that the old one is not activated anymore or smth like that (it belongs to a pal). All in all, I think I'll search for another tool for crawling twitter. Thank you all for the help !

[Yomguithereal]

All in all, I think I'll search for another tool for crawling twitter.

I think you are out of luck. V2 is dead, V1 is half dead (you need to have a key which is still alive by chance), firehose is dead, the new APIs are basically so bad you cannot do any data collection with them. So there is still scraping (something our other tool minet is still able to do) but it is now slow as hell and with Elno latest stupidities this week end it is not unlikely the site will sink soon :)

[bmaz]

I reproduce the error reported by @Ulytor65 with a valid V1 key: ERROR - Error while accessing the Twitter API, please retry: Twitter sent status 404 for URL: 1.1/statuses/filter.json using parameters: (filter_level=none&stall_warnings=true&track=%5B%27the%27%5D) .

Moreover, downgrading to gazouilloire==1.3.1 fixed the problem.

[bmaz]

Hello @Ulytor65 ,

Version 1.5.0 should fix your issue. I know you switched to another tool, but if you had time to upgrade gazouilloire (pip install --upgrade gazouilloire) and re-test that would be great.