unidentified value edit history

[sjgknight]

Problem

I am using the current version of rtweet (1.0.1) and have developer access setup in twitter, with bearer token auth in rtweet.

On running

rtweet::search_fullarchive("query terms", env_name = "myenv")

I get the error:

Error in tweet(x$quoted_status) : Unidentified value: edit_history, edit_controls, editable. Please open an issue and notify the maintainer. Thanks!

Perhaps because of the recent twitter change to introduce edit?

Session info

There are a bunch of very ordinary packages, nothing that had conflicts.

## copy/paste output
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

[llrs]

Yes, twitter decided to add new fields without notifying API developers/users. I'm working to fix this, meanwhile you can use the devel branch to parse the tweets without the edit fields (and no error, just a warning).

[Ariyanjyr]

Great! Is there any indication on how long this may take? I am working on my thesis and if this takes quit some time, then I have to find other ways to get the tweets.

Thanks!

[llrs]

rtweet works with the edit values if you use parse = FALSE but you would need to parse the data yourself. If you want it parsed there is a workaround in the devel version, you can retrieve the parsed tweets but without those edit* columns if you install it (remotes::install_github("ropensci/rtweet@devel")).

I hope to fix this by the end of the month when I hope to make a new release in CRAN.

Another option would be if you contributed to rtweet with a PR to adding this feature, this will help you and all the users of rtweet (in the timeline you wish). I would help and review the changes at good speed.

[Ariyanjyr]

I am quit new to extracting tweets from twitter with the rtweet package. What do you mean by the workaround or the other solution? If I use parse=false in the search_fullarchive I am still not getting the tweets. Only a list of 10 with each list NULL.

Op do 13 okt. 2022 om 09:48 schreef Lluís Revilla @.***>

rtweet works with the edit values if you use parse = FALSE but you would need to parse the data yourself. If you want it parsed there is a workaround in the devel version, you can retrieve the parsed tweets but without those edit* columns if you install it ( @.***")).

I hope to fix this by the end of the month when I hope to make a new release in CRAN.

Another option would be if you contributed to rtweet with a PR to adding this feature, this will help you and all the users of rtweet (in the timeline you wish). I would help and review the changes at good speed.

— Reply to this email directly, view it on GitHub https://github.com/ropensci/rtweet/issues/738#issuecomment-1277178032, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3SEANPLRVYJAWVAXAQ7GRTWC65DBANCNFSM6AAAAAARC2QYTQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Ariyan Jahanyar S2856549 0640695417

[llrs]

Could you post code that when I execute it would reproduce your problem when you use parse = FALSE ?

If you install the devel version you shouldn't have this problem as I converted the error to a warning while omitting parsing that information.

The other solution, I mentioned is that you would write code for yourself and all the other rtweet users. It is nice to have code for free :wink:

[Ariyanjyr]

So this is what I am running:

dataBTC1 <- search_fullarchive("Bitcoin", n = 1000, env_name = "Tweets", fromDate = "201501010000", parse = FALSE)

And this is the output:

[[1]] NULL

[[2]] NULL

[[3]] NULL

[[4]] NULL

[[5]] NULL

[[6]] NULL

[[7]] NULL

[[8]] NULL

[[9]] NULL

[[10]] NULL

I am assuming that I am not doing anything wrong?

EDIT: the other neccessary code is mentioned in my other comment. Do you also need all my tokens? I rather not share them here as anyone can see it

[llrs]

You shouldn't share your tokens, I can use my own. I will test if I can reproduce this and report back, it could be that you have run out of tweets to retrieve via the full archive as they are limited in a monthly based. However, that would be unrelated to the edit problems initially reported.

[Ariyanjyr]

I have one more questions regarding the attributes of rtweet. One of the attributes is: is:retweet, this attributes returns only retweets if I am correct. However, when I try -is:retweet, it returns to me the API error code of 422. Is it not possible to disregard retweets in the package rtweet with the attribute -is:retweet? An example:

search_30days("Bitcoin lang:en -is:retweet", n = 100, fromDate = "202210280000", toDate = "202210282359", parse = FALSE)

The above returns the error of code 422, while the following works:

search_30days("Bitcoin lang:en", n = 100, fromDate = "202210280000", toDate = "202210282359", parse = FALSE)

[llrs]

@Ariyanjyr please open a different issue, if this is an issue with rtweet. Add the necessary information and check that what you report is correct (there is no search_30days but there is a function search_30day) of the problem and what you have done to solve the issue.

The code you get means: This is returned due to invalid parameters in the query -- e.g. invalid PowerTrack rules. I'm not sure if this should work or you should negate it via NOT or other operators, but according to this page it seems you should use "Bitcoin lang:en -filter:retweet", -is:retweet seems to work in the API v2 as per this help page.

[kenkoonwong]

I'm getting `this

Error in tweet(x$quoted_status)

after

test <- search_30day(q = "#IDweek2022", n = 2000, fromDate = "202210190000", toDate = "202210260000", token = auth, env_name = "idweek", premium = TRUE)

[llrs]

@kenkoonwong as I said in the rOpenSci forum please always post the complete error and this was "fixed" in the devel branch as I mentioned there and in this comment.

You asked about what you should do in another question in the forum. You don't provide information about why or what you are trying to do I don't know your situation to recommend one or the other option. However, to users who don't want to parse the data themselves and are not proficient with R I would recommend to wait for next release or if they can manage to parse the data correctly then can use parse = FALSE and parse the fields you want/need. Good luck. Installing devel is just for those that are in a hurry or know what are the risks involved. Please keep in mind that I "work" in this in my own free time and that I am not available 24/7 to answer all you questions.

[kenkoonwong]

@llrs My apologies. I will use parse = F. Thanks for doing what you do!

[llrs]

Closing the issue, this edit_ fields are currently ignored in all the functions (That use API v1), if you want to use it, don't parse the data (parse = FALSE).

The functions supporting the API v2 will provide support for it at some moment (but not at the time of writing this in the 2 streaming endpoints of API v2 that provides access to).