kafka-connect-twitter

kafka-connect-twitter copied to clipboard

Per https://developer.twitter.com/en/docs/tweets/data-dictionary/overview/tweet-object the text field is truncated for retweets, with the full value available in retweeted_status. Can retweeted_status be added to the schema for this connector?

At the moment any RT is only available in its first 140 characters (give or take for user name length etc):

ksql> select retweet, max(len(text)) as max_tweet_length  from all_tweets group by retweet emit changes;
+--------------------------------------------------------------------+--------------------------------------------------------------------+
|RETWEET                                                             |MAX_TWEET_LENGTH                                                    |
+--------------------------------------------------------------------+--------------------------------------------------------------------+
|false                                                               |411                                                                 |
|true                                                                |152                                                                 |

@rmoff in the API I am using getRetweetedStatus and getQuotedStatus are full Status objects. If I pulled these in would it work for you?

yes that would be 💯 what I need 👍