Truncated tweet texts returned, even tho explicitly filtering out retweets

[pannadf]

Describe the bug I'm seeing truncated tweet texts, even though I've explicitly filtered out retweets in my query.

To Reproduce

import requests
import os
import json

# To set your environment variables in your terminal run the following line:
# export 'BEARER_TOKEN'='<your_bearer_token>'
bearer_token = os.environ.get("BEARER_TOKEN")

search_url = "https://api.twitter.com/2/tweets/search/recent"

# Optional params: start_time,end_time,since_id,until_id,max_results,next_token,
# expansions,tweet.fields,media.fields,poll.fields,place.fields,user.fields
query_params = {'query': '(from:lightningai -is:retweet)',
                'tweet.fields': 'author_id,created_at,public_metrics', 
                'max_results':10}

def bearer_oauth(r):
    """
    Method required by bearer token authentication.
    """

    r.headers["Authorization"] = f"Bearer {bearer_token}"
    r.headers["User-Agent"] = "v2RecentSearchPython"
    return r

def connect_to_endpoint(url, params):
    response = requests.get(url, auth=bearer_oauth, params=params)
    print(response.status_code)
    if response.status_code != 200:
        raise Exception(response.status_code, response.text)
    return response.json()


def main():
    json_response = connect_to_endpoint(search_url, query_params)
    print(json.dumps(json_response, indent=4, sort_keys=True))


if __name__ == "__main__":
    main()

Expected behavior I would expect all the tweet texts returned to be the full text.

Additional context An example of what I'm seeing, where it's clear that the tweet is truncated at StableLM, Pythia,...:

{'author_id': '1157283331509235713', 'created_at': '2023-05-09T15:13:09.000Z', 'edit_history_tweet_ids': ['1655954064474279938'], 'id': '1655954064474279938', 'public_metrics': {'impression_count': 14351, 'like_count': 113, 'quote_count': 3, 'reply_count': 3, 'retweet_count': 29}, 'text': '⚡Lit-Parrot 🦜 - the simplest implementation of the best open-source language models out there is now available!\n\nFresh out of the cage👉 https://t.co/TWSVqU6izY\n\n💡“nanoGPT” minimal\n🛠️hackable\n📜comes with pre-training + finetuning scripts\n\nShips with StableLM, Pythia,… https://t.co/RypG8NLUgy https://t.co/nW2W80Cwup'}

Any guidance would be greatly appreciated. Thank you!

[naveedamir488569]

add note_tweet in expansion,

take refrence from this, i am getting full text data for origin tweet,

user.fields=created_at,description,entities,id,location,name,pinned_tweet_id,profile_image_url,protected,url,username,verified,withheld,public_metrics&expansions=attachments.media_keys,author_id,geo.place_id&tweet.fields=attachments,author_id,entities,public_metrics,lang,created_at,note_tweet,text&start_time=2023-09-14T23:35:16.000Z&max_results=10