Are Musk,s new limitations impacting this module?

[banajana]

These lines of code worked fine just a few weeks ago and now I'm getting TypeError: object of type 'NoneType' has no len()

This is the code I'm using: from twitter_scraper_selenium import get_profile_details

twitter_username = "tim_cook" filename = "twitter_dummy_ceo" get_profile_details(twitter_username=twitter_username, filename=filename)

[shaikhsajid1111]

Yes, this module is supposed to scrape public data only. Now, we can't access pages without logging in which leads to a blocker

[ansisme]

so does that mean we cant scrape data at all? without logging in ?

[shaikhsajid1111]

Yeah, without logging in it is not possible currently

[feusagittaire]

Yeah, without logging in it is not possible currently

I how could we implement the logging in? Could you show how in code?

[ansisme]

Yeah, without logging in it is not possible currently

I have logged in, still not working, what now?

[banajana]

So scraping tweets now works, although I'm getting random 10, not the latest 10. I'm using this code: from twitter_scraper_selenium import scrape_profile

scrape_profile(twitter_username="tim_cook",output_format="csv",browser="firefox",tweets_count=10,filename="tim_cook_tweetsV3",directory="C:/Users/MCOBPHD19/OneDrive/Documents/Dissertation")

But scraping profile details doesn't. Using: from twitter_scraper_selenium import get_profile_details

twitter_username = "tim_cook" filename = "timCookProfile" get_profile_details(twitter_username=twitter_username, filename=filename)

The error I get is this. Is this still the authentication issue? If so, why can I now scrape tweets? TypeError Traceback (most recent call last) C:\Users\MCOBPH~1\AppData\Local\Temp/ipykernel_3168/1681914123.py in 4 twitter_username = "tim_cook" 5 filename = "timCookProfile" ----> 6 get_profile_details(twitter_username=twitter_username, filename=filename)

~\anaconda3\lib\site-packages\twitter_scraper_selenium\profile_details.py in get_profile_details(twitter_username, proxy, filename, directory) 56 # if filename was not provided then print the JSON to console 57 return json.dumps(data) ---> 58 elif filename != '' and len(data) > 0: 59 # if filename was provided, save it to that file 60 mode = 'w'

TypeError: object of type 'NoneType' has no len()

[eloragh]

So scraping tweets now works, although I'm getting random 10, not the latest 10. I'm using this code: from twitter_scraper_selenium import scrape_profile

scrape_profile(twitter_username="tim_cook",output_format="csv",browser="firefox",tweets_count=10,filename="tim_cook_tweetsV3",directory="C:/Users/MCOBPHD19/OneDrive/Documents/Dissertation")

But scraping profile details doesn't. Using: from twitter_scraper_selenium import get_profile_details

twitter_username = "tim_cook" filename = "timCookProfile" get_profile_details(twitter_username=twitter_username, filename=filename)

The error I get is this. Is this still the authentication issue? If so, why can I now scrape tweets? TypeError Traceback (most recent call last) C:\Users\MCOBPH~1\AppData\Local\Temp/ipykernel_3168/1681914123.py in 4 twitter_username = "tim_cook" 5 filename = "timCookProfile" ----> 6 get_profile_details(twitter_username=twitter_username, filename=filename)

~\anaconda3\lib\site-packages\twitter_scraper_selenium\profile_details.py in get_profile_details(twitter_username, proxy, filename, directory) 56 # if filename was not provided then print the JSON to console 57 return json.dumps(data) ---> 58 elif filename != '' and len(data) > 0: 59 # if filename was provided, save it to that file 60 mode = 'w'

TypeError: object of type 'NoneType' has no len()

I'm struggling with this same issue. Is there any workaround?

[shaikhsajid1111]

There are still problems for users scraping without logging in. The tweets are not present the same as it is present for logged-in users on Twitter