twitter-scraper icon indicating copy to clipboard operation
twitter-scraper copied to clipboard

Error need its own typing, not just a string.

Open RuofengX opened this issue 1 year ago • 1 comments

My requirements is like these: Fetch all tweets of some users (from a database), if success then mark a done tag for this user in database, so that this user will no longer be fetched.

When calling the GetUserTweets api, there are 2 different error type:

  1. it may return an Error in the TweetResult struct when the user is suspend or delete its account.
  2. Other errors in tweet result.

For the first case, it's obviously that there's no way to fetch tweets that I need, so it is necessary to mark a done tag for this user in database, after that, this account will never be fetched again. However in seconde case, there's some other problem when fetching tweets, which means if I call the search function in a few minutes, it may return a success result.

Then I notice that There is no error type when handleing errors in golang. I must write a string-compare-if sentence to classify those errors.

I'm not sure whether there's another way to handle these.

Here is my code.

func DigUser(username string, num int, scraper *twitterscraper.Scraper) {
	//dig all tweets from a user, num as the limit
	fullSearchDone := true
	log.Printf("~~ START->%s", username)
	results, err := GetUserTweets(username, num, scraper)
	if err != nil {
		log.Printf("!! ERR WHEN SEARCH->%s", err)
		return
	}

	for result := range results {
		if result.Error != nil {
			if result.Error.Error() == fmt.Sprintf("User '%s' not found", username) {
				log.Printf("~! ACCOUNT DELETED->%s", username)
			} else if result.Error.Error() == "Authorization: User has been suspended. (63)" {
				log.Printf("~! ACCOUNT SUSPENDED->%s", username)
			} else {
				log.Printf("!! ERROR FROM RESULT->%s", result.Error)
				fullSearchDone = false  // here is why I need to compare the error string
			}
			continue
		}
		fullTweetsSaveChan <- &result.Tweet
	}
	if fullSearchDone {
		log.Printf("~! DIG DONE->%s", username)
		digUserDone(username)
	} else {
		log.Printf("~! DIG DONE, PARTIALLY")
	}
}

RuofengX avatar Jun 14 '23 05:06 RuofengX

Yes, that would be great, but I don't have time for that yet, PR is welcome

n0madic avatar Jun 14 '23 11:06 n0madic