tweepy 401 error with github actions

[jtsw1990]

I have a short process that I am trying to schedule using github actions with tweepy's client.search_recent_tweets method.

Am getting a 401 unauthorized error, but I know that the bearer tokens are working as it runs perfectly on my local machine as well as on our old repository through github actions

I've read that streaming requires a sync with the timings but from my understanding this is a basic GET request. I've also printed timedatectl on the ubuntu machine and it is showing UTC.

I've also tried other auth methods but it seems like only Oauth2.0 is available for this endpoint.

Below is the error message from github actions, main source code and workflow.yml file.

Any help with this is appreciated.

Main source code

from libs import pg_loader as PGL
from resources.config import API
from datetime import datetime
import pandas as pd
import json
import tweepy
import os


def main_twitter_trend():

    date = datetime.today()
    date_stamp = date.strftime("%Y%m%d")
    client = tweepy.Client(os.environ["TWEEPY_API_KEY"])
    engine = PGL.create_engine(os.environ["DATABASE_URL"])

    influencer = API["TW"]["INFLUENCERS"]
    query = "select * from db_hashtags_list"
    result3 = pd.read_sql_query(query, con=engine)
    hashtags = result3["hashtag"].values.tolist()


    # Influencer Counter
    count = 0
    for influencer_i in influencer:
        query = f"from:{influencer_i} -is:retweet"
        tweets = tweepy.Paginator(
            client.search_recent_tweets,
            query=query,
            tweet_fields=["context_annotations", "created_at", "entities"],
            expansions=["entities.mentions.username", "author_id"],
            user_fields=["username"],
            max_results=100
        ).flatten(limit=1000)
        tweet_influence = pd.DataFrame(tweets)

Workflow.yml file

name: Canary data ingestion

on:
  #schedule:
  #  - cron: "0 0 * * *"
  push:
    branches: [ main ]

permissions:
  contents: read
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python 3.8
      uses: actions/setup-python@v3
      with:
        python-version: "3.8"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install flake8 pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with flake8
      run: |
        # turning off flake8 for now as it will not work with bybt globals
        # stop the build if there are Python syntax errors or undefined names
        # flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide        
        # flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
    - name: Test with pytest
      run: |
        # Remove for now
        # pytest
    - name: Execute data load
      run: |
        timedatectl
        python ingest_main.py
      env:
        DATABASE_URL: ${{ secrets.DATABASE_URL }}
        CMC_API_KEY: ${{ secrets.CMC_API_KEY }}
        TWEEPY_API_KEY: ${{ secrets.TWITTER_API_KEY }}
        BYBT_API_KEY: ${{ secrets.BYBT_API_KEY }}

[caretak3r]

There are 3 levels of support for that endpoint: https://developer.twitter.com/en/docs/twitter-api/tweets/search/api-reference/get-tweets-search-recent

OAuth 2.0 app only OAuth 2.0 PKCE OAuth 1.0

Since twitter's responses are generic, and generally unhelpful, can you ensure that the secret in GH is correct?

[Harmon758]

As noted, that endpoint supports OAuth 1.0a as well. If it works locally and elsewhere through GitHub Actions, then it's almost certainly not an issue with Tweepy. Double check that TWEEPY_API_KEY is your bearer token and that it's correct. Regenerate it if necessary.

[jtsw1990]

Thank you for the help/comments on this. Am closing this issue as it works with a newly generated key for some reason.