twikit icon indicating copy to clipboard operation
twikit copied to clipboard

get_trends deprecation update/fix

Open touchmeangel opened this issue 3 months ago β€’ 2 comments

Check issue #389. Updated from guilds.json to GenericTimelineById

Summary by Sourcery

Switch get_trends to use the GenericTimelineById GraphQL endpoint instead of the legacy v1.1 guide API, introduce feature flags and helper methods for explore_page and generic_timeline_by_id, add a mapping of timeline IDs for trending categories, and update the Trend model to parse snake_case fields from the new response.

Enhancements:

  • Introduce EXPLORE_PAGE and GENERIC_TIMELINE_BY_ID GraphQL endpoints and corresponding client methods
  • Add EXPLORE_PAGE_FEATURES, GENERIC_TIMELINE_FEATURES feature-flag constants and TIMELINE_IDS mapping for trend categories
  • Refactor get_trends to retrieve trends via generic_timeline_by_id and simplify category handling

Summary by CodeRabbit

  • New Features

    • Added support for exploring trends by category (trending, for-you, news, sports, entertainment).
    • Introduced new endpoint for accessing explore page content.
  • Bug Fixes

    • Improved data extraction and cursor handling for trends and replies, ensuring more reliable pagination.
  • Refactor

    • Standardized internal data model naming conventions for consistency.

touchmeangel avatar Oct 04 '25 12:10 touchmeangel

Reviewer's Guide

This PR replaces the old v11 guide-based trends fetch with a GraphQL GenericTimelineById call, adding new endpoints, feature-flag maps, and updating the Trend model to align with the new response shape.

Sequence diagram for updated get_trends flow using GenericTimelineById

sequenceDiagram
    participant Client
    participant GQL
    participant "GenericTimelineById Endpoint"
    participant Trend
    Client->>GQL: generic_timeline_by_id(timeline_id, count)
    GQL->>"GenericTimelineById Endpoint": gql_get(...)
    "GenericTimelineById Endpoint"-->>GQL: Response (entries)
    GQL-->>Client: Response (entries)
    loop For each entry
        Client->>Trend: Trend(self, entry['content']['itemContent'])
    end
    Client-->>Client: Return list of Trend objects

Updated class diagram for Trend model

classDiagram
    class Trend {
        - _client: Client
        - name: str
        - tweets_count: str | None
        - domain_context: str
        - grouped_trends: list[str]
        + __init__(client: Client, data: dict)
        + __repr__() -> str
    }
    Trend <-- Client

Class diagram for new GQL methods and endpoint constants

classDiagram
    class GQL {
        + explore_page()
        + generic_timeline_by_id(timeline_id, count)
    }
    class Endpoint {
        + EXPLORE_PAGE
        + GENERIC_TIMELINE_BY_ID
    }
    GQL --> Endpoint

File-Level Changes

Change Details Files
Refactor get_trends to use GenericTimelineById
  • Map input category to timeline_id via TIMELINE_IDS
  • Replace v11.guide call with gql.generic_timeline_by_id
  • Early-return empty list when timeline_id is missing
  • Adjust retry logic indentation
  • Filter entries by 'trend' prefix and build Trend objects
twikit/client/client.py
Add new GraphQL endpoints and client methods
  • Introduce EXPLORE_PAGE and GENERIC_TIMELINE_BY_ID endpoints
  • Implement explore_page() with EXPLORE_PAGE_FEATURES
  • Implement generic_timeline_by_id() with GENERIC_TIMELINE_FEATURES
twikit/client/gql.py
Define feature-flag constants and timeline IDs
  • Add EXPLORE_PAGE_FEATURES and GENERIC_TIMELINE_FEATURES maps
  • Add TIMELINE_IDS mapping for trending categories
twikit/constants.py
Update Trend model to match new API schema
  • Rename trendMetadata to trend_metadata and related fields
  • Rename metaDescription/domainContext/groupedTrends keys
  • Adjust tweets_count type and TODO-int conversion
twikit/trend.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an issue from a review comment by replying to it. You can also reply to a review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull request title to generate a title at any time. You can also comment @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in the pull request body to generate a PR summary at any time exactly where you want it. You can also comment @sourcery-ai summary on the pull request to (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the pull request to resolve all Sourcery comments. Useful if you've already addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull request to dismiss all existing Sourcery reviews. Especially useful if you want to start fresh with a new review - don't forget to comment @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

  • Contact our support team for questions or feedback.
  • Visit our documentation for detailed guides and information.
  • Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai[bot] avatar Oct 04 '25 12:10 sourcery-ai[bot]

[!WARNING]

Rate limit exceeded

@touchmeangel has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 16 minutes and 20 seconds before requesting another review.

βŒ› How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 4844287f434fe770f9d0b9defacca4db52b00b36 and a51f349b4b4ed110637907042ec2f0922933d059.

πŸ“’ Files selected for processing (1)
  • twikit/client/client.py (5 hunks)

Walkthrough

The PR refactors data retrieval for trends and timelines by introducing new GraphQL endpoints (EXPLORE_PAGE, GENERIC_TIMELINE_BY_ID) with accompanying feature flags and a timeline ID mapping. Client logic is updated to extract data from itemContent and safely handle cursors. The trend module adopts snake_case naming conventions for consistency.

Changes

Cohort / File(s) Summary
Build & Environment
.gitignore
Added __pycache__ to ignored patterns; re-specified /node_modules entry with no functional change.
Constants & Configuration
twikit/constants.py
Added three new public constants: EXPLORE_PAGE_FEATURES (feature flags for explore page), GENERIC_TIMELINE_FEATURES (feature flags for generic timelines), and TIMELINE_IDS (mapping of timeline categories to encoded IDs).
GraphQL API Layer
twikit/client/gql.py
Introduced two new endpoints: EXPLORE_PAGE and GENERIC_TIMELINE_BY_ID in Endpoint enum; added corresponding GQLClient methods explore_page() and generic_timeline_by_id() with appropriate feature flag configurations; updated imports for new feature constants.
Client Logic Updates
twikit/client/client.py
Added TIMELINE_IDS import; refactored _get_more_replies() to extract next_cursor from itemContent and only schedule fetch if cursor exists; updated get_tweet_by_id() to derive sr_cursor from itemContent with null-safety guards; refactored get_trends() to use new generic_timeline_by_id endpoint, extract trend data from itemContent, and apply updated cursor handling for reply continuation.
Data Model
twikit/trend.py
Renamed keys from camelCase to snake_case: trendMetadata β†’ trend_metadata, metaDescription β†’ meta_description, domainContext β†’ domain_context, groupedTrends β†’ grouped_trends; updated tweets_count type hint from `int

Sequence Diagram

sequenceDiagram
    actor User
    participant Client as twikit.client
    participant GQL as gql
    participant API as Twitter API

    User->>Client: get_trends()
    
    alt No timeline_id mapped
        Client-->>User: return []
    else Valid timeline_id exists
        Client->>GQL: generic_timeline_by_id(timeline_id, count)
        GQL->>API: POST GENERIC_TIMELINE_BY_ID endpoint
        API-->>GQL: response with entries
        
        GQL-->>Client: entries
        
        Client->>Client: Extract trend data from itemContent
        
        alt Valid cursor found
            Client->>Client: Schedule follow-up fetch for more replies
        end
        
        Client-->>User: Trend objects
    end

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • twikit/client/gql.py: Review new endpoint definitions and GQLClient methods to verify correct feature flag assignments and parameter passing.
  • twikit/client/client.py: Verify refactored cursor extraction logic (itemContent-based) across _get_more_replies(), get_tweet_by_id(), and get_trends() for consistency and null-safety; confirm early-exit conditions prevent malformed API calls.
  • twikit/trend.py: Validate snake_case renaming consistency; confirm tweets_count type change and TODO note align with actual data structure from API.

Poem

🐰 Whiskers twitching with delight,
New endpoints hopping into sight!
snake_case trails through trends so bright,
Cursors safely guarded right,
Timelines fetch with features tight! πŸŽ‰

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
βœ… Passed checks (2 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check βœ… Passed The title 'get_trends deprecation update/fix' accurately reflects the main change: updating the get_trends function from legacy API to GraphQL.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Oct 04 '25 12:10 coderabbitai[bot]