VueTube-Extractor icon indicating copy to clipboard operation
VueTube-Extractor copied to clipboard

Retrieve information for comments (/next endpoint)

Open 404-Program-not-found opened this issue 2 years ago • 1 comments

This one is slightly complicated by how Youtube's android/ios API responds. Whereas most other aspects of the API respond in a fairly straightforward manner, the order of the comment and the data for the comments are separated.

For example, given this request

curl --request POST \
  --url 'https://www.youtube.com/youtubei/v1/next?key=AIzaSyDCU8hByM-4DrUqRUYnGn-3llEO78bcxq8' \
  --header 'Content-Type: application/json' \
  --data '{
    "context": {
        "client": {
            "clientName": "ANDROID",
            "clientVersion": "17.20",
            "hl": "en"
        }
    },
    "continuation": "Eg0SC2RRdzR3OVdnWGNRGAYyJSIRIgtkUXc0dzlXZ1hjUTAAeAJCEGNvbW1lbnRzLXNlY3Rpb24%3D"
}'

The response will be in two parts as follows (parts are cut for readability)

JSON Response
{
  "continuationContents": {
    "itemSectionContinuation": {
      "contents": [
        {
          "elementRenderer": {
            "newElement": {
              "childElements": [
                {
                  "type": {
                    "componentType": {
                      "model": {
                        "commentThreadModel": {
                          "hidePinnedComments": true,
                          "trackingParams": "CK0REMJ1GI4CIhMIzvLZhsmM_QIVXKFLBR1JegGU",
                          "hideSeparator": false,
                          "commentKey": "EhpVZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZyAoKAE%3D",
                          "sharedKey": "EgAgKygB",
                          "surfaceKeys": {
                            "commentSurfaceKey": "Eh1VZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZy8xMiBPKAE%3D",
                            "toolbarSurfaceKey": "Eh1VZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZy8xMiBGKAE%3D"
                          },
                          "pinnedStateKey": "EhpVZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZyBNKAE%3D",
                          "composerDraftEntityKey": "N/A",
                          "inlineRepliesKey": "empty_inline_replies"
                        }
                      }
                    }
                  }
                }
              ]
            }
          }
        }
      ]
    }
  },
  "frameworkUpdates": {
    "entityBatchUpdate": {
      "mutations": [
        {
          "entityKey": "EhpVZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZyAoKAE%3D",
          "type": "ENTITY_MUTATION_TYPE_REPLACE",
          "payload": {
            "commentEntityPayload": {
              "key": "EhpVZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZyAoKAE%3D",
              "properties": {
                "commentId": "UgzarqjaaPC7TbFINNx4AaABAg",
                "content": {
                  "content": "1 BILLION views for Never Gonna Give You Up!  Amazing, crazy, wonderful! Rick ♥️"
                },
                "publishedTime": "1y ago",
                "replyLevel": 0,
                "authorButtonA11y": "Rick Astley",
                "toolbarStateKey": "EhpVZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZyAsKAE%3D",
                "translateButtonEntityKey": "EhpVZ3phcnFqYWFQQzdUYkZJTk54NEFhQUJBZyD_ASgB"
              },
              "author": {
                "channelId": "UCuAXFkgsw1L7xaCfnd5JJOw",
                "displayName": "Rick Astley",
                "avatarThumbnailUrl": "https://yt3.ggpht.com/BbWaWU-qyR5nfxxXclxsI8zepppYL5x1agIPGfRdXFm5fPEewDsRRWg4x6P6fdKNhj84GoUpUI4=s88-c-k-c0x00ffffff-no-rj",
                "isVerified": false,
                "isCurrentUser": false,
                "isCreator": true,
                "channelCommand": {
                  "innertubeCommand": {
                    "browseEndpoint": {
                      "browseId": "UCuAXFkgsw1L7xaCfnd5JJOw",
                      "canonicalBaseUrl": "/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
                    }
                  }
                },
                "innerBadgeA11y": "Rick Astley, Official Artist Channel",
                "isArtist": true
              },
              "toolbar": {
                "likeCountLiked": "1.1M",
                "likeCountNotliked": "1.1M",
                "replyCount": "500",
                "creatorThumbnailUrl": "https://yt3.ggpht.com/BbWaWU-qyR5nfxxXclxsI8zepppYL5x1agIPGfRdXFm5fPEewDsRRWg4x6P6fdKNhj84GoUpUI4=s88-c-k-c0x00ffffff-no-rj",
                "likeButtonA11y": "Like this comment along with 1.1M other people",
                "engagementToolbarStyle": {
                  "value": "ENGAGEMENT_TOOLBAR_STYLE_VALUE_DEFAULT"
                },
                "likeCountA11y": "1.1M likes",
                "replyCountA11y": "500 replies"
              },
              "avatar": {
                "image": {
                  "sources": [
                    {
                      "url": "https://yt3.ggpht.com/BbWaWU-qyR5nfxxXclxsI8zepppYL5x1agIPGfRdXFm5fPEewDsRRWg4x6P6fdKNhj84GoUpUI4=s88-c-k-c0x00ffffff-no-rj",
                      "width": 88,
                      "height": 88
                    }
                  ]
                },
                "accessibilityText": "Rick Astley",
                "avatarImageSize": "AVATAR_SIZE_XS",
                "endpoint": {
                  "innertubeCommand": {
                    "browseEndpoint": {
                      "browseId": "UCuAXFkgsw1L7xaCfnd5JJOw",
                      "canonicalBaseUrl": "/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
                    }
                  }
                }
              }
            }
          }
        }
      ]
    }
  }
}

Note the commentKey and entityKey fields under continuationContents and frameworkUpdates respectively. I am not sure how best to parse this, so any input will be appreciated.

Of course, there is the workaround of calling the web version of the API instead, but that will mean the comments in VueTube will not be able to access some app-exclusive features from the Youtube app without some serious workarounds.

404-Program-not-found avatar Feb 11 '23 03:02 404-Program-not-found

Of course, there is the workaround of calling the web version of the API instead, but that will mean the comments in VueTube will not be able to access some app-exclusive features from the Youtube app without some serious workarounds.

In my opinion, it might make more sense to use the web API. The Android one is guaranteed to almost always give you a 1-2MB JSON response (especially the browse endpoint), so it'll just slow things down and give you headaches. It works great for the official apps because they're using protobuf, which is much faster and lightweight, but definitely not here.

LuanRT avatar Feb 11 '23 05:02 LuanRT