OAuth 2.0 for Web Server Applications, "auth Related Error (invalid_rapt)" error

[ujja]

trafficstars

Hello,

I have implemented OAuth2 to access Dialogflow CX library (@google-cloud/dialogflow-cx) following this doc

Our application can have multiple dialogflow chatbots, so different refresh tokens. It can happen that 2 refresh tokens belongs to the same Google OAuth2 Client

I made 2 changes:

• I use prompt: 'consent' parameter to get refresh token. (It is because we have multiple connections)
• I use 2 versions of the auth library (see: https://github.com/googleapis/google-auth-library-nodejs/issues/1952)

I initialize the auth to access dialogflow:

  const authClient = new OAuth2Client(CLIENT_ID, CLIENT_SECRET)
  authClient.setCredentials({
    refresh_token: REFRESH_TOKEN
  })

  const params = { authClient }

It works for a day, or so, but then I got auth Related Error (invalid_rapt)

I tried to refresh token manually following the tutorial, and i found a method for it: authClient.refreshAccessToken()

Same result (I mean every failed with 400, but when I called refreshAccessToken(), I have seen this invalid_rapt )

[danieljbruce]

Please give us full steps to reproduce the issue including everything you did starting from scratch. If this includes any code samples, please provide the full code sample so that we can copy/paste the code to try to reproduce the issue.

[ujja]

I can reproduce it this way:

1. authenticate (http://localhost:8080/oauth2)
2. call dialogflow (http://localhost:8080/dialogflowcx)
3. wait a day
4. call dialogflow (http://localhost:8080/dialogflowcx)

.env:

YOUR_CLIENT_ID=
YOUR_CLIENT_SECRET=

DIALOGFLOW_API_ENDPOINT=
DIALOGFLOW_AGENT_PATH=

package.json:

{
  "name": "google-dialogflow-oauth",
  "version": "1.0.0",
  "description": "",
  "author": "",
  "type": "commonjs",
  "dependencies": {
    "@google-cloud/dialogflow-cx": "^5.0.1",
    "@google-cloud/storage-transfer": "^4.0.1",
    "body-parser": "^2.2.0",
    "crypto": "^1.0.1",
    "dotenv": "^16.5.0",
    "express": "^4.21.2",
    "express-session": "^1.18.1",
    "google-auth-library": "^10.0.0-rc.1",
    "googleapis": "^148.0.0",
    "passport": "^0.7.0",
    "passport-local": "^1.0.0"
  }
}

js:

const http = require('http');
const https = require('https');
const fs = require('fs');
const url = require('url')
const crypto = require('crypto');

const express = require('express');
const session = require('express-session');

const dotenv = require('dotenv');
const { AgentsClient } = require("@google-cloud/dialogflow-cx");
const { google } = require('googleapis');
const { OAuth2Client } = require("google-auth-library");

const scopes = [
  'https://www.googleapis.com/auth/dialogflow'
];

const createOAuthClient = () => {
  return new google.auth.OAuth2(
    process.env.YOUR_CLIENT_ID,
    process.env.YOUR_CLIENT_SECRET,
    'http://localhost:8080/oauth2callback'
  );
}

async function main() {
  dotenv.config();
  const app = express();


  app.use(session({
    secret: 'your_secure_secret_key', // Replace with a strong secret
    resave: false,
    saveUninitialized: false,
  }));

  app.get('/oauth2', async (req, res) => {
    const state = crypto.randomBytes(32).toString('hex');
    req.session.state = state;

    const oauth2Client = createOAuthClient();
    const authorizationUrl = oauth2Client.generateAuthUrl({
      access_type: 'offline',
      scope: scopes,
      include_granted_scopes: true,
      state: state,
      // CHANGE: always ask for consent to get refresh token every time
      prompt: 'consent'
    });

    res.redirect(authorizationUrl);
  });

  app.get('/oauth2callback', async (req, res) => {
    let q = url.parse(req.url, true).query;

    if (q.error) {
      console.log('Error:' + q.error);
      res.status(500).json({ message: q.error })
    } else { // Get access and refresh tokens (if access_type is offline)
      const oauth2Client = createOAuthClient();
      let { tokens } = await oauth2Client.getToken(q.code);

      fs.writeFileSync('./tokens.json', JSON.stringify(tokens), 'utf8')
      res.status(200).json({ message: 'OK' })
    }
  });

  app.get('/dialogflowcx', async (req, res) => {
    try {
      const tokens = JSON.parse(fs.readFileSync('./tokens.json', 'utf8'))

      // Using 2 versions of google-auth-library because:
      // https://github.com/googleapis/google-auth-library-nodejs/issues/1952
      const oauth2Client = new OAuth2Client(
        process.env.YOUR_CLIENT_ID,
        process.env.YOUR_CLIENT_SECRET
      );
      oauth2Client.setCredentials({refresh_token: tokens.refresh_token})

      // same result, if i try to get access token
      // const refresTokenResult = await oauth2Client.refreshAccessToken()
      // console.log(`refresTokenResult ===> ${JSON.stringify(refresTokenResult)}`)

      const client = new AgentsClient({
        authClient: oauth2Client,
        apiEndpoint: process.env.DIALOGFLOW_API_ENDPOINT
      })

      const agentData = await client.getAgent({
        name: process.env.DIALOGFLOW_AGENT_PATH
      })

      res.status(200).json({ message: 'OK', displayName: agentData[0]?.displayName })
    } catch (e) {
      console.error('Error:', e)
      res.status(500).json({ message: 'Error', error: e.message })
    }
  });

  const server = http.createServer(app);
  console.log(`Authentication on http://localhost:8080/oauth2`)
  console.log(`Calling dialogflow on http://localhost:8080/dialogflowcx`)
  server.listen(8080);
}
main().catch(console.error);

[ujja]

any news regarding this pls?

[ujja]

Some new facts:

1. The error comes just in case of dialogflow, drive was working after a day
2. I get this url in error response: https://support.google.com/a/answer/9368756

So as I see I can't use oauth2 for dialogflow without user interaction in long term?

ChatGPT said on refreshing access token i got sometimes a new refresh token. I tried to do it, but I get back always the same refresh token.

[sofisl]

What are these values? DIALOGFLOW_API_ENDPOINT= DIALOGFLOW_AGENT_PATH=

[ujja]

you mean what are the values for me?

DIALOGFLOW_API_ENDPOINT=europe-west2-dialogflow.googleapis.com
DIALOGFLOW_AGENT_PATH=projects/dialogflowcx-demo-302308/locations/europe-west2/agents/f17dc3a1-c93d-41f9-ba00-5a752750f380

But you cant use my values in OAuth2 because you are not permitted to use my chatbot.

[danieljbruce]

Thanks for the repro, but the repro needs exact steps that we can do on our laptops. For instance, I recognize DIALOGFLOW_API_ENDPOINT and DIALOGFLOW_AGENT_PATH will be different for us, but please explain how you populated those environment variables and other environment variables, how you authenticated in Step 1, what commands you ran in terminal and exact steps for everything else.

Thanks.

[ujja]

1. create a dialogflow cx agent. (It is sufficent to create an empty one)
2. execute this quickstart: https://www.npmjs.com/package/@google-cloud/dialogflow-cx#quickstart
3. fill .env. All data you need is there in the URL if you visit your agent in Dialogflow CX Console. URL is like: https://dialogflow.cloud.google.com/cx/projects/xxx/locations/xxx/agents/xxx.

• DIALOGFLOW_API_ENDPOINT={location}-dialogflow.googleapis.com. Possible values are here: https://cloud.google.com/dialogflow/cx/docs/reference/rest/v3-overview#rest_endpoints (first one if you are in global location)
• DIALOGFLOW_AGENT_PATH=projects/{project}/locations/{location}/agents/{agent}

I hope I put there everything important?

[sofisl]

I am not able to recreate your problem (I think it's because you didn't set up a handler for your server). Regardless, looking at the issue setup I'm noticing a few issues:

1. We do not recommend using two different versions of auth (that was the source of the issue linked above). If you can get all sources of auth onto v10, that would be best. That means updating all google-cloud and googleapis and auth packages to their latest versions.

2. I don't see a tokens event for handling refresh tokens. Since you aren't listening to a tokens event, I don't think you automatically refresh your access token. Note that you also cannot manually refresh your refresh token, it is only returned on the first consent screen: https://github.com/googleapis/google-api-nodejs-client/issues/750#issuecomment-304521450. I think this will also help direct the flow if a user doesn't have a valid refresh token, you can redirect them to a new consent screen to get a new refresh token.

3. I wonder if you should have different tokens for different chatbots. It's not clear to me what your setup is. For the next steps, can you do the following:

4. run rm -rf package-lock.json && rm -rf node_modules && npm i google-auth-library@latest @google-cloud/dialogflow-cx@latest googleapis@latest

5. Add a tokens event and set credentials there. Redirect the user to a new consent screen if the refresh token is expired.

If you still are experiencing a problem, please provide a full reproduction (a repo would be great). Ideally with a single command we should see the problem appear.

[ujja]

Still the same.

As i see:

The lifetime of the refresh token for dialogflow is different as for drive. For dialogflow the refresh token is invalidated, when my google-user-session is over. And thats why I got this invalid RAPT.

It is pretty bad for me.

I have a "conversation" button. If I push it, then we start to chat with dialoglow on backend side. I could add some check to the "conversation" button on UI side, and force the user to consent screen if the session is over as suggested, but i cant do it on server side. If the token is invalidated while it is running, then the whole process just fails.

Is there a way to do it better?

• is it possible to turn off this token invalidation for dialogflow?
• is it possible to check the expiry date of the refresh token? Or the user session? (At least I could display some warning, or info)
• any other idea?

Some side info: We have now service accounts as authentication. One of our cusomer is using monthly service account rotation. (It is a google security suggestion). It is pretty inconvinient, so we started to move to oauth2.

The whole error:

Error: Error: 400 undefined: Getting metadata from plugin failed with error: {"error":"invalid_grant","error_description":"reauth related error (invalid_rapt)","error_uri":"https://support.google.com/a/answer/9368756","error_subtype":"invalid_rapt"}
    at callErrorFromStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/call.js:32:19)
    at Object.onReceiveStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client.js:193:76)
    at Object.onReceiveStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:361:141)
    at Object.onReceiveStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:324:181)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/resolving-call.js:135:78
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
for call at
    at ServiceClientImpl.makeUnaryRequest (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client.js:161:32)
    at ServiceClientImpl.<anonymous> (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/@google-cloud/dialogflow-cx/build/src/v3/agents_client.js:262:29
    at /xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
    at repeat (/xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/retries.js:114:25)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/retries.js:156:13
    at OngoingCallPromise.call (/xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/call.js:70:27)
    at NormalApiCaller.call (/xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/createApiCall.js:110:30 {
  code: 400,
  details: 'Getting metadata from plugin failed with error: {"error":"invalid_grant","error_description":"reauth related error (invalid_rapt)","error_uri":"https://support.google.com/a/answer/9368756","error_subtype":"invalid_rapt"}',
  metadata: Metadata { internalRepr: Map(0) {}, options: {} },
  note: 'Exception occurred in retry method that was not classified as transient'
}
Error: Error: 400 undefined: Getting metadata from plugin failed with error: {"error":"invalid_grant","error_description":"reauth related error (invalid_rapt)","error_uri":"https://support.google.com/a/answer/9368756","error_subtype":"invalid_rapt"}
    at callErrorFromStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/call.js:32:19)
    at Object.onReceiveStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client.js:193:76)
    at Object.onReceiveStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:361:141)
    at Object.onReceiveStatus (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:324:181)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/resolving-call.js:135:78
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
for call at
    at ServiceClientImpl.makeUnaryRequest (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/client.js:161:32)
    at ServiceClientImpl.<anonymous> (/xxx/Projects/google-dialogflow-oauth/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/@google-cloud/dialogflow-cx/build/src/v3/agents_client.js:262:29
    at /xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
    at repeat (/xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/retries.js:114:25)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/retries.js:156:13
    at OngoingCallPromise.call (/xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/call.js:70:27)
    at NormalApiCaller.call (/xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
    at /xxx/Projects/google-dialogflow-oauth/node_modules/google-gax/build/src/createApiCall.js:110:30 {
  code: 400,
  details: 'Getting metadata from plugin failed with error: {"error":"invalid_grant","error_description":"reauth related error (invalid_rapt)","error_uri":"https://support.google.com/a/answer/9368756","error_subtype":"invalid_rapt"}',
  metadata: Metadata { internalRepr: Map(0) {}, options: {} },
  note: 'Exception occurred in retry method that was not classified as transient'
}

[danieljbruce]

If you still are experiencing a problem, please provide a full reproduction (a repo would be great). Ideally with a single command we should see the problem appear.

The exact steps you took are really important including every button you clicked on the gcp website to set up your environment and the exact changes you did on your laptop. To reproduce the problem we need those exact steps.

[sofisl]

It might be that the dialogflow token is different from the drive token, in which case I'd open an issue with dialogflow itself. However, I will close this issue for now because we still have yet to receive a small application with a complete set of instructions that reproduces the issue with none to minimal changes on our part. Here is an example of what we're looking for: a full project, with a full set of instructions, and I am able to reproduce the issue in the next comment. If you're able to create this faithful reproduction (even if it requires waiting a day, that's fine), please open a new issue and you can reference this one as well.