Access token expiry on OAuth2 authentication for long running script with authentication

[anusham009]

Hi,

1. Problem Statement. : Recursive function call to connect to Box API to get the box file metadata Access token and refresh tokens are refreshed at the expiry (1 hour) and written back to configuration file and the recent tokens are used continuously to authenticate

2. Issue noticed : At the expiry of the tokens (60-80 minutes) Box API fails with the below error message and the script terminates abruptly .

Client id — wfe8a4wdxfaflu6vugr4vtnu19wqhmdu API key — OAuth2.0 SDK version —BoxSDK (2.11.0)

3. Below are the logs .

Authentication Issue happening randomly even though the Refersh and Access Tokens are valid, Below is the error log . No handlers could be found for logger "boxsdk.network.default_network" Message: No "refresh_token" parameter found Status: 400 URL: https://api.box.com/oauth2/token Method: POST Headers: {'Transfer-Encoding': 'chunked', 'Set-Cookie': 'box_visitor_id=602b5dfae771f0.30863027; expires=Wed, 16-Feb-2022 05:54:02 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-44271; expires=Tue, 23-Feb-2021 05:54:02 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=13; expires=Wed, 16-Feb-2022 05:54:02 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Strict-Transport-Security': 'max-age=31536000', 'Connection': 'keep-alive', 'Cache-Control': 'no-store', 'Date': 'Tue, 16 Feb 2021 05:54:03 GMT', 'Content-Type': 'application/json'} Message: No "refresh_token" parameter found Status: 400 URL: https://api.box.com/oauth2/token Method: POST Headers: {'Transfer-Encoding': 'chunked', 'Set-Cookie': 'box_visitor_id=602b5dfc21a093.04408389; expires=Wed, 16-Feb-2022 05:54:04 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-44271; expires=Tue, 23-Feb-2021 05:54:04 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=91; expires=Wed, 16-Feb-2022 05:54:04 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Strict-Transport-Security': 'max-age=31536000', 'Connection': 'keep-alive', 'Cache-Control': 'no-store', 'Date': 'Tue, 16 Feb 2021 05:54:04 GMT', 'Content-Type': 'application/json'} Message: No "refresh_token" parameter found Status: 400 URL: https://api.box.com/oauth2/token Method: POST Headers: {'Transfer-Encoding': 'chunked', 'Set-Cookie': 'box_visitor_id=602b5dfd495a32.42643028; expires=Wed, 16-Feb-2022 05:54:05 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-44271; expires=Tue, 23-Feb-2021 05:54:05 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=21; expires=Wed, 16-Feb-2022 05:54:05 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Strict-Transport-Security': 'max-age=31536000', 'Connection': 'keep-alive', 'Cache-Control': 'no-store', 'Date': 'Tue, 16 Feb 2021 05:54:05 GMT', 'Content-Type': 'application/json’}

4. Upon debugging it is noticed that the tokens which are renewed and written back to configuration file is valid . This is verified by making a separate authentication check through CURL command . But the script terminates stating no refresh token found .

[PJSimon]

Hi @anusham009 ,

Thanks for submitting this Issue! We will take a look and get back to you ASAP!

@PJSimon

[anusham009]

Are there any updates /solutions for this issue?

[PJSimon]

Hi @anusham009 ,

Sorry for the delay. We are looking into our logs to see if we can better understand what's going on with you issue. We'll get back to you soon.

@PJSimon

[swfree]

Hi @anusham009, could you share the code snippet of how you're authenticating with OAuth2? There are a few different options to do so. I tested out OAuth2 authentication using the code in this demo, and the token refreshed automatically after running the script for over an hour. Are you seeing the 400 error every time your token expires or only intermittently?

[anusham009]

from future import print_function, unicode_literals

#import bottle import os from threading import Thread, Event import webbrowser from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, make_server

from boxsdk import OAuth2

import configparser from configparser import ConfigParser import sched, time import schedule

def authenticate(CLIENT_ID,CLIENT_SECRET,REFRESH_TOKEN,oauth_class=OAuth2): #Needed to get new refresh token

oauth = oauth_class(
    client_id=CLIENT_ID,
    client_secret=CLIENT_SECRET,
    refresh_token=REFRESH_TOKEN
)

try:
    access_token, refresh_token = oauth.refresh(None)
    #print(refresh_token)
    #print (access_token)
    return access_token, refresh_token
except Exception as e:
    print(e)
    print ('Refresh token expired')
   
#Needed to get new refresh token  
    

def Creds_Fetcher(): try: global creds_path #creds_path = '/Users/vishal.tp/Desktop/final/flask/CBot_Box_Creds.cfg' cwd = os.getcwd() creds_path = os.path.join(cwd,'Search-Test_Creds.cfg') config = configparser.ConfigParser() config.read(creds_path) ACCESS_TOKEN_Old = config.get('Search','ACCESS_TOKEN') REFRESH_TOKEN_Old = config.get('Search','REFRESH_TOKEN') print ('ACCESS_TOKEN_Old',ACCESS_TOKEN_Old) print ('REFRESH_TOKEN_Old',REFRESH_TOKEN_Old) return ACCESS_TOKEN_Old,REFRESH_TOKEN_Old except Exception as e: print ('Nope',e)

def Store_New_Logs(REFRESH_TOKEN_NEW,ACCESS_TOKEN_NEW): try: global creds_path config = configparser.ConfigParser() config.read(creds_path) config.set('Search','ACCESS_TOKEN',ACCESS_TOKEN_NEW) config.set('Search','REFRESH_TOKEN',REFRESH_TOKEN_NEW) with open(creds_path, 'w+') as configfile: config.write(configfile) except Exception as e: print (e)

def Token_Refresher():
ACCESS_TOKEN_Old,REFRESH_TOKEN_Old = Creds_Fetcher() CLIENT_ID = 'xxxx' # Insert Box client ID here CLIENT_SECRET = 'xxx' # Insert Box client secret here REFRESH_TOKEN = REFRESH_TOKEN_Old ACCESS_TOKEN_NEW,REFRESH_TOKEN_NEW = authenticate(CLIENT_ID,CLIENT_SECRET,REFRESH_TOKEN) print ('ACCESS_TOKEN_NEW',ACCESS_TOKEN_NEW) print ('REFRESH_TOKEN_NEW',REFRESH_TOKEN_NEW) Store_New_Logs(REFRESH_TOKEN_NEW,ACCESS_TOKEN_NEW) return ACCESS_TOKEN_NEW

Token_Refresher()

[anusham009]

Hi Patrick,

Above is the code snippet we used for token refreshment . could you please review the same and let know for any issues/ improvements here ? Store_New_Logs method stores the log in a txt file and the Token_Refresher method refresh the tokens using the existing stored tokens .

[Jeff-Meadows]

The SDK tries to handle token refresh automatically. If a client makes a request and the Box API indicates that the access token it used to make the request has expired, then the SDK will try to refresh it. If the client doesn't have access to the refresh token when this happens, it can result in the above error.

Can you share a little more info about how you're setting up the SDK to make the requests? I'm also curious as to why you need to manually refresh and store the credentials the way you're doing it, if you wouldn't mind sharing some context there.

Finally, it looks as though you've pasted a client ID and secret above. If these are real secrets, please rotate them immediately.

[anusham009]

The requirement is that we need to acces a box folder of very large data (1 million records approx) . So the intent behind storing the access token and refresh token in a file such that it can use the refresh token as and when the access token expires and authenticate again to connect to the folder. Earlier this was working at some point of time last year and hence we continued this approach . so do you mean to say the storing of tokens is what is creating the issue . Is there a sample code snippet where token refreshments are done while connecting to box and reading any files ?

[sujaygarlanka]

@anusham009 This seems like a python versioning problem. Are you using python 2.7? If so, it was deprecated in January this year and the cryptography libraries we use for no longer support 2.7, so this could be causing the issue. I got a similar PyNo handlers could be found for logger "boxsdk.network.default_network" that prevented my code from running when I ran it with Python 2.7, but it was resolved when ran with Python 3.

[anusham009]

@sujaygarlanka -- The script was originally written an year back with python2 . But I've now modified it accordingly to python 3 and the execution environment is python3 itself . Please could you recall if you had particularly made any changes in the modules imported for python3 which resolved the above error apart from execution envrionment as python3 ?

[Jeff-Meadows]

What I meant is that the SDK automatically refreshes when needed. And you can pass a callback into store_tokens to be made aware of when it happens, so you can store to a file. eg

oauth = oauth_class(
    client_id=CLIENT_ID,
    client_secret=CLIENT_SECRET,
    refresh_token=REFRESH_TOKEN,
    store_tokens= Store_New_Logs,
)

The callback passed into store_tokens will get called with the (access_token, refresh_token) pair whenever the SDK refreshes tokens. Your code already has a method to store the tokens in your file (although it has refresh token as the first parameter, so you'd need to reverse that).

Then you can get rid of your timer code that proactively refreshes your token.

[PJSimon]

Hi @anusham009,

Just trying to close the loop here. Did the above comment from @Jeff-Meadows help you? If not, please let us know, otherwise we'll go ahead and close this Issue.

Thanks!

[anusham009]

Yes thanks . The issue is resolved now ..

On Wed, May 5, 2021 at 3:50 AM Patrick Simon @.***> wrote:

Hi @anusham009 https://github.com/anusham009,

Just trying to close the loop here. Did the above comment from @Jeff-Meadows https://github.com/Jeff-Meadows help you? If not, please let us know, otherwise we'll go ahead and close this Issue.

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/box/box-python-sdk/issues/587#issuecomment-832286693, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRKNW7YUNSMTZCG5JHW4GLTMBXJ3ANCNFSM4ZURTFYQ .

[RevathiB12]

@anusham009 @PJSimon I am facing this issue when I use above code snippet

"POST https://api.box.com/oauth2/token" 400 69 {'Date': 'Wed, 18 May 2022 17:35:09 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000', 'Set-Cookie': 'box_visitor_id=62852e4db5b1a3.31574470; expires=Thu, 18-May-2023 17:35:09 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-45228; expires=Wed, 25-May-2022 17:35:09 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=89; expires=Thu, 18-May-2023 17:35:09 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Cache-Control': 'no-store'} {'error': 'invalid_grant', 'error_description': 'Invalid refresh token'}

Message: Invalid refresh token Status: 400 URL: https://api.box.com/oauth2/token Method: POST Headers: {'Date': 'Wed, 18 May 2022 17:35:09 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000', 'Set-Cookie': 'box_visitor_id=62852e4db5b1a3.31574470; expires=Thu, 18-May-2023 17:35:09 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-45228; expires=Wed, 25-May-2022 17:35:09 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=89; expires=Thu, 18-May-2023 17:35:09 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Cache-Control': 'no-store'} Refresh token expired Traceback (most recent call last): File "/Users/revathi/Desktop/refresh_tokenlogic.py", line 84, in main() File "/Users/revathi/Desktop/refresh_tokenlogic.py", line 81, in main Token_Refresher() File "/Users/revathi/Desktop/refresh_tokenlogic.py", line 70, in Token_Refresher ACCESS_TOKEN_NEW,REFRESH_TOKEN_NEW = authenticate(CLIENT_ID,CLIENT_SECRET,REFRESH_TOKEN) TypeError: cannot unpack non-iterable NoneType object

Note: I am using environment python3. Can you pls help me on this?

[stale[bot]]

This issue has been automatically marked as stale because it has not been updated in the last 30 days. It will be closed if no further activity occurs within the next 7 days. Feel free to reach out or mention Box SDK team member for further help and resources if they are needed.

[stale[bot]]

This issue has been automatically closed due to maximum period of being stale. Thank you for your contribution to Box Python SDK and feel free to open another PR/issue at any time.