exact-php-client icon indicating copy to clipboard operation
exact-php-client copied to clipboard

Could not connect to Exact: Could not acquire or refresh tokens [http 400]

Open meeshoogendoorn opened this issue 2 years ago • 31 comments

I have seen some issues about this topic going around tried all possible solution but couldn't find any solution for the problem. I can't see what i am doing/is going wrong. This is y connect function

 public function connect($request)
    {
        $connection = new \Picqer\Financials\Exact\Connection();
        $connection->setRedirectUrl(env("EXACT_CALLBACK_URL"));
        $connection->setExactClientId(env("EXACT_CLIENT_ID"));
        $connection->setExactClientSecret(env("EXACT_CLIENT_SECRET"));
    
        // Retrieves authorizationcode from database
        if ($request->session()->has('authorizationcode')) {
            $connection->setAuthorizationCode($request->session()->get('authorizationcode'));
        }
    
        // Retrieves accesstoken from database
        if (getValue('accesstoken')) {
            $connection->setAccessToken(getValue('accesstoken'));
        }
    
        // Retrieves refreshtoken from database
        if (getValue('refreshtoken')) {
            $connection->setRefreshToken(getValue('refreshtoken'));
        }
    
        // Retrieves expires timestamp from database
        if (getValue('expires_in')) {
            $connection->setTokenExpires(getValue('expires_in'));
        }

        // Set callback to save newly generated tokens
        $connection->setTokenUpdateCallback('tokenUpdateCallback');
        // Make the client connect and exxchange tokens
        try {
            $connection->connect();
        } catch (\Exception $e) {
            throw new Exception('Could not connect to Exact: ' . $e->getMessage());
        }
    
        return $connection;
    }

Authorize function

 public function authorize()
    {
        $connection = new Connection();
        $connection->setRedirectUrl(env("EXACT_CALLBACK_URL"));
        $connection->setExactClientId(env("EXACT_CLIENT_ID"));
        $connection->setExactClientSecret(env("EXACT_CLIENT_SECRET"));
        $connection->redirectForAuthorization();
    }

Authorization token check

 if (isset($_GET['code']) && is_null(getValue('authorizationcode'))) {
            setValue('authorizationcode', $_GET['code']);
            $request->session()->put('authorizationcode',$_GET['code']);
        }
        // If we do not have a authorization code, authorize first to setup tokens
        if (!$request->session()->has('authorizationcode')) {
            $exact = new \App\Http\Controllers\ExactOnlineController();
            $exact->authorize();
        }

and the dump of my connection class:

-baseUrl: "https://start.exactonline.nl"
  -apiUrl: "/api/v1"
  -authUrl: "/api/oauth2/auth"
  -tokenUrl: "/api/oauth2/token"
  -exactClientId: "SECRET"
  -exactClientSecret: "SECRET"
  -authorizationCode: "ha1W!IAAAACBP-J0pQ9fA8Z5-KcHLG9KVPpfKFLNQlEMIqouZyAJbwQEAAAE6oJu5PSd87_F33JdNhCT_MaWKfYbZ2nHBFjy-oPOO-t9lllqIw7a-tW_627_B36xfwUkFz5dHHsSUWC_Ox9oAL6c7geEe0n5-UVi ▶"
  -accessToken: null
  -tokenExpires: null
  -refreshToken: null
  -redirectUrl: "http://127.0.0.1:8000/home"
  -division: null
  -client: null
  -tokenUpdateCallback: "tokenUpdateCallback"
  -acquireAccessTokenLockCallback: null
  -acquireAccessTokenUnlockCallback: null
  #middleWares: []
  +nextUrl: null
  #dailyLimit: null
  #dailyLimitRemaining: null
  #dailyLimitReset: null
  #minutelyLimit: null
  #minutelyLimitRemaining: null

As u can see the authorization code is filled in connect function but the access token keeps empty and is throwing this error.

Could not connect to Exact: Could not acquire or refresh tokens [http 400]

Somebody knows a solution?

meeshoogendoorn avatar Aug 19 '21 19:08 meeshoogendoorn

I would guess you did not save the refresh token. Every successful request will return a refresh token you need in your next request (in your case something you would like to process setTokenUpdateCallback). If you get this error chances are you performed a request with the wrong (none or old) refresh token.

See also this part in the readme.md:

// Save the new tokens for next connections
setValue('accesstoken', serialize($connection->getAccessToken()));
setValue('refreshtoken', $connection->getRefreshToken());

Another possibility (but don't think this applies in your case) is you run into a race conditions kinda scenario where multiple requests are fired at the same time resulting in the first request being valid (correct refresh token) but the second fails (as it uses the old token while it should use the token returned in the first request).

It might help to read the Exact Online documentation regarding their oAuth implementation: https://support.exactonline.com/community/s/knowledge-base#All-All-DNO-Content-oauth-eol-oauth-devstep1

remkobrenters avatar Aug 20 '21 06:08 remkobrenters

@remkobrenters Thanks for your reaction, I was setting the accessToken and RefreshToken in the setTokenUpdateCallback function. I changed it regarding the readme but still no success. I created a new laravel instance locally with a fresh project but still no success. Do u know anything else what could be the reason for this problem?

meeshoogendoorn avatar Aug 24 '21 16:08 meeshoogendoorn

I am having the exact same problem. I receive my authorization code and then follow up to make a request for retrieving the access tokens and get the Could not connect to Exact: Could not acquire or refresh tokens [http 400]

Before with this exact same code everything worked perfectly...

TjardoOrtan avatar Aug 26 '21 08:08 TjardoOrtan

Same here, stopped working since a week ago. Still haven't been able to find any solution

meeshoogendoorn avatar Aug 26 '21 08:08 meeshoogendoorn

Same here, since I updated to version v3.31.0. v2.26.0 was working fine. I think it has something to do with this release: https://github.com/picqer/exact-php-client/releases/tag/v3.29.0.

Running this from Symfony it throws this error while trying to connect:

Attempted to load class "Message" from namespace "GuzzleHttp\Psr7".
Did you forget a "use" statement for "Symfony\Component\Mime\Message"?

Trying again then comes up with Could not acquire or refresh tokens [http 400].

Not sure if I'm in conflict with my Symfony code now.

timo002 avatar Aug 30 '21 21:08 timo002

@timo002 : I doubt that's the case. We started getting this particular error more and more often in different implementations (some of which use DB storage for the tokens and some using the example).

Our current workaround is removing all oauth tokens and reauthenticating. However, this is starting to get a bit frustrating as for 1 of the implementations, we have to do this like 10 times per week now.

sebastianberm avatar Sep 03 '21 08:09 sebastianberm

One additional thought: On our production platforms (using different frameworks and implementations through the years) we sometimes get false 400 errors on nightly requests. If we retry the request with the 'old' token it works. We suspect server maintenance on the Exact side to sometime return incorrect statuscodes. Not sure if that is your case.

Beside this: yes, sometimes working connections break for no obvious reason. Same calls, same amount of data, no code changes whatsoever. It run fine for weeks or months and than breaks. We reinstate the connection and all runs well again.

remkobrenters avatar Sep 06 '21 05:09 remkobrenters

@timo002 : I doubt that's the case. We started getting this particular error more and more often in different implementations (some of which use DB storage for the tokens and some using the example).

Well, v3.28 has no issues, > 3.28 is giving issues.

Attempted to load class "Message" from namespace "GuzzleHttp\Psr7".
  Did you forget a "use" statement for "Symfony\Component\Mime\Message"?

timo002 avatar Sep 13 '21 18:09 timo002

I think there are two separate questions mixing here. The potential issue with the use statement (have not seen it myself yet) is separate from the 400 error responses returned by Exact Online. @timo002 will you be so kind to create a new issue for this specific issue so it can be investigated further and, if needed, fixed with a PR linked to the issue?

remkobrenters avatar Sep 14 '21 05:09 remkobrenters

@remkobrenters I think you are right, So I created a new issue. Now on version 3.28 I'm also getting this issue, however it returns a 401 code. Could not acquire or refresh tokens [http 401] Not once, but continously.

I authorized again by deleting the json file with the refreshtoken. Problem gone for now.

timo002 avatar Sep 15 '21 20:09 timo002

@meeshoogendoorn @sebastianberm @TjardoOrtan Did any of you found out what the problem was? Started getting the exact same thing 2 days ago. First few calls work perfectly but after a short period it stops working and I get the Exception Could not connect to Exact: Could not acquire or refresh tokens [http 400] I'm wondering if it has anything to do with the token expiry. 🤷‍♂️

samzzi avatar Sep 15 '21 21:09 samzzi

I did what @sebastianberm said:

Our current workaround is removing all oauth tokens and reauthenticating. However, this is starting to get a bit frustrating as for 1 of the implementations, we have to do this like 10 times per week now.

After that everything worked as before again. Did this only one time and it is still working as usual. No clue what the exact problem was...

TjardoOrtan avatar Sep 16 '21 10:09 TjardoOrtan

I did what @sebastianberm said:

Our current workaround is removing all oauth tokens and reauthenticating. However, this is starting to get a bit frustrating as for 1 of the implementations, we have to do this like 10 times per week now.

After that everything worked as before again. Did this only one time and it is still working as usual. No clue what the exact problem was...

hmm, it works for a few minutes with me and then it doesn't anymore. If I clear it again it works again for a few minutes. I feel like the refresh token isn't used or something. Tx for the reply though 🤷

samzzi avatar Sep 16 '21 10:09 samzzi

I did what @sebastianberm said:

Our current workaround is removing all oauth tokens and reauthenticating. However, this is starting to get a bit frustrating as for 1 of the implementations, we have to do this like 10 times per week now.

After that everything worked as before again. Did this only one time and it is still working as usual. No clue what the exact problem was...

hmm, it works for a few minutes with me and then it doesn't anymore. If I clear it again it works again for a few minutes. I feel like the refresh token isn't used or something. Tx for the reply though 🤷

I don't know if that's the actual issue. Because we've not been able to replicate it. We have multiple instances of this system running. Instance 1: Has never broken down (ancient version of the package) Instance 2: Only broke down once so far (up to date version of the package), although this also killed our instance 1 (they share the tokens) Instance 3: Breaks down multiple times per week, and after resetting, it will work for a month or so (modified ancient version of the package) Instance 4: Breaks down 1 or 2 times per month (ancient version of the package) (German Exact)

We're in the process of introducing an instance 5 and 6 for different clients at the moment, but I really do hope this will be more stable soon.

Note; my personal fear is that's it's not something the package is doing, but it's a chance sometimes triggered in Exact Online.

sebastianberm avatar Sep 17 '21 07:09 sebastianberm

I agree with your personal fear. We have numerous (think 20 - 30 at least) instances. Some on older platforms (and older versions of the package) and some on newer version. Complete token disruptions happen but very rare. Most of the times it is a 400 error which work perfectly on the next call (without renewing the oath consent).

I am confident it is not the package but a mix of sometimes broken responses from the Exact API. Also I want to add that the busier an application is (more calls to Exact) the more often we see weird issues / complete disconnects.

remkobrenters avatar Sep 17 '21 07:09 remkobrenters

Hi all, we have a pretty heavy used Exact implementation tool running in production in Belgium. I think the main issue with your code is the refresh token requesting.

Your code is like this:

    // Retrieves expires timestamp from database
    if (getValue('expires_in')) {
        $connection->setTokenExpires(getValue('expires_in'));
    }

Change this to so it refreshes 30 seconds earlier:

      // Retrieves expires timestamp from database
      if (getValue('expires_in')) {
          $connection->setTokenExpires(getValue('expires_in') - 30);
      }

The reason for this is that your access token has to be valid till the end of the request. This is because Exact internally uses load balancers and they forward the request with the token to the internal API servers. They can start slightly later with your requests or have some delay processing. So basically even though your token is valid at the beginning at the request it might not be valid at the end of request and Exact API server rejects the token.

We have 0 disconnects over 1000s over companies even with concurrency.

PS if you are using Exact API in a concurrent environment make sure to add a mutex around the refresh process to prevent race conditions

alexjeen avatar Sep 30 '21 08:09 alexjeen

I am having the same issue here, the strange thing is it only stops working after i will try to do a PUT request. When i am performing a GET to the endpoint, my connection is a success but when i try to update a account my whole connection fails with a Could not acquire or refresh tokens..

i have got this for the token updatecallback:

 public static function tokenUpdateCallback(Connection $connection)
{
        Settings::setValue('EXACT_ACCESS_TOKEN', $connection->getAccessToken());
        Settings::setValue('EXACT_REFRESH_TOKEN', $connection->getRefreshToken());
        Settings::setValue('EXACT_EXPIRES_IN', $connection->getTokenExpires() - 30);
}

And this for my aquireLock

 /**
    * Acquire refresh lock to avoid duplicate calls to exact.
    */
   public static function acquireLock(): bool
   {
       /** @var Repository $cache */
       $cache = app()->make(Repository::class);
       $store = $cache->getStore();

       if (!$store instanceof LockProvider) {
           return false;
       }

       self::$lock = $store->lock(self::$lockKey, 30);
       return self::$lock->block(30);
   }

   /**
    * Release lock that was set.
    */
   public static function releaseLock()
   {
       return optional(self::$lock)->release();
   }

and of course i have de setacquireaccestokenLockcallback and unlocallbacks

  $connection->setAcquireAccessTokenLockCallback([Exact::class, 'acquireLock']);
        $connection->setAcquireAccessTokenUnlockCallback([Exact::class, 'releaseLock']);
        // Set callback to save newly generated tokens
        $connection->setTokenUpdateCallback('\Modules\ExactOnline\Entities\Exact::tokenUpdateCallback');

What is it that i am missing that i am getting this strange error.. every time i deleted the Accestokens en refresh tokens its working for like 2min and then when i preform a put call or a other get call and it fails again with the could not acquire or refresh tokens error

Hope some one has any idea :-)

pepijndik avatar Nov 03 '21 12:11 pepijndik

I am having the same issue here, the strange thing is it only stops working after i will try to do a PUT request. When i am performing a GET to the endpoint, my connection is a success but when i try to update a account my whole connection fails with a Could not acquire or refresh tokens..

i have got this for the token updatecallback:

 public static function tokenUpdateCallback(Connection $connection)
{
        Settings::setValue('EXACT_ACCESS_TOKEN', $connection->getAccessToken());
        Settings::setValue('EXACT_REFRESH_TOKEN', $connection->getRefreshToken());
        Settings::setValue('EXACT_EXPIRES_IN', $connection->getTokenExpires() - 30);
}

And this for my aquireLock

 /**
    * Acquire refresh lock to avoid duplicate calls to exact.
    */
   public static function acquireLock(): bool
   {
       /** @var Repository $cache */
       $cache = app()->make(Repository::class);
       $store = $cache->getStore();

       if (!$store instanceof LockProvider) {
           return false;
       }

       self::$lock = $store->lock(self::$lockKey, 30);
       return self::$lock->block(30);
   }

   /**
    * Release lock that was set.
    */
   public static function releaseLock()
   {
       return optional(self::$lock)->release();
   }

and of course i have de setacquireaccestokenLockcallback and unlocallbacks

  $connection->setAcquireAccessTokenLockCallback([Exact::class, 'acquireLock']);
        $connection->setAcquireAccessTokenUnlockCallback([Exact::class, 'releaseLock']);
        // Set callback to save newly generated tokens
        $connection->setTokenUpdateCallback('\Modules\ExactOnline\Entities\Exact::tokenUpdateCallback');

What is it that i am missing that i am getting this strange error.. every time i deleted the Accestokens en refresh tokens its working for like 2min and then when i preform a put call or a other get call and it fails again with the could not acquire or refresh tokens error

Hope some one has any idea :-)

Have founded my issue... I made for every time i wanted to do something a new connection call.. This way my first connection was outdated en the tokens mismatch.. Hope that some else can learn from my misstake😉

pepijndik avatar Nov 04 '21 10:11 pepijndik

I agree with a similar problem. I have a connection that works, but it will expire regularly. I have the idea that it comes because two connections are made at the same time and thereby gets confused the token.

The system I am using 1 server by multiple accounts. A refresh or a click ensures that the connect () script runs again. I suspect it assumes it.

  1. I make a connection, after the login with Exact comes to the Storage.json a token and expiredate and the coupling works.
  2. I think you will automatically (after more than 10 minutes or after the expirationate) a new one when the key is still correct. That also works.

What I've done:

  1. I have written a script that, if the link goes wrong with the Connect (), the Storage.json save as backup with the current token in it and then empty the storage.json. If I then put those backup txt back in Storage.json, it works again.

  2. I thought, it will have to do with the lifetime of the token. That is why I wrote a cronjob that runs a scriptje every 8 minutes to make a new token. But I think it's not.

Suspicions:

I think that if multiple connect () takes place at the same time from the server, that the Connect () goes wrong or a wrong token returns due to a difference in request time.

In the last reply of @pepijndik he writes that it comes because a new connection is made every time. But I don't know how I can ensure that it is not necessary. Can someone show me the way?

finallynl avatar Dec 16 '21 20:12 finallynl

@finallynl use a custom lock to prevent the tokens to update if the token is still valid. And as i already said make sure you dont make for every thing you wanted to do with the exact api a new connection. So if you wanted to update a account and get the account salesinvoices. Create a var which holds the connection and use this connection todo the 2 api calls.

Hope this will help you

pepijndik avatar Dec 17 '21 07:12 pepijndik

Thanks for you reply. I'm going to check it out.

finallynl avatar Dec 17 '21 07:12 finallynl

Anyone got a solution for this? Got the same problem.

notfalsedev avatar Jan 13 '22 19:01 notfalsedev

Not yet really, I'm still struggling with the keeping the connection steady.

We had a system where all kinds of documents were send to Exact realtime, but there are multiple users working in the system so I thought maybe sometimes the tokens get confused, because every call is seperate.

So then I remade the system and now the systems saves a que and then once every hour by a cronjob it's pushed to Exact.

This is only not working also yet, still the connection fails most of the time.

finallynl avatar Jan 24 '22 15:01 finallynl

Not yet really, I'm still struggling with the keeping the connection steady.

We had a system where all kinds of documents were send to Exact realtime, but there are multiple users working in the system so I thought maybe sometimes the tokens get confused, because every call is seperate.

So then I remade the system and now the systems saves a que and then once every hour by a cronjob it's pushed to Exact.

This is only not working also yet, still, the connection fails most of the time.

are you using custom PHP or a laravel as a framework. Because i use laravel Jobs scheduling with custom timeouts en error handling so that when the job fails it waits for 5 till 10 min and then tries again and if that fails the connection is made again.

this is working for me perfectly and i never have a token expire error

pepijndik avatar Jan 24 '22 15:01 pepijndik

Thanks for your reply. I really like the idea to have 'never' a token expire error :-)

It's all custom, quite basic PHP. I just use this example: https://github.com/picqer/exact-php-client/blob/main/example/example.php to get the tokens.

  1. That works for the first time and the tokens are saved in storage.json. Fine, it works.

  2. And then every hour (in dev is 30 minutes now) a cronjob call's the script to push new Clients and Orders to Exact Online. That also works and it stays working when there is no information pushed to Exact Online. Sometimes there is just nothing to push and then the tokens stays in tact and next call is fine...and so on.

  3. But when one time some information is pushed to Exact, that works still, but afterwards the next call is broken and is not automatically fixed with new tokens.

finallynl avatar Jan 24 '22 21:01 finallynl

The issue on my end was using a wrong parameter in the setTokenExpires method while initializing the connection.

Mistake: $connection->setTokenExpires($numberOfSecondsToExpiry);

Fix: $connection->setTokenExpires($unixTimestampOfExpiryTime);

Example: You can use Carbon\Carbon to parse the given timeout to a given timestamp. $connection->setTokenExpires(Carbon::parse(($accessToken->created_at)->addSeconds($accessToken->expires_in))->timestamp)

watitwa avatar Apr 21 '22 12:04 watitwa

https://support.exactonline.com/community/s/knowledge-base#All-All-DNO-ReleaseNote-rn-2101-rn-appcenter-applmts

As of 1 July 2021, Exact Online will impose extra limits on the API. Exact Online currently processes more than 20 Million API requests per day. These changes are necessary to ensure platform stability, API efficiency, and to improve customer experience.

The following limits will take effect on 1 July 2021:

Maximum of 200 token endpoint calls per API key, per user, per day. You must not request new access token more than once every 10 minutes. To learn more, see Get and use access tokens. No more than 10 errors per API key, per user, per company, per endpoint, and per hour. When you exceed this limit, your API key will be temporarily blocked from making further requests. The block will automatically be lifted after one hour and will gradually increase when you continue making these errors. To learn more, see Response codes and error handling. Mandatory filtering on single and bulk endpoints where sync APIs are available.Exact Online has sync APIs available to allow actions on larger sets of data without running into API limits. To learn more, see API types to make your API calls more efficient. The limits above are in addition to the following API limits that are already in effect since 1 January 2021:

New apps will directly be throttled to 5.000 API calls per app, per company, per day. The shaping limit is 60 API calls, per company, per minute. Based on fair use policy, we will take corrective action by throttling your app if the limit has been exceeded excessively. Some examples of excessive actions are; repeated download of unchanged data everyday, log in attempts to inactive accounts, and overloading the token endpoint. See section 7.4 of our terms and conditions.

For more information about our current API limits, see API limits.

To learn about tips and trick to comply with API rate limits see, Integrate efficiently with OData.

hansrossel avatar Jun 22 '22 10:06 hansrossel

Any updates on this? This is an annoying issue.

sietse85 avatar Jul 18 '22 12:07 sietse85

Hi @sietse85 This is a community managed package. I understand that you, like a lot of developers, are having a hard time implementing the API limits imposed by Exact Online. If you have constructive tips for other developers in this community please add them to this issue. You can always contact Exact Online to tell them how you feel of course.

remkobrenters avatar Jul 18 '22 13:07 remkobrenters

@remkobrenters i think i didnt explain myself enough. the limits are no problem and arent the problem in my case. i build my application in laravel to the point that there was zero concurrency and everything works, then still all of a sudden this error gets thrown (almost random it looks). I don't have any clue what is causing it. But it's not my code, so it's either the package, or Exact Online endpoint.

sietse85 avatar Jul 18 '22 13:07 sietse85