pipedream

pipedream copied to clipboard

Name of app / service Exact Online

Link to developer documentation https://support.exactonline.com/community/s/knowledge-base#All-All-DNO-Content-dev-getstrtd

Is lack of support preventing you from building workflows, or do you have a workaround? Lack of support for handling OAuth2 authenticated requests for Exact Online is preventing me to use PipeDream and switch over from Integromat.

Are there specific actions, or triggers, you'd like to see for this app? Please let us know here or use the Action and Trigger issue templates to open requests for each! The most important part is to handle the authentication for the request. After that I can use http calls or axios to fetch data, but I need the Authentication token for this.

We'll need to register a Pipedream app on the Exact Online app store.

Questions to answer

1. Exact Online seems to require MFA for logins - even with OAuth. Does this same flow apply to 3rd party OAuth apps?

Every user can register Pipedream url in MyApps in ExactOnline, you just need to give us the full url that will communicate to ExactOnline.

1. First step of every EO integration is registering an app with an app name and redirect url. That app is per user and does not need to be approved by EO. With registering the app, the user gets client id and secret that will use in the further authentication process
2. Next step, from the connecting app authentication request is sent to EO with client id and secret id in the url and redirect url where refresh token is sent in the end (same url entered in step 1),
3. Then there is two step login in EO, user-pass and google authenticator code
4. When this is successful, refresh token is sent back to the redirect url. That token is valid for one year and is used to retrieve access tokens

I hope this is helpful for you. Please feel free to contact me, we have some experience with integrations with EO

@gnikolov84 we've just enabled this integration -- can you confirm if this works for you and if you have any feedback or questions? https://pipedream.com/apps/exact/

Awesome! I successfully connected account with our ExactOnline and tested the /current/Me request. Looks good so far. We will continue today testing rest of our integrations and get back to you

@dannyroosevelt, it seems something is not 100% working correctly yet. We have a worklow executing more or less 25 requests to Exact Online, we run this workflow a couple of times without problems. The next time we run the workflow the Exact Online request results in the error below. After a couple of minutes of waiting everything works again.

Error Request failed with status code 401

DETAILS at null.createError

    at null.settle (/tmp/__pdg__/dist/code/f87597d03ed7e67582e97ae3698a0d408af6ab4d5d30318e5f0d5153df717250/node_modules/.pnpm/[email protected]/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/tmp/__pdg__/dist/code/f87597d03ed7e67582e97ae3698a0d408af6ab4d5d30318e5f0d5153df717250/node_modules/.pnpm/[email protected]/node_modules/axios/lib/adapters/http.js:236:11)
    at IncomingMessage.emit (events.js:412:35)
    at null.endReadableNT (internal/streams/readable.js:1333:12)
    at process.processTicksAndRejections (internal/process/task_queues.js:82:21)

@dannyroosevelt Looks like it happens after 10 minutes when the access token expires and there is some period (few seconds up to few minutes) we get 401 and all of a sudden new token is retrieved and all continues good.

Here are some timestamps when we got 401:

Thu, 21 Jul 2022 12:17:17 GMT Thu, 21 Jul 2022 12:17:28 GMT Thu, 21 Jul 2022 12:17:37 GMT Thu, 21 Jul 2022 12:17:47 GMT 200

Thu, 21 Jul 2022 12:28:58 GMT Thu, 21 Jul 2022 12:29:18 GMT Thu, 21 Jul 2022 12:29:50 GMT Thu, 21 Jul 2022 12:29:59 GMT 200

Thu, 21 Jul 2022 12:30:07 GMT Thu, 21 Jul 2022 12:30:27 GMT Thu, 21 Jul 2022 12:30:34 GMT Thu, 21 Jul 2022 12:30:40 GMT Thu, 21 Jul 2022 12:30:45 GMT Thu, 21 Jul 2022 12:30:50 GMT Thu, 21 Jul 2022 12:31:08 GMT Thu, 21 Jul 2022 12:31:14 GMT Thu, 21 Jul 2022 12:31:19 GMT Thu, 21 Jul 2022 12:31:24 GMT Thu, 21 Jul 2022 12:31:31 GMT 200

@wspies @gnikolov84 Thanks for the reports.

Sounds like it's possible ExactOnline's refresh token process isn't exactly standard. We don't have much flexibility on our end on altering the refresh behavior in our dashboard, but asking if we can have a low level alteration to address this stale token issue.

But in the meantime, I encourage you to try out the $.flow.rerun which is available in custom actions and Node.js code steps.

You can retry the request in the same code step if there's a stale token up to 10 times and with a delay as well.

thttps://pipedream.com/docs/code/nodejs/rerun/#flow-rerun

Could you do us a favor and start a ticket with ExactOnline and include [email protected]?

We're not able to rule out it's an ExactOnline sourced bug.

@dannyroosevelt @ctrlaltdylan I remember when we were developing Exact integrations internally in our software, we had the same problem, so we subtract 30 second from the token validity and instead of requesting a new access token every 10 minutes, we are doing that every 9.5 minutes and it works good. Exact allows earlier requests up to 30 seconds https://support.exactonline.com/community/s/knowledge-base#All-All-DNO-Content-oauth-eol-oauth-devstep3 Can we please try if this solves the whole problem?

Also, not a problem for me to report a problem to Exact, but let's point Exact where their problem is. Do you keep some log data that you can send me of access token requests? For example some log data when we hit this issue: Success access token response from Exact around Thu, 21 Jul 2022 12:28:58 GMT +/- 2 minutes 401 Thu, 21 Jul 2022 12:28:58 GMT 401 Thu, 21 Jul 2022 12:29:18 GMT 401 Thu, 21 Jul 2022 12:29:50 GMT 401 Thu, 21 Jul 2022 12:29:59 GMT Success access token response from Exact around Thu, 21 Jul 2022 12:30

Hi @gnikolov84 ,

Unfortunately this is not normal API refresh token behavior. I don't see an option to ignore the refresh token expiry date and instead subtract an arbitrary time.

We may not have this level of customization for refresh operations.

Since ExactOnline's refresh token expiration date isn't accurate makes me believe this is bug report worthy for them.

I'll work on getting the logs we can, but could you go ahead and start a ticket with them first and I can share logs directly in that thread? You can include us by adding [email protected] to the thread.

The description you just gave should be enough to create reproduction steps, they should also have server side logs to show these 401's.

@ctrlaltdylan I created a support ticket at ExactOnline explaining the situation. It's a support portal they have, not a mail communication, so I couldn't include you in the communication, but I'll keep you up to date as soon as we get some response.

Can you please share some logs data? I think it will be valuable to add that to the ticket and will speed up solving this

@ctrlaltdylan @dannyroosevelt Can you please investigate this further? We got response from Exact on a 401 from today. In their log files they see a token request at 14:15 and a later api bulk request at 14:26, without requesting a new token meanwhile, knowing that the token is valid for 10 minutes and expires at 14:25. It will be really really helping if you can share some logs from your side... I've already asked few times before... will be very helpful.

Exact integration is our top 1 prio atm and feels like PD and EO is just tossing the ball on this

@gnikolov84 I apologize for the delay getting this fixed for you. We were indeed failing to refresh tokens within the 10 min expiry. We've pushed out a fix to handle this for the Exact integration, and that code is deploying now. Can you let your workflows run over the next ~24 hours and confirm you're no longer seeing 401s from Exact when making requests from Pipedream?

@dylburger , we are still seeing the 401s from Exact. It does look like something has changed though. Before, we got the 401, waited a couple of minutes and ran the workflow again and everything was working ok. Now, we really need to reconnect the Exact Online account in Pipedream (fill in a userid/password and token) every time the token has expired.

@wspies Thanks for the response. I'll get our team to take a look at this. Note that it may take a few days for us to prioritize it, but we'll get back to you ASAP.

@dylburger, Thanks for the update. If you need any testing, I'm more then happy to help.

@dylburger any update on this? We would really like to get our first workflows up and running using Exact Online but this issue is blocking us for now.

No update right now, but it’s on the team’s backlog to take a look at. We’ll update you as soon as possible.

@dylburger, are you able to give us a rough timeline for fixing this issue, will it be weeks/months/..?

Our workflows are ready to go but this issue is holding us back to go into production.

We’ve prioritized it with the team and I expect it will be solved within weeks and not months, but I’ll let you know as soon as we have an update.

Hi @wspies it looks like we're getting rate limited by Exact now that we are attempting to refresh access tokens every 10 minutes. Here is the 400 response we're seeing when making the Refresh Token Request:

{
  "error": "access_denied",
  "error_description": "Rate limit exceeded: access_token not expired"
}

Are you able to loop our team in with the Exact team?

@dannyroosevelt, Exact is quite strict regarding API limits. An access token is valid for 10 minutes and you can only request a new access token after 9,5 minutes. If you do it before you get the response you mentioned.

https://support.exactonline.com/community/s/knowledge-base#All-All-DNO-Simulation-gen-apilimits

You must not request new access token more than once every 10 minutes. You can only request for a new access token after 570 seconds from the time you successfully received the previous access token.

Okay, thanks for the info. That is definitely non-standard, and I haven't run into such a strict and aggressive refresh strategy before (we've integrated hundreds of other OAuth apps).

I have this on the team's backlog to implement custom support and I'll let you know as soon as I have an update to share.

We pushed a potential fix earlier today. Will leave this open for a few days in case the issue is still not resolved for some users.

This should be completed! Please let us know if you see any issues