mindconnect-nodejs icon indicating copy to clipboard operation
mindconnect-nodejs copied to clipboard

Handling API limit

Open viktorsebok opened this issue 2 years ago • 5 comments

All Mindsphere API call (e.g. Mindconnect API) can answer with an error "API rate limit exceeded"... This special error also gives an info about a relative time (Retry-After) which indicates that how long in 'seconds' have to wait before making a new request.

My opinion is that, the mindconnect-nodejs should handle this error, and automatically resend the requested message when the defined time is elapsed. Probably the best place is all implementation of TokenRotation.HttpAction interface function: https://github.com/mindsphere/mindconnect-nodejs/blob/d4470fbadbca8c92fd3a7ed7bc1c2068411f5919/src/api/mindconnect-base.ts#L51

viktorsebok avatar Feb 08 '23 15:02 viktorsebok

The library provides the

export const retry = async (n: number, func: Function, timoutinMilliseconds: number = 300, logFunction?: Function) 

which can be used to repeat the calls. Now that mindsphere provided retry-after (which is a relatively new feature) , maybe we can add a new function which can wrap such calls and use the retry-after suggestion, instead of preconfigured timeout.

I don't want to have implicit error handling for whole framework, because the API behavior has sometimes subtle differences from API to API - but a better retry method would be worth looking at.

sn0wcat avatar Feb 08 '23 16:02 sn0wcat

Example for using the retry function:

 const assetMgmt = sdk.GetAssetManagementClient();
 const asset = (await retry(options.retry, () =>
   assetMgmt.GetAsset(options.assetid)
)) as AssetManagementModels.AssetResourceWithHierarchyPath;

sn0wcat avatar Feb 08 '23 16:02 sn0wcat

Sounds promising, but you should consider some special SDK functions, e.g.: https://github.com/mindsphere/mindconnect-nodejs/blob/ece7151e81e9550102f659d8e877f4536321b9fe/src/api/mindconnect-agent.ts#L218

In case of MindConnectAgent.PutDataMapping, the number of API calls is depending on the input argument. For instance the input argument has 100 elements, which causes 100 API calls, but the API exceed the limit at 50th element. We can't handle this situation outside of the SDK function with a "retry" mechanism.

viktorsebok avatar Feb 09 '23 08:02 viktorsebok

I never understood why there is no bulk method for creation of data mappings.

sn0wcat avatar Feb 09 '23 11:02 sn0wcat

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jun 18 '23 07:06 stale[bot]