SDK hangs if it encounters too many 403s from the Apify API

[tobice]

Issue description

I have an Actor that tries to access all kinds of data in the Apify account, and for each request it uploads the result to the default Key-value store.

• When I run the Actor with a token that has access to all the user data (all requests succeed), the Actor works fine (run).
• When I run the Actor with a token that has no access at all (all requests fail), the actor gets stuck mid-run, and eventually times out (run). If I replace Apify SDK calls with Apify JS client calls, it also works fine.

More context on Slack.

Code sample

No response

Package version

Visible in the Actor build.

Node.js version

Visible in the Actor build.

Operating system

No response

Actor or run link

https://console.apify.com/actors/KxN6y6vxqedApRveq/source

I have tested this on the next release

No response

Other context

No response

[B4nan]

The client has expo backoff, which can end up waiting for minutes. The SDK has no retrying mechanism, it just calls the client.

If I replace Apify SDK calls with Apify JS client calls, it also works fine.

Where can we see that?

[tobice]

I didn't create a standalone Actor but the integration tests (thanks to which I stumbled across this issue) have one embedded.

[B4nan]

What if you use Actor.apifyClient in there? I am having a hard time believing this would be an SDK problem, could be about different params for the client constructor, if Actor.newClient() works.

[tobice]

Tested it locally and Actor.apifyClient also works.

[stepskop]

I came across the same issue. We have this Actor, that implements the example logic to show how to migrate the "Cache storages" when migrating Actor to limited permissions.

This is the code that we wanted to provide as example in the docs:

const OLD_CACHE_STORE_NAME = 'my-actor-cache';
const NEW_CACHE_STORE_NAME = 'my-actor-cache-updated';
let store;

try {
	// If this succeeds, the Actor is still running with full permissions and we should use
	// the existing store.
	store = await Actor.openKeyValueStore(OLD_CACHE_STORE_NAME);
} catch (e) {
	if (e.statusCode === 403) {
		// If it fails, the Actor is running with limited permissions and we need to create
		// a new store. The platform will remember that the store was created by this Actor
		// and will allow access in all follow-up runs.
		store = await Actor.openKeyValueStore(NEW_CACHE_STORE_NAME); // <--- This hangs if the first openKeyValueStore() ends up with 403
	} else {
		throw e;
	}
}

I understand it can be solved by using new client, but not sure if it is something that we want to "promote".