threads.js icon indicating copy to clipboard operation
threads.js copied to clipboard

pool.terminate() Crashing Entire Application

Open Camuvingian opened this issue 2 years ago • 0 comments

I have the following code to spin up a workerpool.

const pool = Pool(
	() => spawn(new Worker('../workers/createCubeMapFaceWorker')),
	this.#cubeMapWorkerCount > Constants.FacePositions.length // Only makes sense to spin-up 6 - anymore and they are idle.
		? Constants.FacePositions.length
		: this.#cubeMapWorkerCount
);
logger?.info(`${this.#cubeMapWorkerCount} workers spun-up successfully`);

let workerId = 1;
const promises = Constants.FacePositions.map(async (face: 'nx' | 'ny' | 'nz' | 'px' | 'py' | 'pz') => {
	return pool.queue(async ({ createCubeMapFaceWorker }) => {
		return await createCubeMapFaceWorker(
			path,
			userDetails.organizationDetails,
			requestDto.tourId,
			requestDto.cameraId,
			requestDto.imageVersionName,
			requestDto.resolutions,
			requestDto.interpolation,
			requestDto.maxWidth,
			requestDto.sharpenLevel,
			requestDto.compression,
			face,
			this.#azureStorageOptions,
			workerId++
		);
	});
});

try {
	await Promise
		.all(promises)
		.then(async (results: CubeMapResult[]) => {

			let insertUpdateResult = await this.inserUpdateTourImageAudit(results.flatMap(r => r), requestDto);
			if (insertUpdateResult.succeeded == null || !insertUpdateResult.succeeded || insertUpdateResult.errors.length > 0) {

				responseDto.errors = insertUpdateResult.errors;
				responseDto.succeeded = false;

				logger?.warning(`Upload to toursImageAudit collection failed (tourId = ${requestDto.tourId})`);
			}

			this.checkAndConstructErrors(results, responseDto);

			if (responseDto.errors.length > 0) {

				responseDto.succeeded = false;
				httpStatusCode = HttpStatusCode.BadRequest;

				throw new Error('Cloud persistance and/or database-arbiter failure');
			}
			responseDto.succeeded = true;
		})
		.catch((exception) => {
			var exceptionDetails = Utils.getExceptionMessage(exception);
			let message = `${exceptionDetails.message}\n${exceptionDetails.stack}`;

			responseDto.errors.push(new RequestError(
				HttpStatusCode[HttpStatusCode.BadRequest],
				message
			));
			responseDto.succeeded = false;
			httpStatusCode = HttpStatusCode.BadRequest;

			logger?.error(message);
		});
	logger?.info(`Cubemap processing completed on ${this.#cubeMapWorkerCount} workers, terminating...`);

	await pool.completed(true);
	await pool.terminate();

	logger?.info(`${this.#cubeMapWorkerCount} workers completed successfully`);
}
finally { 
	const dir = getImageDir(requestDto.tourId, requestDto.cameraId);
	if (fs.existsSync(dir)) {
		logger?.info(`Attempting to remove temporary local storage...`);
		fs.rmSync(dir, {
			recursive: true, 
			force: true
		});
		logger?.info(`Temporary local storage directory \"${dir}\" removed successfully`);
	}
}

This is running great, but the entire program crashes on await pool.terminate();. This does not happen if I use await pool.completed(); but then the app just hangs.

I cannot seem to debug the cause of this and there is an idential issue here (https://github.com/andywer/threads.js/issues/247). Any help here would be appreciated as the code works perfectly apart from this and I might be doing something silly.

Camuvingian avatar Aug 11 '22 13:08 Camuvingian