Error on proto files with external dependencies on protobuf.js version: 7.2.6

[leomongeg]

protobuf.js version: 7.2.6

I'm using NestJS and the communication between internal modules is through GRPC, all was working fine but after running some updates the proto loader stopped working crashing with the following error:

[Nest] 21941  - 02/21/2024, 12:45:15 AM   ERROR [ClientProxy] ENOENT: no such file or directory, open '/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@my-org/internal-client-package/protos/google/api/annotations.proto'
Error: The invalid .proto definition (file at "/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@my-org/internal-client-package/protos/google/api/annotations.proto" not found)
    at ClientGrpcProxy.loadProto (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/client/client-grpc.js:242:39)
    at ClientGrpcProxy.createClients (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/client/client-grpc.js:210:34)
    at new ClientGrpcProxy (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/client/client-grpc.js:32:33)
    at Function.create (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/client/client-proxy-factory.js:30:24)
    at /Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/module/clients.module.js:13:80
    at Array.map (<anonymous>)
    at Function.register (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/module/clients.module.js:11:48)
    at generateGRpcClient (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@my-org/microservice-utils/src/protos/proto.loader.ts:53:24)
    at Function.register (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@my-org/internal-client-package/internal-client-package.module.js:24:61)
    at Object.<anonymous> (/Users/leonardo/Development/my-org/my-nestjs-api-gateway/src/app.module.ts:18:32)
    at Module._compile (node:internal/modules/cjs/loader:1233:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Function.Module._load (node:internal/modules/cjs/loader:938:12)
    at Module.require (node:internal/modules/cjs/loader:1115:19)
    at require (node:internal/modules/helpers:119:18)

/Users/leonardo/Development/my-org/my-nestjs-api-gateway/node_modules/@nestjs/microservices/client/client-grpc.js:242
            const invalidProtoError = new invalid_proto_definition_exception_1.InvalidProtoDefinitionException(err.path);

It is important to mention, not all protos are presenting issues only proto files with external dependencies (protos with imports and not all imports) are failing, for example, protos with this kind of import directives:

syntax = "proto3";

package MyProtoPackage;

import "google/protobuf/empty.proto"; // 👈 No problem here
import "google/api/annotations.proto"; // ❌ Issue here
import "google/api/field_behavior.proto"; // ❌ Issue here
import "google/api/visibility.proto"; // ❌ Issue here
import "http.response.proto"; // 👈 No problem here internal proto defined in the same directory as this proto
import "protoc-gen-openapiv2/options/annotations.proto"; // ❌ Issue here

option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_swagger) = {

All was working perfectly fine on the version 7.2.5 bug after the update things stopped working, following the changes in the release notes we noticed that this change https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.2.5...protobufjs-v7.2.6#diff-45840cbec0137d0498a3dbcf38eba6a358e2b8bfd9f1eca20abdfc7b386d13b1L101 is responsible for the problem.

I don't know if with some tuning in the proto loader config, this issue can be avoided or some configuration in NestJS, to give you a little bit more context the issue is present in our API Gateway the internal microservices share the protos using internal npm packages, generating a GRPC client Module exposing the service function that the API Gateway maps in RestFul API's.

If I manually change back the position of the if (sync) throw err; after the callback = null all works fine. For now, we are forcing the protobufjs version to 7.2.5 but this is not a long-term solution since we cannot update for security or critical issues.

We appreciate your help in understanding this behavior of if we should use different configurations to help avoid this problem.

Let me know if you need more information or details.

Regards, Leonardo!

[rjanda774]

I have the same problem.