Cannot load Elastic API file with available versions

[kalinchernev]

Hi, and thanks so much for this time-saver module!

I have a setup in which I use graphql-compose-elasticsearch within an AWS Lambda handler.

When I run it locally with the serverless framework, all works well, but when I deploy the function, I start to get error messages.

The first one was about a wrong parameter passed to the function creating the search() resolver. It was strange, because I used 1:1 the examples from the documentation.

When I deployed a non-minified version of the code, howerver, I received another error:

{
  "errorMessage":"Cannot load Elastic API file with avaliable versions from /node_modules/elasticsearch/src/lib/apis/index.js",
  "errorType":"Error",
  "stackTrace":[
    "Function.loadApiListFile (/var/task/src/api/graphql.js:164571:13)",
    "Function.findApiVersionFile (/var/task/src/api/graphql.js:164581:42)",
    "new ElasticApiParser (/var/task/src/api/graphql.js:164541:121)",
    "createSearchResolver (/var/task/src/api/graphql.js:170075:18)",
    "composeWithElastic (/var/task/src/api/graphql.js:165022:84)",
    "createHandler (/var/task/src/api/graphql.js:262504:111)",
    "<anonymous>"
  ]
}

I feel I'm getting closer to the problem, because I found #6 where there was a discussion about getting the versions from the Elasticsearch client, rather than the file system.

In a lambda function, I can't rely on the file system or globals (like in containers), because the contained environment the functions live should be stateless.

Is there a specific reason the fix in #6 didn't use the suggestion?

const es = require('elasticsearch');
console.log(es.Client.apis);

Maybe the information format which is expected is different?

[kalinchernev]

I spent some time debugging and found out that in https://github.com/graphql-compose/graphql-compose-elasticsearch/blob/master/src/ElasticApiParser.js

this.parsedSource = ElasticApiParser.parseSource(source);

is not the same as:

this.parsedSource = elasticsearch.Client.apis[this.apiVersion];

That's although structure seems same on first look. I haven't gone too deep in the details yet, but I guess the doxygen documentation being currently parsed also plays a role.

[kalinchernev]

As a continuation of my journey, sharing my findings so far, the issues I've been experiencing were coming from several factors.

1. The parser looking for a missing physical file.

A temporary workaround, I added the needed file from the elasticsearch apis index and patched the createSearchResolver and createFindByIdResolver to make use of opts.apiVersion, as opts is going around consistently and it felt as a the smallest update I could make in order to keep the current way of composeWithElastic and the parser.

To be honest, it was not hard to prove, but this part of the class finding the path to the file doesn't seem to work as it's supposed to be. Again, not 100% sure, but I think the path.resolve() has to be called on the result of whether the option is set or the class has found one.

2. Transpilation issues

When using webpack (via this plugin) when the code is bundled and minified, this part of code fails consistently:

  if (!sourceTC || sourceTC.constructor.name !== 'TypeComposer') {
    throw new Error(
      'Second arg for Resolver search() should be instance of TypeComposer.'
    );
  }

So far, it seems that the variable "drops" the reference to its parent class. It's a case which AWS SDK solves by:

AWS.setSDKInstance(AWS_SDK);

Where the AWS_SDK is the instance which should be correctly re-bound to parent class.

To solve this, I stopped minifying the code ...

3. Types helpers

When the code has been finally patched, unminified and pushed to the AWS Lambda service, there is still a problem with type compositions which I still can't figure out, for instance:

Using the getFindByIdOutputTC:

The mapping from Elasticsearch has been successfully discovered.

Whereas using the getSearchOutputTC:

It defaults to JSON, which is not as useful/granular as the first one.

Fact is, locally both cases are working well and as expected, discovering the mapping.

Question: What is the most effective workflow to work with the repository and test changes without patching npm code? Maybe part of the issues I'm experiencing is the fact modules are published in .mjs which needs/or does not need further transpilation?

I'm willing to spend time on making corrections in opts and, maybe, removing some repetitions, unused code, etc., but I'm not sure what would be the best way to work with repo in order to make changes accessible and more convenient?

[floydnoel]

I'm currently getting the Error: Cannot load Elastic API file with avaliable versions from /node_modules/elasticsearch/src/lib/apis/index.js error message. Weird part is the query loads and works fine, then the error page jumps in for some reason.

Edit: I've found the problem in my case is when using Next.js for universal rendering; the fs module is not available on the client. I'm trying to work through the code to use import statements instead of reading files with the fs module directly.

[thealjey-eib]

@kalinchernev @floydnoel have you guys figured out how to overcome this? it seems really strange that readFileSync is used to load a javascript file as a string

[floydnoel]

@thealjey-eib I couldn't get the issue resolved and moved on to another solution.