Unable to use @shopify/shopify-api on lambda aws function

[renanbronchart]

Issue summary

Write a short description of the issue here ↓

I tried to use @shopify/shopify-api on lambda aws. But when my lambda function I have an error with this package and napi-v6-linux-glibc-x64/node_sqlite3.node.

On local in my macOs, with nodev14, the function works. But when it is executed on lambda, it doesn't work.

Expected behavior

@shopify/shopify-api should work and I don't want to have this error.

Actual behavior

I receive this error on lambda :

{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module '/var/task/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node'\nRequire stack:\n- /var/task/node_modules/sqlite3/lib/sqlite3-binding.js\n- /var/task/node_modules/sqlite3/lib/sqlite3.js\n- /var/task/node_modules/@shopify/shopify-api/dist/auth/session/storage/sqlite.js\n- /var/task/node_modules/@shopify/shopify-api/dist/context.js\n- /var/task/node_modules/@shopify/shopify-api/dist/index.js\n- /var/task/utils.js\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module '/var/task/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node'",
        "Require stack:",
        "- /var/task/node_modules/sqlite3/lib/sqlite3-binding.js",
        "- /var/task/node_modules/sqlite3/lib/sqlite3.js",
        "- /var/task/node_modules/@shopify/shopify-api/dist/auth/session/storage/sqlite.js",
        "- /var/task/node_modules/@shopify/shopify-api/dist/context.js",
        "- /var/task/node_modules/@shopify/shopify-api/dist/index.js",
        "- /var/task/utils.js",
        "- /var/task/index.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at _loadUserApp (/var/runtime/UserFunction.js:202:13)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:242:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
        "    at Module.load (internal/modules/cjs/loader.js:950:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
        "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)",
        "    at internal/main/run_main_module.js:17:47"
    ]
}

Tip: include an error message (in a <details></details> tag) if your issue is related to an error

Steps to reproduce the problem

• Import Shopify api on lambda function js (Node v14)
• Use it on lambda function and execute it

Here the beginning of my file :

const Shopify = require('@shopify/shopify-api').default;

… 

  const client = new Shopify.Clients.Graphql(
     process.env.SHOP, // shop
    process.env.ACCESS_TOKEN, // access token
  );

[shopsideau]

Seeing the same error using the latest version of this package (3.1.3), though it worked fine on the older version I had previously (2.1.0).

[viraths]

Having a similar issue on aws lambda with shopify and sqllite3

Version: @shopify/shopify-api": "^3.1.3

{ "errorType": "Error", "errorMessage": "/var/package.jsondoes not exist", "stack": [ "Error: /var/package.jsondoes not exist", " at Object.exports.find (/var/task/index.js:70410:15)", " at node_modules/sqlite3/lib/sqlite3-binding.js (/var/task/index.js:70683:31)", " at __require (/var/task/index.js:9:50)", " at node_modules/sqlite3/lib/sqlite3.js (/var/task/index.js:70733:19)", " at __require (/var/task/index.js:9:50)", " at node_modules/@shopify/shopify-api/dist/auth/session/storage/sqlite.js (/var/task/index.js:71055:45)", " at __require (/var/task/index.js:9:50)", " at node_modules/@shopify/shopify-api/dist/context.js (/var/task/index.js:71301:20)", " at __require (/var/task/index.js:9:50)", " at node_modules/@shopify/shopify-api/dist/index.js (/var/task/index.js:145371:21)" ] }

[ty-dev]

Same issue here. Is there a previous 3.x.x version that worked?

[renanbronchart]

I've found an ugly fix, but it works.

• Download the archive napi-v6-linux-glibc-x64/node_sqlite3.node at the address: https://github.com/TryGhost/node-sqlite3/releases
• Put the napi-v6-linux-glibc-x64 directory in src directory of your lambda function.
• Create a script called scripts/afterInstall.js, with this following code:

var fs = require('fs-extra')

fs.copy('napi-v6-linux-glibc-x64', 'node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64', function (err) {
  if (err) throw err
  console.log('Successfully renamed - AKA moved!')
})

• On your lambda package.json, add this script:

"scripts": {
    "postinstall": "node scripts/afterInstall.js"
  },

• amplify push

And it works 🎉 I hope ;)

[ty-dev]

Ended up just reverting to hitting REST endpoint directly until this gets resolved.

[mirkocesaro]

Same problem here.

Thank you @renanbronchart I hope they will fix the SQLite requirement if is not used.

[thibautvdu]

Same here, using firebase functions Machine : mac os m1

[iosart]

Found an easy temporary workaround - essentially prevent node from trying to load sqlite3 and choking. Put this before you require @shopify/shopify-api

var Module = require('module');
var originalRequire = Module.prototype.require;
Module.prototype.require = function() {
  if (arguments[0] === 'sqlite3') {
    return {}
  } else {
    return originalRequire.apply(this, arguments);
  }
};

[Wilddata]

This is crazy, the latest version I can use with my Serverless app is 2.1.0, but the current version is 5.0.1 T_T

Any way we can ignore sqlite3 using import?

import Shopify from '@shopify/shopify-api';

[iosart]

Try something like this:

import Module from 'module';
import { createRequire } from 'node:module';

var originalRequire = Module.prototype.require;
Module.prototype.require = function() {
  if (arguments[0] === 'sqlite3') {
    console.log('Returning fake sqlite3!!!!')
    return {}
  } else {
    return originalRequire.apply(this, arguments);
  }
};

const require = createRequire(import.meta.url);
let Shopify = require('@shopify/shopify-api')

[IzaacBarratt]

Also having this problem, I can see the workarounds but this seems like a pretty significant bug

[deldrid1]

+1 for this issue - I'm trying to use with webpack (for Lambda deployment) and running into lots of fun and unexpected issues related to this.

FYI - looking at things a little closer, it looks like the current round of PRs and https://github.com/Shopify/shopify-api-node/tree/shopify_api_next branch will fix a lot of problems for me this issue is likely already addressed on that branch.

@mkevinosullivan or @paulomarg - any timelines for when this branch is ready to be tagged @next on npm and/or ready to start playing with?

I've got things working locally with full offline + online OAuth session support in a TypeScript, Webpack 5 bundled project using Express + @vendia/serverless-express backed by custom DynamoDB Session Storage. Things fall apart when I move to AWS Lambda deployment due to the sqlite3 dependency. I could probably get across the finish line with some of the hacks mentioned above + some webpack config magic, but I like the direction the next branch is heading a lot better :)

[future-media]

I just patched context.js in sqlite3 to use the /tmp directory instead of /var/runtime. What fun!

[sajadghawami]

what a nasty bug... even with @renanbronchart 's fix i am still getting the same error. I ve also tried @iosart fix, but i keep getting:

web:dev: error - TypeError: sqlite3_1.default.Database is not a constructor
web:dev:     at new SQLiteSessionStorage (/XXX/node_modules/@shopify/shopify-api/dist/auth/session/storage/sqlite.js:16:19)
web:dev:     at Object.initialize (/XXX/node_modules/@shopify/shopify-api/dist/context.js:57:76)
web:dev:     at eval (webpack-internal:///(api)/./src/models/vendor/resolver.ts:373:47)

:(

[bernier]

I just tried 6.0.1, and I'm still getting the same issue.

In my Shopify app, available from the Partner dashboard, insights tab > API health I'm getting: "Metafield.valueType has been removed. See Metafield.type for type information. - UPDATE DEADLINE: JAN 1, 2023" I'm still using 2.0.0 due to this issue and just changed the API Version to 2022-04. I'll probably need to find another way to make this API call since I cannot upgrade @shopify/shopify-api. This stupid use of SQLite becomes a critical blocker.

[paulomarg]

Hi folks! Sorry for the delay in responding to this issue, but we have been listening to the points raised here and trying to solve this problem.

As mentioned, none of the database dependencies are now coming directly from this package so that apps have more control over the way they store their sessions. Updating to v6+ will completely remove sqlite as a dependency so you should be able to install the library.

We're still supporting the same options as before with the packages in https://github.com/Shopify/shopify-app-js, but you'll have to import them individually (or create your own) and call them when you need to store / load sessions.

Since the original problem doesn't apply any more, I'm closing this issue. Please do open new issues if you have any problems running the new version!