libsql-client-ts

libsql-client-ts copied to clipboard

I was switching my database from planetscale to turso but I'm getting the following error

 {
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module '@libsql/linux-x64-gnu'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/index.mjs",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module '@libsql/linux-x64-gnu'",
    "Require stack:",
    "- /var/task/index.js",
    "- /var/runtime/index.mjs",
    "    at _loadUserApp (file:///var/runtime/index.mjs:1061:17)",
    "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1093:21)",
    "    at async start (file:///var/runtime/index.mjs:1256:23)",
    "    at async file:///var/runtime/index.mjs:1262:1"
  ]
}

I use the aws cdk to build and deploy my aws lambda function internally it uses esbuild here is a simple reproduction step

the aws lambda function code /nodejs/index.ts

import { drizzle } from 'drizzle-orm/libsql';
import { createClient } from '@libsql/client';
const client = createClient({ url: 'DATABASE_URL', authToken: 'DATABASE_AUTH_TOKEN' });

const db = drizzle(client);

export const handler = async (event) => {
  console.log('event', event);
}

the aws cdk code to deploy the function:

import {Construct} from 'constructs';
import {NodejsFunction} from 'aws-cdk-lib/aws-lambda-nodejs';
import {Architecture, Runtime} from 'aws-cdk-lib/aws-lambda';

export class SendUserToDynamo extends Construct {
  function: NodejsFunction;
  constructor(scope: Construct, id: string) {
    super(scope, id);

    // lambda function that triggers on aws lambda user created event
    this.function = new NodejsFunction(this, 'sendUserToDynamo', {
      entry: __dirname + '/nodejs/index.ts',
      handler: 'handler',
      architecture: Architecture.X86_64,
      runtime: Runtime.NODEJS_18_X,
    });
  }
}

Everything works when I'm using the PlanetScale driver, so the error must come from libsql and not the CDK build step. I tried using a Dockerfile without TypeScript and ESBuild, and it did work

Similar error happens when deploying to vercel. Any advice?

Update I reverted back to version 0.3.2 and the issue is gone

same here and confirm downgrading to 0.3.2 fixed it

0.3.3 also works

bump

Confirming I'm seeing this too, also switching from Planetscale to Turso. Interestingly the error doesn't happen in local env, only when deployed to Netlify. Running node 16. Pulling @libsql/client down to 0.3.2 from 0.5.3 fixed things for me.

Ran into this too and got it working with 0.3.3.

Having a similar issue, but I'm on Windows 11 and getting:

Cannot find module '@libsql/win32-x64-msvc'

Works with 0.3.3 for now, but no later versions.

I'm experiencing this on an arm lambda: https://github.com/tursodatabase/libsql-js/issues/70

edit: 0.3.3 does fix this.

I ended up using the web/HTTP client to be able to run the latest version:

import { createClient } from '@libsql/client/web';

it does say in the docs that you should use it for vercel functions and the like. the docs could be a bit clearer here. nothing mentions what to use for lambda

Just wanted to chime in that I also ran into this exact same issue deploying to AWS Lambda using Architect and on version 0.6.0 of @libsql/client. Including that it all worked fine locally. (using the Architect's sandbox mode for local dev)

The suggestion to use web worked for me. 👍

The same problem here using versions 0.5.x and 0.6.x with Deno Deploy with NPM comparability layer. I tried to use it with esm.sh but I didn't have any success.

If you want to use the native client that uses the libsql binary under the hood in Lambda, you'll have to build the function in a similar arch/environment or use a Lambda layer, also built under the same conditions (i.e. Docker).

A bundler like esbuild won't be able to include the correct version (i.e. @libsql-linux-arm64-gnu) unless it's itself running on that same hardware.

This worked for me:

import { Stack, type StackProps, CfnOutput } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Architecture, Code, LayerVersion, Runtime } from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';

export class LibsqlStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const libsqlLayer = new LayerVersion(this, 'LibsqlLayer', {
      compatibleArchitectures: [Architecture.ARM_64],
      compatibleRuntimes: [Runtime.NODEJS_20_X],
      code: Code.fromAsset('./layers/libsql/nodejs', {
        bundling: {
          image: Runtime.NODEJS_20_X.bundlingImage,
          environment: {
            npm_config_cache: "/tmp/npm_cache",
            npm_config_update_notifier: "false",
          },
          command: [
            'bash',
            '-xc',
            [
              'cd $(mktemp -d)',
              'cp /asset-input/package* .',
              'npm --prefix . i @libsql/client',
              'cp -r node_modules /asset-output/'
            ].join(' && '),
          ]
        }
      }),
    });

    const fn = new NodejsFunction(this, 'MyFunction', {
      runtime: Runtime.NODEJS_20_X,
      architecture: Architecture.ARM_64,
      entry: './src/index.ts',
      handler: 'handler',
      bundling: {
        minify: true,
        mainFields: ['module', 'main'],
        sourceMap: true,
        format: OutputFormat.ESM,
        externalModules: ['@libsql/client'],
      },
      layers: [libsqlLayer],
    });

    new CfnOutput(this, 'FunctionArn', {
      value: fn.functionArn,
    });
  }
}

then in my function I use @libsql/client as normal:

import { Logger } from "@aws-lambda-powertools/logger";
import { createClient } from "@libsql/client";

const logger = new Logger();
const db = createClient({
  url: "file:/tmp/sqlite.db",
});

export const handler = async () => {

  await db.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)");

  await db.execute({ sql: "INSERT INTO users (name) VALUES (?)", args:  ["Alice"]});

  const rows = await db.execute("SELECT * FROM users");

  logger.info("Rows: ", {rows});

  return {
    statusCode: 200,
    body: JSON.stringify("Hello, World!"),
  };
};

which when invoked prints this:

{
    "level": "INFO",
    "message": "Rows: ",
    "sampling_rate": 0,
    "service": "service_undefined",
    "timestamp": "2024-06-27T18:14:43.388Z",
    "xray_trace_id": "1-667dac12-2624c2867735f80f34586f58",
    "rows": {
        "columns": [
            "id",
            "name"
        ],
        "columnTypes": [
            "INTEGER",
            "TEXT"
        ],
        "rows": [
            [
                1,
                "Alice"
            ]
        ],
        "rowsAffected": 0,
        "lastInsertRowid": null
    }
}

The advantage of using a Lambda layer like that is that you can still continue using esbuild for the function(s) - the only caveat there is to make sure to specify externalModules: ['@libsql/client'], so that the dependency is not included in the bundle - this is because it comes from the Layer.

If you want to use the native client that uses the libsql binary under the hood in Lambda, you'll have to build the function in a similar arch/environment or use a Lambda layer, also built under the same conditions (i.e. Docker).

A bundler like esbuild won't be able to include the correct version (i.e. @libsql-linux-arm64-gnu) unless it's itself running on that same hardware.

This worked for me:

import { Stack, type StackProps, CfnOutput } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Architecture, Code, LayerVersion, Runtime } from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';

export class LibsqlStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const libsqlLayer = new LayerVersion(this, 'LibsqlLayer', {
      compatibleArchitectures: [Architecture.ARM_64],
      compatibleRuntimes: [Runtime.NODEJS_20_X],
      code: Code.fromAsset('./layers/libsql/nodejs', {
        bundling: {
          image: Runtime.NODEJS_20_X.bundlingImage,
          environment: {
            npm_config_cache: "/tmp/npm_cache",
            npm_config_update_notifier: "false",
          },
          command: [
            'bash',
            '-xc',
            [
              'cd $(mktemp -d)',
              'cp /asset-input/package* .',
              'npm --prefix . i @libsql/client',
              'cp -r node_modules /asset-output/'
            ].join(' && '),
          ]
        }
      }),
    });

    const fn = new NodejsFunction(this, 'MyFunction', {
      runtime: Runtime.NODEJS_20_X,
      architecture: Architecture.ARM_64,
      entry: './src/index.ts',
      handler: 'handler',
      bundling: {
        minify: true,
        mainFields: ['module', 'main'],
        sourceMap: true,
        format: OutputFormat.ESM,
        externalModules: ['@libsql/client'],
      },
      layers: [libsqlLayer],
    });

    new CfnOutput(this, 'FunctionArn', {
      value: fn.functionArn,
    });
  }
}

then in my function I use @libsql/client as normal:

import { Logger } from "@aws-lambda-powertools/logger";
import { createClient } from "@libsql/client";

const logger = new Logger();
const db = createClient({
  url: "file:/tmp/sqlite.db",
});

export const handler = async () => {

  await db.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)");

  await db.execute({ sql: "INSERT INTO users (name) VALUES (?)", args:  ["Alice"]});

  const rows = await db.execute("SELECT * FROM users");

  logger.info("Rows: ", {rows});

  return {
    statusCode: 200,
    body: JSON.stringify("Hello, World!"),
  };
};

which when invoked prints this:

{
    "level": "INFO",
    "message": "Rows: ",
    "sampling_rate": 0,
    "service": "service_undefined",
    "timestamp": "2024-06-27T18:14:43.388Z",
    "xray_trace_id": "1-667dac12-2624c2867735f80f34586f58",
    "rows": {
        "columns": [
            "id",
            "name"
        ],
        "columnTypes": [
            "INTEGER",
            "TEXT"
        ],
        "rows": [
            [
                1,
                "Alice"
            ]
        ],
        "rowsAffected": 0,
        "lastInsertRowid": null
    }
}

The advantage of using a Lambda layer like that is that you can still continue using esbuild for the function(s) - the only caveat there is to make sure to specify externalModules: ['@libsql/client'], so that the dependency is not included in the bundle - this is because it comes from the Layer.

Hey @dreamorosi , thanks for this.

I am currently trying to get this to work with SST Ion that uses Pulumi under the hood, and it seems it won't bundle this automatically as CDK does.

How would I go about bundling this manually? Here you can see what Pulumi offers from an AWS lambda layer standpoint: https://www.pulumi.com/registry/packages/aws/api-docs/lambda/layerversion/#constructor-example

As you can see, there is no option to set the command or the bundling configuration. Same with the Lambda itself, as seen here: https://www.pulumi.com/registry/packages/aws/api-docs/lambda/function/#constructor-example

I was able to solve this by adding the @libsql/client as a nodejs dependency which allows SST to do an NPM install of them in the function bundle:

import { usersTable } from "./database";
import { TursoAuthToken, TursoDatabaseUrl } from "./database";

export const api = new sst.aws.Function("Api", {
  url: true,
  link: [usersTable, TursoAuthToken, TursoDatabaseUrl],
  handler: "./packages/functions/api/index.handler",
  architecture: "arm64",   
  nodejs: {
    install: ["@libsql/client"]
  }        
});

I ended up using the web/HTTP client to be able to run the latest version:

import { createClient } from '@libsql/client/web';

this works

I ended up using the web/HTTP client to be able to run the latest version:

import { createClient } from '@libsql/client/web';

this works

This worked for me too! Thanks!

fyi this is still an active issue. @libsql/client/web is also not a feasible workaround as it doesn't support embedded replicas.

Just ran into this. The @libsql/client/web workaround also doesn't seem to work. I'm deploying to Lambda using Serverless Framework. Does anyone have a workaround?

running into same issue here but not for serverless but for electron build in production

target: arm64 dep,

the error only shows in production not in development and is hard to debug

so any help is welcome