Allow knex to infer config from a knex instance (when using the --knexpath flag)

[CodyEakins]

Environment

Knex version: 0.19.3 Database + version: Postgres 11 OS: Linux

Bug / Documentation Error

This bug is related to a documentation issue: knex/knex#5884

Expected Behavior:

A developer should be able to the --knexpath flag to pass in a pre-configured knex instance to the CLI for use in operations like migrate:latest.

Test Code & Error Message:

If I have a file that exports a valid knex instance, for example:

// Located in './services/knex.js'

const knex = require('knex');

const config = require('./config');

module.exports = knex({
  client: 'pg',
  connection: {
    host: config.postgres.host,
    port: config.postgres.port,
    database: config.postgres.database,
    user: config.postgres.user,
    password: config.postgres.password,
  },
  // `migrationsDirectory:` should use the default path './migrations' without being specified
});

I should be able to use this instance to run my latest database migrations:

knex migrate:latest --knexpath ./services/knex.js

Note: I purposely do not create a default knexfile.js file or pass one in using the --knexfile flag, because the CLI should infer all the configuration data it needs from the passed knex instance.

However, when running this command, I encounter the following error:

Error: No default configuration file [...] found and no commandline connection parameters passed

This means that knex still expects a configuration file (or command-line parameters) despite already having a valid knex instance...

---

If I have to pass in the configuration "a second time", then what is the point of supporting the --knexpath flag anyway?

A developer should have options besides using a knexfile.js file to give configuration to knex.

[kibertoad]

@CodyEakins Why not just use Knex API for running migrations then? https://knexjs.org/#Migrations-API (probably could be documented better) Personally I find knex CLI cumbersome and much prefer writing JS scripts for invoking knex migrations programmaticaly in all my projects.

[CodyEakins]

@kibertoad That is my work around for the time being... I've seen pros and cons to both methods`.

Either way, this is still a bug, correct? It should be fixed to help those who choose to use Knex like this, for as long as it is a supported option.

[kibertoad]

@CodyEakins I'm not sure it's a bug, considering that I don't think we ever supported passing knex instance from knexfile.js, only configuration itself. I would consider it more of an improvement, but PR for it would still be welcome. What is the advantage that you see with using migrations API via CLI while instantiating knex programmatically?

[CodyEakins]

@kibertoad If it is unsupported, then why not just remove the flag then? Or am I misunderstanding you?

One benefit of using knex through the command-line is to help keep app initialization clean when using Docker (e.g. - CMD knex migrate:latest && yarn run start).

While I don't use this approach, I have seen it recommended by other developers as a method for running the latest migrations during app startup.

I prefer to run migrations through the migrations API, however.

---

Rather than directly impacting my workflow, this bug is more about the --knexpath flag itself.

How to use it and why? In the current API, this is unclear.

And if --knexpath and --knexfile are meant to be used together, this just means that --knexpath has little to no effect on Knex's behavior (given the current API and presumed behavior).

[kibertoad]

@CodyEakins To the same effect you could be calling node scripts/migrateLatest.js && yarn run start that would be invoking knex.migrate.latest(yourConfig) inside of it.

I guess --knexpath needs better explanation about what it is, and how it is supposed to be used. Looking at the tests I happen to think that "knex instance" does not mean what it appears to mean in this context.

[CodyEakins]

@kibertoad In Knex's current state, a developer has to make certain sacrifices:

• They must include a knexfile.js in their project's root.
• They cannot pass a valid config file to knex using the --knexfile flag if the config file is not strictly named knexfile.<ext>
• They cannot pass a valid, pre-configured instance to knex using --knexpath.
• (If relying only on the API, avoiding a knexfile.js) They must make their own scripts to run even basic commands like creating a new blank migration.

All these sacrifices negatively impact the developer experience of using knex...

---

In my opinion, the --knexpath and --knexfile flags should be dropped in support of better, more usable flags like --config and --instance.

Using --config would allow a developer to pass a valid config file of any name and in any location to knex.

Using --instance would allow a developer to pass a valid, pre-configured instance of knex to knex, which would infer any required configuration values from it.

Libraries like babel and webpack follow approaches similar to these. I don't see why knex shouldn't...

[kibertoad]

We can't drop existing flags for the sake of backwards compatibility. However, we would accept PRs that would implement more intuitive and convenient to use configuration approaches.

[CodyEakins]

Right, my mistake. But thanks for understanding.

I'll try to make some PRs in the near future to correct this behavior. See if it makes the library easier to use.

[briandamaged]

From what I can tell, it looks like the support for --knexpath might have been broken in a previous release. More specifically: the --knexpath and --knexfile flags are being passed to the Liftoff library incorrectly; as a result, the values are being dropped before the Knex CLI can use them.

Fortunately, it looks like you can still provide a custom Knex path via the KNEX_PATH environment variable. Ex:

KNEX_PATH=./services/knex.js knex migrate:latest

I might end up inadvertently fixing this as part of another CLI bug fix. Stay tuned!

[MarcosRoch4]

perhaps you had created knexfile.ts in a wrong directory. take a look at that.

[mauroere]

1. They must include a knexfile.js in their project's root.
2. Re install sqlite3 (npm install sqlite3 --save)
3. Run migrations command ( ex: npx knex migrate:latest --knexfile knexfile.ts migrate:latest)

[mercmobily]

I think this should work more intuitively.

The intention behind --knexpath was to let you point the CLI at a local install of knex (or a helper script), not to pass a pre‑instantiated client. As a result, the CLI still expects configuration via knexfile or flags. Two actionable items:

• Documentation: clarify what --knexpath does today and point to KNEX_PATH=... knex ... and the programmatic migrations API when you want to use an instantiated client.
• Enhancement: accept a module that exports a knex instance and infer the config from it when no knexfile is provided. The CLI already has a flexible import path (bin/cli.js:1–120 and openKnexfile via lib/migrations/util/import-file.js);

As a workaround, run migrations programmatically (knex.migrate.latest() in a small script), or set KNEX_PATH=./services/knex.js knex migrate:latest (this works today to point the CLI at your project’s knex install).