logto icon indicating copy to clipboard operation
logto copied to clipboard

bug: Errors that happen during tenant initialization are not logged to console

Open vavido opened this issue 1 year ago • 2 comments

Describe the bug

Errors that happen during tenant initialization are not reported on stdout.

Expected behavior

Whenever an error happens in the application, it should be printed to stdout to give system admins actionable information on what went wrong and how it might be fixed.

How to reproduce?

  1. Modify the database of a logto instance so that login is not possible for the logto_tenant_logto_admin user (e.g. change password)
  2. Start logto
  3. Navigate to admin console (localhost:3001)

The console only returns "Internal server error" with status code 500. No error information is printed to stdout.

Context

  • [ ] Logto Cloud
  • [x] Self-hosted, initially happened on 1.11.0, but same behavior with 1.15.0
    • [x] Container (Docker image)
    • [ ] Raw Node.js

Additional Info:

I discovered this bug today while trying to migrate the database used by logto to a new cluster. After starting logto with the new Database, the console only returned "Internal Server Error" and no error message was visible in the container logs. In the end, I had to roll back the migration because I couldn't figure out what was wrong.

Later, I cloned the repo and started adding additional error logging in various places to try and figure out what the actual error was. In the end, adding consoleLog.error(error) to the

    const tenant = await trySafe(tenantPool.get(tenantId), (error) => {

callback in init.ts provided me with the hidden error message. With this additional line, my stdout now is

 ...
 info Init tenant: admin
error error: password authentication failed for user "logto_tenant_logto_admin"
    at Parser.parseErrorMessage (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:287:98)
    at Parser.handlePacket (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:39:38)
    at TLSSocket.<anonymous> (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/index.js:11:42)
    at TLSSocket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at Readable.push (node:internal/streams/readable:278:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23) {
  length: 120,
  severity: 'FATAL',
  code: '28P01',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'auth.c',
  line: '335',
  routine: 'auth_failed'
}

If I had the information during my initial migration, it would be simple to fix, but without the error info, i had to roll everything back after ~2h of trying various random things.

vavido avatar Apr 28 '24 19:04 vavido

Might also be related to https://github.com/logto-io/logto/issues/4911

vavido avatar Apr 28 '24 19:04 vavido

Thanks for the detailed information. We'll take a look!

charIeszhao avatar Apr 29 '24 01:04 charIeszhao