Worker Test Example

[williscool]

Was looking through the code for how to test as its not really clear from the readme. Here is something I hacked up.

Would appreciate any feedback on it. Once its decent I would be happy to make a pr. This example is for jest with @babel/preset-env

Thanks for the great lib!

// tasks/hello.js
export default async (payload, helpers) => {
  const { name } = payload;
  helpers.logger.info(`Hello, ${name}`);
};

// test/tasks/hello.test.js
import hello from '../../src/tasks/hello';
import config from 'config'; // 
import { runOnce, quickAddJob } from 'graphile-worker';

const connectionString = config.get('pg_db_url'); // i.e. "postgres://user:pass@host:port/db?ssl=true"
const payloadName = 'Bobby Tables';

// based on https://github.com/graphile/worker#quickstart-library
describe('hello task', () => {
  it('does things', async (done) => {
    const spy = jest.spyOn(global.console, 'info');
    // Or add a job to be executed:
    const job = await quickAddJob(
      // makeWorkerUtils options
      { connectionString },

      // Task identifier
      'hello',

      // Payload
      { name: payloadName }
    );

    console.log(job);
    expect(job).toBeTruthy();

    // Run a worker to execute jobs:
    // eslint-disable-next-line
    const runner = await runOnce({
      connectionString,
      concurrency: 5,
      // Install signal handlers for graceful shutdown on SIGINT, SIGTERM, etc
      noHandleSignals: false,
      pollInterval: 1000,
      // you can set the taskList or taskDirectory but not both
      taskList: {
        hello,
      },
      // or:
      //   taskDirectory: `${__dirname}/tasks`,
    });

    // until https://github.com/graphile/worker/issues/28 is resolved with events
    // or i write my own event emmitter. we'll just test the logger
    //
    // this also is ok https://github.com/bpedersen/jest-mock-console
    // but i don't want to swallow logs this early in teh dev cycle
    //
    // https://stackoverflow.com/a/56677834/511710
    // https://jest-bot.github.io/jest/docs/expect.html#expectstringcontainingstring

    /**
     * graphile-worker's build in logger passes console calls like this
     * 
     * 1: "[%s%s] %s: %s", "job", "(worker-e03827077c435f77f5: hello{15})", "INFO", "Hello, Bobby Tables"
     * 2: "[%s%s] %s: %s", "worker", "(worker-e03827077c435f77f5)", "INFO", "Completed task 15 (hello) with success (20.03ms)"
     * 
     */
     expect(spy).toHaveBeenCalledWith(
      expect.anything(),
      expect.anything(),
      expect.anything(),
      expect.anything(),
      expect.stringContaining(payloadName)
    );

    done();
  });
});

[benjie]

I don’t have time to review this today; but you might like to look at the tests in Graphile Starter.l, a few of those involve worker methinks.

[benjie]

Since the worker tasks are just functions, do you actually want to test the DB infrastructure around them or do you just want to test the function itself? It seems that we could expose makeJobHelpers then you could do something like:

it('does things', async (done) => {
  const client = /* get client from pool */
  const withPgClient = (cb) => cb(client);
  const job = mockJob({task_identifier: 'hello', payload: {...payload...}})
  const helpers = makeJobHelpers({...options...}, job, {withPgClient});

  const spy = jest.spyOn(global.console, 'info');
  await hello(job.payload, helpers);
  expect(spy.mock.calls[0][4].includes(payloadName)).toBeTruthy();
});

I think it makes sense to include mockJob as part of graphile-worker because the Job definition may change over time. We can apply defaults to most of the fields, the only required one would be task_identifier.

What do you think?

[williscool]

yeah all i want to test is that given a certain payload to a task the things I want it to do are done.

I want the db stuff as extracted away as possible.

also I had a lot of flakiness in my tests until I added DROP SCHEMA graphile_worker CASCADE; to a beforeEach on all of my tests that use the workers. has been really solid ever since

[benjie]

You could just do delete from graphile_worker.jobs; I think; it might be more efficient time-wise.

[williscool]

updated to waaaay better way of writting spy tests

    expect(spy).toHaveBeenCalledWith(
      expect.anything(),
      expect.anything(),
      expect.anything(),
      expect.anything(),
      expect.stringContaining(payloadName)
    );

[slifty]

Just following up on this thread as I want to test one of my workers as well.

Did any tooling end up being exposed to support mocks so that the tests don't require a database connection? (I saw @benjie's thought about exposing makeJobHelpers but am not sure if that made it into the code base)

(Thank you for the great library!)

[benjie]

makeJobHelpers is indeed not exposed; I think you would need to mock it anyway.