faktory_worker_haskell

Haskell client and worker process for the Faktory background job server.

Architecture overview from Ruby client README:

                       +--------------------+
                       |                    |
                       |     Faktory        |
                       |     Server         |
        +---------->>>>|                    +>>>>--------+
        |              |                    |            |
        |              |                    |            |
        |              +--------------------+            |
+-----------------+                            +-------------------+
|                 |                            |                   |
|    Client       |                            |     Worker        |
|    pushes       |                            |     pulls         |
|     jobs        |                            |      jobs         |
|                 |                            |                   |
|                 |                            |                   |
+-----------------+                            +-------------------+

• Client - an API any process can use to push jobs to the Faktory server.
• Worker - a process that pulls jobs from Faktory and executes them.
• Server - the Faktory daemon which stores background jobs in queues to be processed by Workers.

This package contains only the client and worker parts. The server part is here

Installation

• Hackage: http://hackage.haskell.org/package/faktory
• Stackage: Coming soon

Faktory Documentation

See the wiki for more details.

Usage

Job

Any value can be a "Job" that is pushed and pulled to and from Faktory via its ToJSON and FromJSON instances:

newtype MyJob = MyJob
  { myJobMessage :: String
  }
  deriving stock Generic
  deriving anyclass (ToJSON, FromJSON)

Worker

workerMain = runWorkerEnv $ \job -> do
  -- Process your Job here
  putStrLn $ jobJid job
  putStrLn $ myJobMessage $ jobArg job

  -- If any exception is thrown, the job will be marked as Failed in Faktory
  -- and retried. Note: you will not otherwise hear about any such exceptions,
  -- unless you catch-and-rethrow them yourself.

Producer

Producer wraps Client for push-only usage.

producerMain = do
  producer <- newProducerEnv

  jobId <- perform mempty producer $ MyJob "Hello world"

  print jobId

  closeProducer producer

Configuration

When using envSettings, the following variables will be used:

• FAKTORY_PROVIDER: the name of another environment variable where the connection string can be found. Defaults to FAKTORY_URL.
• FAKTORY_URL (or whatever you named in FAKTORY_PROVIDER): connection string to the Faktory server. Format is tcp(+tls)://(:password@)host:port(/namespace). Defaults to tcp://localhost:4719. namespace is prependend to queue names on job submission and worker consumption.

When using envWorkerSettings, the following variables are also used:

• FAKTORY_QUEUE: the name of the queue to consume from. Default is "default".
• FAKTORY_WORKER_ID: the Id to use for this Worker. Default is to assign a random one.

Examples

See the examples. To run them:

1. Run a local Faktory server

   docker run --rm \
     --publish 7419:7419 \
     --publish 7420:7420 \
     contribsys/faktory
   

2. Run the consumer example

   % stack exec faktory-example-consumer
   Starting consumer loop
   

   (Assumes you've built the project.)

3. Submit a Job through the producer example

   % stack exec faktory-example-producer hello world
   Pushed job: "ljcjlbexbgun"
   

   NOTE: if you submit "BOOM" as a Job, the processing loop will raise an exception, so you can see how a Failed Job looks in Faktory.

4. See that your Job was processed back in the consumer

   % stack exec faktory-example-consumer
   Starting consumer loop
   hello world
   

Development & Tests

stack build --dependencies-only --test --no-run-tests
stack build --pedantic --test --no-run-tests
stack build --pedantic --test

• FactorySpec requires a local Faktory server is running, and it will flush all Jobs from this server as part of running the tests.
• The tests for BATCH require testing against an Enterprise Faktory image

CHANGELOG | LICENSE