bull-prom icon indicating copy to clipboard operation
bull-prom copied to clipboard

How to use it actually locally or in cloud?

Open Zachery2008 opened this issue 3 years ago • 1 comments

Hey, firstly thank you for sharing this package. Could you add some supported document about how to run/use it actually?

I am trying use this package for my project. I have 2 bull job queues, the purpose is to send the metrics of bull queue to prometheus in order to autoscale my pods. For initial try/test locally, I wrote simple express server to create bull queue, add jobs into the queue by GET request, and I consume the job trivially.

import Queue from 'bull'
import express = require('express')
import sleep = require('sleep');

const producer = new Queue('testQueue', {
  redis: {
    port: parseInt(process.env.REDIS_PORT || '6379'),
    host: '127.0.0.1',
  }
})

const consumer = new Queue('testQueue', {
  redis: {
    port: parseInt(process.env.REDIS_PORT || '6379'),
    host: '127.0.0.1',
  }
})

const app: express.Application = express()

let countIdx  = 0;
app.get('/add-job', async function (req, res) {
  producer.add({
    jobIndex: countIdx
  })

  let jobCnt = await getQueueLenth()
  console.log('Number of jobs in queue: ' + jobCnt)

  countIdx++
  console.log('Total jobs have been added: ' + countIdx)

  res.send('a job has added sucessfully!')
})

function getQueueLenth(){
  return consumer.count()
}

consumer.process(1, async (job) => {
  //sleep.sleep(30)
  await pauseTime(5000)
  console.log(job.data)
})

function pauseTime(ms: number){
  return new Promise((resolve) => {
    setTimeout(resolve, ms)
  })
}

app.listen(3000)

And I have set up a local prometheus server locally. I'm not sure how to set up the configuration yml file properly. And I don't understand the meaning of promClient and how to send metrics to prometheus. The following is my code:

import Queue from 'bull';
import promClient from 'prom-client';
import * as bullProm from 'bull-prom';
 
const queue = new Queue('testQueue', {
  redis: {
    port: parseInt(process.env.REDIS_PORT || '6379'),
    host: '127.0.0.1',
  }
})
 
const bullMetric = bullProm.init({
  promClient, //promClient: '0.0.0.0:9090', // optional, it will use internal prom client if it is not given
  interval: 5000, // optional, in ms, default to 60000
})

const started = bullMetric.start(queue);

Zachery2008 avatar Sep 08 '20 23:09 Zachery2008

Hi, sorry, for delayed reply.. I currently don't use bull-prom anymore, so can't exactly remember. I will however accept pull requests, in case someone from the community is reading it.

At the moment I can only recommend you have a look at the source code or documentation of prom-client.

pbadenski avatar Oct 29 '20 08:10 pbadenski