Skip to content

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis

License

Notifications You must be signed in to change notification settings

AsysKevin/bullmq

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation




The fastest, most reliable, Redis-based distributed queue for Node.
Carefully written for rock solid stability and atomicity.

Read the documentation

Follow @manast for *important* Bull/BullMQ/BullMQ-Pro news and updates!

🛠 Tutorials

You can find tutorials and news in this blog: https://blog.taskforce.sh/

Official FrontEnd

Taskforce.sh, Inc

Supercharge your queues with a professional front end:

  • Get a complete overview of all your queues.
  • Inspect jobs, search, retry, or promote delayed jobs.
  • Metrics and statistics.
  • and many more features.

Sign up at Taskforce.sh

🚀 Sponsors 🚀

Dragonfly Dragonfly is a new Redis™ drop-in replacement that is fully compatible with BullMQ and brings some important advantages over Redis™ such as massive better performance by utilizing all CPU cores available and faster and more memory efficient data structures. Read more here on how to use it with BullMQ.
Memetria for Redis If you need high quality production Redis instances for your BullMQ project, please consider subscribing to Memetria for Redis, leaders in Redis hosting that works perfectly with BullMQ. Use the promo code "BULLMQ" when signing up to help us sponsor the development of BullMQ!

Used by

Some notable organizations using BullMQ:

Microsoft Vendure Datawrapper Nest
Curri Novu NoCodeDB

The gist

Install:

$ yarn add bullmq

Add jobs to the queue:

import { Queue } from 'bullmq';

const queue = new Queue('Paint');

queue.add('cars', { color: 'blue' });

Process the jobs in your workers:

import { Worker } from 'bullmq';

const worker = new Worker('Paint', async job => {
  if (job.name === 'cars') {
    await paintCar(job.data.color);
  }
});

Listen to jobs for completion:

import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents('Paint');

queueEvents.on('completed', ({ jobId }) => {
  console.log('done painting');
});

queueEvents.on(
  'failed',
  ({ jobId, failedReason }: { jobId: string; failedReason: string }) => {
    console.error('error painting', failedReason);
  },
);

This is just scratching the surface, check all the features and more in the official documentation

Feature Comparison

Since there are a few job queue solutions, here is a table comparing them:

Feature BullMQ-Pro BullMQ Bull Kue Bee Agenda
Backend redis redis redis redis redis mongo
Observables
Group Rate Limit
Group Support
Batches Support
Parent/Child Dependencies
Priorities
Concurrency
Delayed jobs
Global events
Rate Limiter
Pause/Resume
Sandboxed worker
Repeatable jobs
Atomic ops
Persistence
UI
Optimized for Jobs / Messages Jobs / Messages Jobs / Messages Jobs Messages Jobs

Contributing

Fork the repo, make some changes, submit a pull-request! Here is the contributing doc that has more details.

Thanks

Thanks for all the contributors that made this library possible, also a special mention to Leon van Kammen that kindly donated his npm bullmq repo.

About

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 79.9%
  • Python 10.4%
  • Lua 9.4%
  • Other 0.3%