The Project | About Us | Contribute | Donations | License

HOME

We reach the lesson that settles three debts at once. The first we took on in 10-01: when we moved to seven workers, the currency cache, the session store and the limiter's counter fragmented into seven copies, and Marc gets logged out every other request while a malicious script enjoys a quota multiplied by seven. The second comes from Module 7: getCatalog() recomputes the entire catalog on every request only to return the same thing every time. The third was left by 10-02: the Auditorio Ribera organizer is still waiting four and a half seconds for the 500 Festival de Jazz opening-night tickets to be generated.

All three are solved with the same piece of infrastructure: Redis.

What Redis is and which data types we will use
Running it and connecting with ioredis
Caching the catalog: the cache-aside pattern
Well-designed keys and the hard problem: invalidation
Cache stampede and its remedies
What must never be cached, and measurement
Shared sessions and rate limiting
Job queues: the change of model and the 202
BullMQ: producer, consumer and the options that matter
Idempotency, scheduled jobs and observability
Persistence: Redis can lose data

What Redis is and which data types we will use

Redis is an in-memory key-value store. Three characteristics define how to use it well: it lives in RAM (reads and writes in microseconds, not milliseconds); it runs commands on a single thread, just like our JavaScript, so every command is atomic —there are no race conditions between clients— but a slow command blocks everybody (which is why KEYS * is banned in production); and it has data structures, not just strings, which is where its real power lies.

Type	Key commands	Use in Escena Viva
String	`GET`, `SET`, `INCR`	Catalog serialized as JSON; limiter counters
Hash	`HSET`, `HGETALL`, `HINCRBY`	Data for a user session; capacity per show session
List	`LPUSH`, `RPOP`, `BRPOP`	The basis of job queues (BullMQ uses it internally)
Set	`SADD`, `SISMEMBER`	Identifiers of jobs already processed (idempotency)
Sorted set	`ZADD`, `ZRANGEBYSCORE`	Jobs scheduled by timestamp; best-seller ranking
TTL (cross-cutting)	`EXPIRE`, `TTL`, `SET ... EX`	Automatic expiry for everything above

TTL is not a type, but it is the feature that turns Redis into a cache: any key can expire on its own, which saves us from writing cleanup code.

Running it and connecting with `ioredis`

For development, a container is enough:

# Redis 7 with AOF persistence enabled, on the standard port.
docker run -d --name redis-escena-viva -p 6379:6379 -v redis-data:/data \
  redis:7-alpine redis-server --appendonly yes
docker exec -it redis-escena-viva redis-cli ping   # must answer PONG

In a docker-compose.yml it would be one more service alongside PostgreSQL, with its image, its command, its port and its volume. We will not go deeper: containers and composition are covered in Module 11. We install the client (npm install ioredis) and connect, respecting the project's discipline: src/config/index.js is the only file that reads process.env.

'use strict';

const Redis = require('ioredis');
const { configuration } = require('../config/index.js');

let redisClient = null;

// A single connection per process. ioredis multiplexes: no pool needed.
function getRedisClient() {
  if (redisClient) return redisClient;
  redisClient = new Redis(configuration.redis.url, {
    retryStrategy: (attempt) => Math.min(attempt * 200, 2000), // growing wait
    maxRetriesPerRequest: null,             // BullMQ requirement
    keyPrefix: `${configuration.nodeEnv}:`, // isolates environments
  });
  // We never let a Redis failure take the process down.
  redisClient.on('error', (error) => console.error('[redis]', error.message));
  return redisClient;
}

// closeRedisClient() calls quit() and hooks into the graceful shutdown.
module.exports = { getRedisClient, closeRedisClient };

Caching the catalog: the cache-aside pattern

getCatalog() is the perfect candidate: it returns the same thing to everybody (it does not depend on the user or the role), it is read a great deal and written rarely, it is expensive to compute (it aggregates the 3 events, their 7 sessions, the available capacity and the prices) and it tolerates a small lag, because the actual sale is validated against the database with SELECT ... FOR UPDATE (M7).

The cache-aside pattern has four steps: look; if it misses, compute; store with a TTL; return.

'use strict';

const { getRedisClient } = require('../db/redis.js');

const CATALOG_TTL_SECONDS = 60;
const SCHEMA_VERSION = 'v2'; // bumped when the shape of the object changes
// Namespace : entity : version : normalized parameters
const catalogKey = ({ category = 'all', page = 1, sort = 'date' } = {}) =>
  `catalog:events:${SCHEMA_VERSION}:${category}:${sort}:p${page}`;

// Wraps the M7 repository without the controllers noticing.
const createCachedCatalog = ({ eventRepository, redis = getRedisClient() }) => ({
  async getCatalog(params) {
    const key = catalogKey(params);
    let cached = null;

    // 1. Look. A Redis outage must not take the catalog down: we degrade.
    try { cached = await redis.get(key); } catch (e) { console.error('[cache]', e.message); }
    if (cached) return { data: JSON.parse(cached), source: 'cache' };

    // 2. Miss: compute with the usual repository.
    const data = await eventRepository.getCatalog(params);

    // 3. Store with a TTL ('EX' expresses the expiry in seconds).
    const json = JSON.stringify(data);
    try { await redis.set(key, json, 'EX', CATALOG_TTL_SECONDS); }
    catch (e) { console.error('[cache]', e.message); }

    return { data, source: 'database' }; // 4. return
  },
});

module.exports = { createCachedCatalog, catalogKey };

The elegance lies in what we have not touched. Thanks to the repository pattern from Module 7 and the dependency injection from Module 9, this object exposes the same interface as src/repositories/events.js. In the wiring we change what gets injected —configuration.cache.enabled ? createCachedCatalog({ eventRepository }) : eventRepository— and neither the controllers, nor the routes, nor the tests notice a thing. Another important decision: if Redis fails, the application keeps working, just more slowly. A cache must never be a single point of failure for a read that can be recomputed.

Well-designed keys and the hard problem: invalidation

Part of the key	Example	Why
Namespace	`catalog:`	Enables pattern deletions and sharing one instance
Entity and identifier	`events:evt-003`	Readable while debugging with `redis-cli`
Schema version	`v2`	Changing the format invalidates everything without deleting anything
Normalized parameters	`:date:p1`	Two different queries do not share an entry

The version deserves emphasis. If tomorrow you add the availableTickets field to the catalog object and deploy, the seven new workers will read old objects without that field and fail. Bumping SCHEMA_VERSION to v3 means no old key is ever found: a safe deployment, with no FLUSHDB and no maintenance window. And now the hard part. There is a classic Phil Karlton joke that says there are only two hard things in computer science: cache invalidation and naming things. It describes something real: storing is trivial, knowing when what you stored has stopped being true is not.

Strategy	How	Advantages	Risks
Short TTL	The key expires after 60 s	Dead simple; self-healing	Stale data for up to 60 s; unnecessary recomputation
Explicit invalidation	On a sale, the key is deleted	Fresh data almost instantly	If you forget one write path, you serve false data forever

Combining them is the sane practice: explicit invalidation as the main path, TTL as a safety net.

'use strict';

// Called from the same place that already listens for 'sale-recorded' on
// the SalesManager (domain, Module 2).
const createCatalogInvalidator = ({ redis }) => ({
  async invalidateOnSale({ eventId }) {
    let cursor = '0';
    const keys = [];
    do {
      // SCAN, never KEYS: KEYS blocks Redis's only thread.
      const [next, batch] = await redis.scan(cursor, 'MATCH', 'catalog:events:*', 'COUNT', 100);
      cursor = next;
      keys.push(...batch);
    } while (cursor !== '0');
    if (keys.length > 0) await redis.del(...keys);
    await redis.del(`event:${eventId}:detail:v2`); // direct invalidation
  },
});

module.exports = { createCatalogInvalidator };

If the number of variants grows, SCAN stops being convenient. The professional alternative is an index set (when you store a key you add its name to a SET; to invalidate you read them and delete them in one go) or, even simpler, a catalog:generation counter that is part of the key: incrementing it makes every previous key unreachable, and they expire on their own.

Cache stampede and its remedies

Opening-night scenario: the catalog key expires at 20:59:31. In that millisecond there are 1,000 requests in flight. All 1,000 miss the cache, all 1,000 call getCatalog(), all 1,000 hit PostgreSQL with the expensive query. This is the cache stampede (thundering herd), and it usually takes the database down at exactly the worst moment. The first remedy is a lock with SET NX: only one process recomputes, and the rest wait a moment and read again.

'use strict';

async function getWithLock({ redis, key, ttlSeconds, compute }) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  // NX: only writes if it does not exist. PX: expires in 5 s, so a process
  // that dies halfway does not leave the lock held forever.
  if (await redis.set(`${key}:lock`, '1', 'PX', 5000, 'NX')) {
    try {
      const data = await compute();
      await redis.set(key, JSON.stringify(data), 'EX', ttlSeconds);
      return data;
    } finally { await redis.del(`${key}:lock`); }
  }
  // We are not the chosen one: wait briefly and read again. If it is still
  // missing, we compute anyway: better slow than an error.
  await new Promise((resolve) => setTimeout(resolve, 50));
  const retry = await redis.get(key);
  return retry ? JSON.parse(retry) : compute();
}

Early recomputation: the value is stored together with its expiry instant and, when little time is left, every read has a growing probability of refreshing it in the background while serving the current one, so nobody ever waits and the key never expires all at once. It is more elegant and a little more code; for Escena Viva, the SET NX lock is enough.

What must never be cached, and measurement

Data	Cache it?	Reason
Public event catalog	Yes, TTL 60 s	The same for everyone, expensive, tolerates lag
An event's detail page	Yes, TTL 300 s	Changes very rarely
Currency exchange rates	Yes, TTL 3,600 s	Already cached in memory (M8); now shared
Lucía's orders	No	Personal data; it could end up served to another user
Responses that depend on the role	No (or with the role in the key)	An administrator sees fields an attendee does not
Exact capacity when selling	Never	Selling against stale capacity is overselling
Tokens and passwords	No	Neither in the cache nor in logs (M8)

The capacity case is worth pausing on. In a listing, showing "42 tickets left" with 60 seconds of lag is acceptable, indicative information. But the decision to sell must be made against the database, inside the transaction with SELECT ... FOR UPDATE in src/repositories/purchases-sql.js. If somebody "optimized" buyTickets by reading capacity from Redis, on opening night we would sell the same seats to two people. A cache is for reading, not for deciding. And one classic security bug: caching a response that includes user data without putting the user's identifier in the key, so Marc ends up seeing Lucía's orders. Defensive rule: if the response depends on who is asking, either you do not cache it, or the "who" is part of the key.

The same load test as in lesson 10-01, now with caching:

Metric (`GET /api/events`, 50 connections, 7 workers)	Without cache	With cache-aside
Requests per second	3,105	11,840
p50 / p99 latency	15 ms / 54 ms	3 ms / 11 ms
PostgreSQL queries in 10 s	31,050	7
PostgreSQL CPU usage	78%	4%

The figure that really matters is not requests per second: it is the 7 queries versus 31,050. We have freed the database, the shared bottleneck that lesson 10-01 could not touch.

Shared sessions and rate limiting

We settle the debt from lesson 10-01 with npm install connect-redis rate-limit-redis:

'use strict';

const session = require('express-session');
const { RedisStore } = require('connect-redis');
const rateLimit = require('express-rate-limit');
const { RedisStore: LimitStore } = require('rate-limit-redis');
const { getRedisClient } = require('../db/redis.js');
const { configuration } = require('../config/index.js');

// SESSIONS. Before: an in-memory store (one copy per worker). Now: a
// shared store; all 7 workers see the same session.
const createSessionMiddleware = () => session({
  store: new RedisStore({ client: getRedisClient(), prefix: 'session:', ttl: 60 * 60 * 8 }),
  name: 'escena_viva_sid',
  secret: configuration.session.secret,
  resave: false,
  saveUninitialized: false,
  cookie: { httpOnly: true, sameSite: 'lax', maxAge: 1000 * 60 * 60 * 8 },
});

// LIMIT. The store uses INCR + EXPIRE atomically in Redis: the counter is
// unique across the 7 workers and the limit becomes real again.
const purchaseLimit = rateLimit({
  windowMs: 60_000,
  limit: 20,
  standardHeaders: 'draft-7',
  store: new LimitStore({ prefix: 'limit:purchase:',
    sendCommand: (...args) => getRedisClient().call(...args) }),
  message: { error: { code: 'TOO_MANY_REQUESTS', status: 429, details: [],
    message: 'You have exceeded the purchase limit per minute.' } },
});

module.exports = { createSessionMiddleware, purchaseLimit };

Verified with exercise 2 from lesson 10-01: with generalLimit at 20 per minute and 4 workers, request 21 now gets a 429, whichever worker it lands on. The same applies to src/services/currency-exchange.js: replacing its in-memory Map with currency:EUR:USD keys with a one-hour TTL leaves a single call to the external API per hour across the whole fleet, instead of seven.

Job queues: the change of model and the 202

Lesson 10-02 moved PDF generation off the main thread, but the Auditorio Ribera organizer is still waiting 4.4 seconds with the HTTP connection open. And if those 500 tickets also have to be e-mailed, we are talking minutes: proxy timeouts fire, the user reloads the page and duplicates the work, and if the process restarts halfway through, the work is lost without a trace.

sequenceDiagram
  participant O as Organizer
  participant A as API
  participant R as Redis (queue)
  participant C as Consumer
  O->>A: POST /api/orders/ord-77/tickets
  A->>R: enqueue generate-tickets
  A-->>O: 202 Accepted + { jobId, statusUrl }
  C->>R: take job and process (QR + PDF + e-mail)
  O->>A: GET /api/jobs/job-9f3 (polling)
  A-->>O: 200 { status: 'completed', downloadUrl }

The API moves from "do this and wait" to "I accept your order, here is your tracking number". That is exactly what the 202 Accepted status code means, and we will formalize it as a design decision in lesson 10-05.

BullMQ: producer, consumer and the options that matter

BullMQ (npm install bullmq) implements reliable queues on top of Redis using lists and sorted sets, with Lua scripts to guarantee atomicity. The producer lives in the web application:

'use strict';

const { Queue } = require('bullmq');
const { configuration } = require('../config/index.js');

const ticketQueue = new Queue('generate-tickets', {
  connection: { url: configuration.redis.url },
  defaultJobOptions: {
    attempts: 3,                                   // maximum attempts
    backoff: { type: 'exponential', delay: 2000 }, // 2 s, 4 s, 8 s
    removeOnComplete: { age: 3600, count: 1000 },  // automatic cleanup
    removeOnFail: { age: 24 * 3600 },              // failures last one day
  },
});

// The job id is the order id: enqueuing the same order twice does NOT
// create two jobs. Idempotency in the producer.
const enqueueTicketGeneration = ({ orderId, sessionId, email }) =>
  ticketQueue.add('generate-pdf',
    { orderId, sessionId, email, requestedAt: new Date().toISOString() },
    { jobId: `tickets:${orderId}` });

module.exports = { ticketQueue, enqueueTicketGeneration };

The consumer lives in a separate process (node src/processes/ticket-consumer.js), not in the web server: the heavy work no longer shares a state machine with HTTP requests.

'use strict';

const { Worker } = require('bullmq');
const { configuration } = require('../config/index.js');
const { ThreadPool } = require('../workers/pool.js');
const { getOrderById } = require('../repositories/orders.js');
const { sendTicketsEmail } = require('../services/email.js');
const { alreadyProcessed, markProcessed } = require('../services/idempotency.js');

const pool = new ThreadPool({ size: 2 });

const consumer = new Worker('generate-tickets', async (job) => {
  const { orderId, email } = job.data;
  // The same job can run twice (a retry after a network failure, or the
  // process dying after the work but before acknowledging it).
  if (await alreadyProcessed(`tickets:${orderId}`)) return { skipped: true };

  const order = await getOrderById(orderId);
  await job.updateProgress(20);
  const { buffer, generated } = await pool.run({
    orderId: order.id, eventTitle: order.eventTitle, tickets: order.tickets,
    venueName: order.venueName, sessionDate: order.sessionDate,
  });
  await job.updateProgress(70);
  await sendTicketsEmail({ to: email, attachment: buffer });
  await markProcessed(`tickets:${orderId}`, 7 * 24 * 3600);
  return { generated, sentTo: email };
}, {
  connection: { url: configuration.redis.url },
  concurrency: 2,                          // simultaneous jobs here
  limiter: { max: 30, duration: 60_000 },  // the e-mail provider's quota
});

consumer.on('failed', (j, error) => console.error(`[queue] ${j?.id}: ${error.message}`));

// Graceful shutdown, as in M6: we finish the job in progress.
for (const signal of ['SIGTERM', 'SIGINT']) {
  process.on(signal, async () => { await consumer.close(); await pool.close(); });
}

module.exports = { consumer };

Option	What it does	Criterion
`attempts`	Retries before giving up on the job	3-5 with external I/O; 1 if it is not idempotent
`backoff`	Wait between retries	`exponential`: do not hammer an already failing service
`removeOnComplete`	Cleans up finished jobs	Without it, Redis grows indefinitely
`removeOnFail`	Retention of failures	Keep them: they are your diagnosis
`concurrency`	Jobs at a time per consumer	Limited by CPU and by the thread pool
`limiter`	Rate ceiling	Quotas of external APIs (e-mail, payment gateway)
`jobId`	The job's identity	Deduplication in the producer

When a job exhausts its attempts it moves to the failed set (the "failed queue"). It does not disappear: it stays there with its payload, its error and its stack, and it can be retried with job.retry() once the cause is fixed. An alert on the size of failed is one of the most useful production signals there is.

Idempotency, scheduled jobs and observability

Consumer idempotency. Every serious queue guarantees at least once, not exactly once. If the job sends e-mails, charges cards or creates orders, duplication is a real incident. The protection in src/services/idempotency.js is a marker in Redis: markProcessed runs SET processed:<key> 1 EX <ttl> NX —atomic, and therefore safe with N consumers— and alreadyProcessed checks with EXISTS before acting.

Better still when you can: make the operation naturally idempotent. If the PDF is stored as order-ord-77.pdf and the e-mail is recorded with a unique key per order, repeating the job does no harm even if nobody checks anything.

Scheduled and repeatable jobs. In Module 3 we generated the nightly occupancy report with a system cron and a standalone script; with BullMQ it becomes part of the application:

const reportQueue = new Queue('reports', { connection: { url: configuration.redis.url } });

// Repeatable: every day at 03:00, Madrid time.
const scheduleNightlyReport = () => reportQueue.upsertJobScheduler(
  'daily-occupancy-report',
  { pattern: '0 3 * * *', tz: 'Europe/Madrid' },
  { name: 'occupancy', data: { scope: 'all-venues' } }
);

// Delayed: a reminder 24 h before the session, with the 'delay' option.
const scheduleReminder = ({ orderId, sessionDate }) =>
  reportQueue.add('reminder', { orderId },
    { delay: new Date(sessionDate).getTime() - Date.now() - 24 * 3600 * 1000 });

The advantage over the system cron is twofold: it works with N workers without running N times (Redis coordinates), and the job inherits retries, history and observability. On that last point precisely, this is the minimum you have to watch in a queue:

Signal	How you get it	What it means if it spikes
Waiting jobs	`queue.getWaitingCount()`	The consumers cannot keep up
Age of the oldest one	`queue.getJobs(['waited'], 0, 0)`	The real delay the user perceives
Failed jobs	`queue.getFailedCount()`	Something is genuinely broken
Active jobs	`queue.getActiveCount()`	Compared with `concurrency`, it indicates saturation

There is bull-board to view all this in a web interface; in Escena Viva it is enough to expose the counters at GET /api/health/queues and let the Module 11 monitoring pick them up.

Persistence: Redis can lose data

Redis lives in memory and offers two persistence mechanisms, which can be combined:

Mechanism	How it works	Risk of loss
RDB (snapshot)	Dumps the dataset to disk every X changes	Everything since the last snapshot (minutes)
AOF (operation log)	Appends every write to a file	Up to 1 second with `everysec`
Both	RDB to restore fast, AOF for durability	Recommended in production

Even with AOF you can lose the last second. Therefore: for the cache it does not matter (losing it only means recomputing); for sessions it is annoying but tolerable; for queues it matters, because a lost job is an e-mail that is never sent. That is why the order is written to PostgreSQL first and the job is enqueued afterwards: if the job is lost, the order is still there and a reconciliation process can re-enqueue it. The rule that closes the lesson: the source of truth is still the Module 7 database. Redis is a speed and coordination layer, never the definitive record of the business.

Common Mistakes and Tips

Caching without a TTL. A key with no expiry is a memory leak with stale data inside it.
Using KEYS in production, which blocks Redis's only thread and freezes the whole application. SCAN, always.
Caching user-dependent responses without the user in the key. That is a personal-data leak.
Reading capacity from the cache to decide a sale. Guaranteed overselling on opening night.
Letting a Redis failure take the application down. Wrap the reads in try/catch and degrade to the database.
Putting the queue consumer inside the web server, or assuming a job runs exactly once: it runs at least once, so design for repetition.
Tip: prefix the keys with the environment (production:, development:). It avoids the classic "I flushed the production cache by accident".
Tip: in development, return an X-Cache: HIT|MISS header. Seeing the hit ratio live teaches more than any chart.

Exercises

Exercise 1: measure the hit ratio

Instrument createCachedCatalog to count hits and misses in Redis (INCR on metrics:cache:hits and :misses) and expose them at GET /api/health/cache. Fire 2,000 requests with autocannon and calculate the ratio with a TTL of 60 s and of 5 s. Interpret the difference.

Exercise 2: a queue with retries and a failed queue

Create a send-emails queue whose processor fails deterministically when the recipient ends in @invalid.test. Configure 3 attempts with exponential backoff. Enqueue 5 jobs, 2 of them invalid, and check the final state of each one and the contents of the failed set.

Exercise 3: idempotency under duplication

Enqueue the same ticket-generation job for ord-77 twice, once with a fixed jobId and once without it. Check how many times the processor runs in each case. Then simulate a retry by forcing a failure after sending the e-mail and verify that the idempotency protection prevents the second send.

Solutions

Exercise 1. With a 60 s TTL and a 10-second test, the hit ratio is around 99.9%: only the first request for each variant misses. With a 5 s TTL there is one miss every 5 seconds per variant, so the ratio drops to ~99.4% and the PostgreSQL queries are multiplied by 12. The lesson is that the TTL is a dial that regulates the balance between freshness and load, and that there is a point of diminishing returns: going from 60 to 600 seconds barely improves the ratio, but it multiplies the stale-data window by ten. With explicit invalidation properly in place, you can raise the TTL with confidence.

Exercise 2. The 3 valid jobs land in completed on the first attempt. The 2 invalid ones are retried after 2 s, 4 s and 8 s, accumulate attemptsMade: 3 and end up in failed, keeping failedReason and the stack; queue.getFailedCount() returns 2, and const [j] = await queue.getFailed(); await j.retry(); sends them back to waiting. An important detail: with exponential backoff, 3 attempts consume 14 seconds, so with many jobs failing at once the queue clogs up. That is why it is wise to combine moderate attempts with an alert on failed.

Exercise 3. With jobId: 'tickets:ord-77', the second call to add does not create a new job: BullMQ returns the existing one and the processor runs exactly once. Without jobId, two jobs are created, the processor runs twice and, without the idempotency check, Lucía would receive two e-mails with 500 tickets each. In the second part, when it fails after the send, BullMQ retries and the processor starts again, but alreadyProcessed('tickets:ord-77') returns true and it exits with { skipped: true }. The key lesson: jobId protects against duplicates in the producer, and the marker in Redis protects against re-execution in the consumer. You need both.

Conclusion

Redis has solved three problems with a single piece. The catalog is served from cache with the cache-aside pattern, versioned keys, invalidation on sale, a TTL as a safety net and stampede protection: from 3,105 to 11,840 requests per second, and from 31,050 PostgreSQL queries to 7. Sessions and limiter counters are correct again with seven workers, settling the debt from lesson 10-01. And the heavy work has moved out of the web process into a BullMQ queue with retries, exponential backoff, a failed queue, idempotency in both producer and consumer, scheduled jobs and observability: the Auditorio Ribera organizer now receives an immediate 202 Accepted and a tracking identifier. We have also marked the limits: personal data is not cached, a sale is not decided against the cache, and Redis can lose data, so the source of truth is still PostgreSQL.

At this point, Escena Viva uses every core, does not block the event loop, caches and enqueues. But every figure we have quoted along the way —641, 3,105, 11,840 req/s; p99 of 204, 54 and 11 ms— appeared through a method we have not formalized. In the next lesson, Performance Optimization, we turn that practice into a discipline: what to measure and why the average lies, how to design an honest load test, how to read a flamegraph, how to hunt a memory leak by comparing heap snapshots, and what the correct order of the levers is before you buy a bigger machine.

Caching and Job Queues with Redis

Contents

What Redis is and which data types we will use

Running it and connecting with `ioredis`

Caching the catalog: the cache-aside pattern

Well-designed keys and the hard problem: invalidation

Cache stampede and its remedies

What must never be cached, and measurement

Shared sessions and rate limiting

Job queues: the change of model and the 202

BullMQ: producer, consumer and the options that matter

Idempotency, scheduled jobs and observability

Persistence: Redis can lose data

Common Mistakes and Tips

Exercises

Exercise 1: measure the hit ratio

Exercise 2: a queue with retries and a failed queue

Exercise 3: idempotency under duplication

Solutions

Conclusion

Node.js Course: From Beginner to Advanced

Module 1: Introduction to Node.js

Module 2: Core Concepts

Module 3: File System and I/O

Module 4: HTTP and Web Servers

Module 5: NPM and Package Management

Module 6: The Express.js Framework

Module 7: Databases and ORMs

Module 8: Authentication and Authorization

Module 9: Testing and Debugging

Module 10: Advanced Topics

Module 11: Deployment and DevOps

Module 12: Real-World Projects

Caching and Job Queues with Redis

Contents

What Redis is and which data types we will use

Running it and connecting with ioredis

Caching the catalog: the cache-aside pattern

Well-designed keys and the hard problem: invalidation

Cache stampede and its remedies

What must never be cached, and measurement

Shared sessions and rate limiting

Job queues: the change of model and the 202

BullMQ: producer, consumer and the options that matter

Idempotency, scheduled jobs and observability

Persistence: Redis can lose data

Common Mistakes and Tips

Exercises

Exercise 1: measure the hit ratio

Exercise 2: a queue with retries and a failed queue

Exercise 3: idempotency under duplication

Solutions

Conclusion

Node.js Course: From Beginner to Advanced

Module 1: Introduction to Node.js

Module 2: Core Concepts

Module 3: File System and I/O

Module 4: HTTP and Web Servers

Module 5: NPM and Package Management

Module 6: The Express.js Framework

Module 7: Databases and ORMs

Module 8: Authentication and Authorization

Module 9: Testing and Debugging

Module 10: Advanced Topics

Module 11: Deployment and DevOps

Module 12: Real-World Projects

Running it and connecting with `ioredis`