For six modules, Escena Viva has lived off a single file: data/events.json. It served us well. It taught us fs.promises, it let us build a rich domain with Event and Session, and it fed both the node:http server of Module 4 and the Express API of Module 6. But when we closed that module we left a promise unkept: the capacity and overselling problem was not going to be solved with validation or with well-typed errors, because it is not a problem of shape or of state — it is a problem of concurrency and persistence. That problem is the doorway into this module.
We are not going to write a Mongoose schema or a CREATE TABLE statement just yet. We are going to do something more important: understand what we are missing, what a database management system gives us and at what price, how to choose between the relational and the document worlds without joining a tribe, and how to design Escena Viva's data model. By the end you will have the complete blueprint of what we will build across the next five lessons.
Contents
- Why a JSON file stops being enough
- What a DBMS actually gives you
- Relational versus document
- ACID, BASE and the CAP theorem applied to capacity
- The database landscape for Node.js
- Native driver versus ORM/ODM
- Designing Escena Viva's data model
- The final model in a diagram
- Isolating persistence: the repository pattern
- Installing MongoDB and checking the connection
Why a JSON file stops being enough
Let's be honest and concrete. src/catalog-data.js is not a bad module: it reads data/events.json with fs.promises, memoizes the result and exposes getCatalog() and getEventById(). The problem is not its code, it is its storage model. These are the real — not theoretical — failures Escena Viva already has today:
- Concurrent writes that overwrite each other. If two requests sell tickets at the same time, both read the file, both modify their in-memory copy and both rewrite it whole. The last write wins and the first sale disappears. There is no lock, no arbitration, nobody deciding the order. It is exactly the overselling scenario.
- Non-atomic writes. A
writeFileof the entire catalog can be interrupted (disk failure,SIGKILL, container restart) leaving a truncated JSON file. A truncated JSON file is not a catalog with one event missing: it is a file that no longer parses, and the application will not start. - The whole catalog in memory. With 3 events and 7 sessions it is free. With 40,000 historical events, every Node process loads megabytes it never queries. Process memory is an expensive resource, shared with the event loop.
- No efficient queries. "Give me the sessions at Teatro Almendra between March 1st and 15th that still have tickets available" is solved today by walking the entire array in JavaScript. That is O(n) over the whole data set, on the main thread, with no way to improve it unless you write an index yourself.
- No integrity. Nothing stops an order from pointing at
ses-999-9, a session that does not exist. Nothing stopssoldfrom exceedingcapacity. The invariants live only in the domain classes, and if somebody edits the JSON by hand or a script bypasses the domain, the data is corrupt forever. - No history. The file only holds the current state. How many tickets were sold last Tuesday? Who cancelled the order? There is no answer: the previous state was overwritten.
- No access from multiple processes. This is the one that hurts most looking ahead. In Module 10 we will meet
clusterand start several Node processes to use every core on the machine. With a JSON file, each process would have its own in-memory cache and its own idea of capacity; they would not even find out about each other's sales. File persistence makes scaling impossible. And in Module 11, deploying with PM2 or with several container replicas, the problem is multiplied by the number of instances.
The conclusion is not that files are bad. A JSON file is perfect for configuration, for seed data, for exporting a report. It is bad as a store of shared, mutable, concurrent state. And that is precisely what a session's capacity is.
What a DBMS actually gives you
A database management system (DBMS) is a specialized process, usually on another machine or in another container, whose only job is to look after data. In exchange for learning it and operating it, it gives you six things you are not going to reimplement well:
| Guarantee | What it means | What it solves in Escena Viva |
|---|---|---|
| Durable persistence | A committed write survives a power cut, thanks to the write-ahead log (WAL/journal) | A sale already charged is never lost |
| Controlled concurrency | Locks and version control so N clients can write at once without corrupting anything | Two simultaneous buyers do not overwrite each other |
| Query language | Filtering, sorting, grouping and aggregating inside the engine, not in your process | Revenue reports without reading the whole catalog |
| Indexes | Auxiliary structures that turn O(n) searches into O(log n) | Searching by venue or by date range is instant |
| Integrity | Constraints the engine enforces no matter what | sold <= capacity as a law of physics, not as good intentions |
| Transactions | Several operations that all happen or none do | Reserving capacity + creating the order + issuing tickets as a single act |
Look at the last row. It is the one that solves the problem we have been dragging along, and we will devote half of lesson 07-06 to it.
Relational versus document
There are two big families a Node developer runs into every day. Neither is "modern" and the other "old"; neither is "serious" and the other "a toy". They are models with different trade-offs.
| Criterion | Relational (PostgreSQL, MySQL) | Document (MongoDB) |
|---|---|---|
| Unit of data | A row in a table, with fixed columns | A BSON document, much like a nested JSON object |
| Schema | Rigid and declared; changing it requires a migration | Flexible; the engine demands no shape, your ODM imposes one |
| Relationships | Foreign keys and JOINs resolved by the engine |
References resolved with extra queries, or embedded data |
| Referential integrity | Yes, guaranteed by the engine | Not across collections; you guarantee it |
| Transactions | Native, multi-table, from day one | Atomic per document; multi-document since 4.0 with replica sets |
| Complex queries | SQL, extremely expressive and optimized for decades | Aggregation framework, powerful but different |
| Scaling | Vertical plus read replicas; partitioning is more costly | Horizontal partitioning (sharding) out of the box |
| Ideal cases | Highly related data, reporting, money, hard invariants | Self-contained documents, changing schema, high read volume |
The healthy way to read this table: pick the model that looks like your access pattern. If your application almost always reads "one event with all its sessions", the document model gives you that in a single disk read. If your application almost always joins five entities to produce a report, the relational model gives you that in a single optimized query.
Escena Viva, curiously, has both faces: the catalog is document-shaped by nature (an event contains its sessions) and selling is relational by nature (orders, tickets, users and money). That is why this module does both: MongoDB will be the official persistence layer in lessons 2 to 4, and in lessons 5 and 6 we will model the same thing in PostgreSQL to see what changes — and discover that overselling is solved there in a particularly elegant way.
ACID, BASE and the CAP theorem applied to capacity
ACID describes the guarantees of a transaction. Atomicity: the transaction happens whole or it does not happen. Consistency: when it finishes, every schema rule still holds. Isolation: two simultaneous transactions never see each other half-done, and the result is as if they had run in some order. Durability: once committed, it survives a server failure. Relational systems were born with ACID and document systems have been adopting it. BASE is the opposite trade-off, typical of massively distributed systems: Basically Available (it always answers), Soft state (state may be in transit), Eventually consistent (if you stop writing, at some point every replica will agree). BASE trades immediate correctness for availability and scale.
The CAP theorem explains why that trade exists. In a distributed system where the network can partition (and it always can), you cannot keep strong consistency (C) and total availability (A) at the same time: when two nodes stop seeing each other, either you reject writes so as not to diverge, or you accept them and you will diverge. Since partition tolerance (P) is not optional on a real network, the practical decision is between CP (I reject doubtful writes and stay correct) and AP (I accept everything and reconcile later).
| ACID | BASE | |
|---|---|---|
| Prioritizes | Immediate correctness | Availability and scale |
| On a network partition | Rejects doubtful writes (CP) | Accepts them and reconciles later (AP) |
| State after writing | Final and visible to everyone | May take time to propagate |
| Example in Escena Viva | A session's sold, the order status |
View counter, recommendations |
Applied to Escena Viva: capacity demands strong guarantees. Selling one ticket too many is not a cosmetic detail that gets fixed "eventually"; it is a person holding code EV-2026-000431 standing at the door of Teatro Almendra with no seat, plus a refund, plus a complaint. The sold counter is the textbook example of data that needs immediate consistency: we would rather reject a doubtful purchase (CP) than accept it and discover afterwards that there was no room. Other data on the very same platform, by contrast, tolerates eventual consistency perfectly well: the view count on an event page, the recommendations, or the catalog cache we will build with Redis in Module 10. The guarantee is chosen per piece of data, not per application.
The database landscape for Node.js
| Engine | Family | Strength | When to choose it | npm package |
|---|---|---|---|---|
| PostgreSQL | Relational | The most complete: JSONB, rich types, solid transactions, extensions | The default when in doubt and money or hard invariants are involved | pg |
| MySQL / MariaDB | Relational | Enormous install base, well-known operations, very fast on simple reads | An ecosystem or hosting provider that already imposes it | mysql2 |
| SQLite | Embedded relational | No server: the database is a file | Desktop applications, CLIs, tests, prototypes | better-sqlite3 |
| MongoDB | Document | Nested documents, flexible schema, sharding built in | Self-contained data whose shape keeps changing | mongodb |
| Redis | In-memory key-value | Microsecond latency, data structures, TTL | Cache, sessions, queues — it belongs to Module 10, not a primary store | redis |
Redis deserves a clarification because it is widely misread: it is not "a faster database", it is an in-memory database with a different durability model. You put it in front of the primary database, not in its place. In Module 10 we will use it to cache the catalog and to queue confirmation emails.
Native driver versus ORM/ODM
Between your code and the engine there is always a driver: it speaks the database's binary protocol and exposes a low-level API. On top of it there may be an ORM (Object-Relational Mapper, the SQL world) or an ODM (Object-Document Mapper, the document world), which translates between rows or documents and JavaScript objects.
| Aspect | Native driver | ORM / ODM |
|---|---|---|
| Control over the query | Total, you write exactly what runs | Indirect: the library generates it |
| Peak performance | The engine's ceiling | Slightly less, because of the translation layer |
| Schema and validation | You write it | Declarative, included |
| Relationships | You resolve them by hand | populate / include |
| Migrations | Yours | Built-in tooling |
| Learning curve | You learn the engine | You learn the engine and the library |
| Risk | Repetitive code, manual mistakes | Inefficient queries you never see, "magic" that is hard to debug |
What an ORM/ODM gives you: less repetitive code, declarative validation, coherent types, hooks, migrations and one shared mental model for the whole team. What it takes away: transparency. One innocent line can generate 200 queries (the N+1 problem we will see in lesson 07-04). The professional rule is: use the ORM for 95% of the code and do not be afraid to drop down to raw queries for the 5% that matters, measuring before you decide. In this module we will use Mongoose (ODM) and Sequelize (ORM), and in both cases we will see how to escape to the raw query.
Designing Escena Viva's data model
Before writing a schema you have to decide what counts as an entity. An entity is something with its own identity and its own life cycle: it exists before and after the operation that created it, and it makes sense to look it up on its own. An event is an entity. An order is an entity. The venue's name, by contrast, is an attribute.
The second decision is what gets grouped together and what gets kept apart. In the document world this is called embedding versus referencing and we will study it thoroughly in lesson 07-04, but we can already apply the criterion with two questions:
- Are they always read together? If you never ask for a session without its event, group them.
- Does it grow without bound? If the child collection grows indefinitely, keep it apart.
Let's apply that to our entities:
Eventwith its sessions embedded. An event has between 1 and 7 sessions in our catalog; a play in a long run might have 60. That is a bounded number, known in advance. On top of that, the event detail screen and the API itself (GET /events/:id) always ask for them together: even today thelistEventSessionscontroller starts from a loadedEvent. And their size is tiny. They are embedded.Orderas a separate collection. Orders grow without a ceiling: a popular event can accumulate tens of thousands. They have their own life cycle (pending → paid → issued | cancelled) and are queried on their own ("my orders"). Putting them inside the event would grow the document until it burst the size limit, and would force rewriting it whole on every purchase. They are kept apart.Ticketas a separate collection. Every ticket has its unique codeEV-2026-000431, its status (valid → used | cancelled) and is validated individually at the door, by scanning. It is the smallest unit of access in the system and the one with the highest volume. It is kept apart, with references to the order and to the session.Useras a separate collection, with itsrole(attendee, organizer, administrator). In this module we create it with no passwords and no authentication: that is entirely Module 8. Here we only leave the slot ready.
And a third decision, deliberately uncomfortable: the sold counter lives inside the session, even though it could be computed by counting valid tickets. It is denormalization on purpose, so that checking capacity is a single field read instead of an aggregation. The price is keeping the counter coherent with the real tickets; we pay that price with the techniques of lesson 07-06.
The final model in a diagram
erDiagram
USER ||--o{ ORDER : "places"
EVENT ||--|{ SESSION : "contains (embedded)"
ORDER ||--|{ TICKET : "groups"
SESSION ||--o{ TICKET : "is reserved by"
USER {
string _id
string email
string name
string role
}
EVENT {
string _id
string eventId
string title
string venue
string organizerId
string category
string status
int durationMinutes
}
SESSION {
string sessionId
date dateTime
int capacity
int sold
int priceCents
}
ORDER {
string _id
string userId
string sessionId
int quantity
int totalCents
string status
string channel
}
TICKET {
string _id
string code
string orderId
string sessionId
string status
}
Read it like this: SESSION appears as an entity in the diagram because conceptually it is one, but physically it lives inside the EVENT document as a subdocument. ORDER and TICKET are independent collections related by reference. This is the blueprint we will implement in lesson 07-02 with Mongoose and translate into tables in 07-05 with Sequelize — where SESSION will be a table of its own, and we will see why.
Isolating persistence: the repository pattern
Here comes the most important architectural warning of the module. If controllers call Event.find(...) directly, your application is married to MongoDB forever. Changing engines, or testing with fake data, or querying two different sources, would mean rewriting every controller.
The solution is a thin layer: src/repositories/. A repository exposes operations in the language of the business (getCatalog, getEventById, reserveCapacity, createOrder) and hides completely how they are fulfilled. Above it, controllers and domain do not know whether Mongo, Postgres or an array sits underneath.
// src/repositories/events.js (the contract; the implementation arrives in lesson 07-03)
'use strict';
/**
* Returns the full catalog as domain instances.
* The caller does not know whether it comes from Mongo, from Postgres or from a file.
*/
async function getCatalog() {
throw new Error('not implemented');
}
/** Returns a domain Event, or null if it does not exist. */
async function getEventById(eventId) {
throw new Error('not implemented');
}
module.exports = { getCatalog, getEventById };Notice the decisive detail: it is exactly the public signature of src/catalog-data.js. That is why we will be able to swap the whole module out without touching src/controllers/events.js. And that is why, in lesson 07-05, we will write a second implementation on top of Sequelize without changing a single line of the application. The repository returns domain objects (Event, Session), not Mongoose documents: that way the domain keeps its getters (occupancy, soldOut) and its invariants, exactly as we left them in Module 2.
Installing MongoDB and checking the connection
Three routes, all valid. Pick one.
Option A: local service. Install MongoDB Community Server from the official site and run it as a system service:
# Check that the service is alive (Linux with systemd)
sudo systemctl status mongod
sudo systemctl start mongodOption B: Docker. The cleanest one, because it does not dirty your machine and it is removed with a single command. In Module 11 we will formalize this with docker compose:
# Start MongoDB 7 on the standard port, with a persistent volume
docker run -d --name mongo-escena-viva \
-p 27017:27017 \
-v escena-viva-data:/data/db \
mongo:7Option C: MongoDB Atlas. The managed service in the cloud. Its free tier is enough for this course; it will give you a URL like mongodb+srv://user:[email protected]/escena_viva. This is what you will use in production if you would rather not operate the engine yourself.
Whichever option you choose, check the connection with the official mongosh client:
And inside the shell:
// See which database we are on (it is created on the first write, not before)
db.getName(); // 'escena_viva'
// Insert a test document and read it back
db.test.insertOne({ venue: 'Teatro Almendra', createdAt: new Date() });
db.test.find();
// Clean up
db.test.drop();One quirk that surprises people: MongoDB does not create the database or the collection until the first document. If show dbs does not list escena_viva right after you connect, that is not an error.
The connection URL will never be written by hand in the code. It goes in .env and is read from src/config/index.js, already the only point in the application that touches process.env:
And with the connection verified, it pays to settle in advance what questions the application will ask, because a data model is designed backwards from the queries. These are Escena Viva's five most frequent queries, and they are the ones that will govern the indexes in lesson 07-02:
| Query | Frequency | Starting entity |
|---|---|---|
| Catalog of published events | Very high | Event |
| Detail of one event with its sessions | Very high | Event |
| Check the available capacity of a session | Very high (critical) | Session inside the Event |
| A user's orders | Medium | Order |
| Validate a ticket by its code at the door | High during show times | Ticket |
Common Mistakes and Tips
- Choosing the engine because it is fashionable. "We use Mongo because it's JavaScript" is not a criterion. The criteria are the access pattern, the guarantees you need and what your team knows how to operate.
- Believing that "schemaless" means "no design". MongoDB does not force you to declare a shape, but your data has one anyway. If you do not decide it, every
insertyou write will decide it for you, and you will end up with seven variants of the same document. - Putting the connection URL in the code. With credentials inside it, pushed to git. It goes in the configuration, always. You already have the place:
src/config/index.js. - Leaving the database with no authentication "because it's development". A MongoDB exposed on 27017 with no user is one of the favorite targets of automated scanners. Locally, at the very least do not publish the port outside
localhost. - Tip: before modeling, write down the five queries your application will run most often. A data model is designed backwards from the queries, not forwards from the entities.
- Tip: keep
data/events.json. We will not delete it: in lesson 07-06 it becomes the seed that loads the database. It retires as a store, not as a source.
Exercises
Exercise 1: audit your own code
Open src/catalog-data.js and src/controllers/orders.js and find, without running anything, the exact points where two simultaneous requests can corrupt the state. For each one, write down: which line reads, which line writes, and what happens if another request sneaks in between them.
Exercise 2: decide the model
Escena Viva wants to add two things: (a) the reviews attendees leave about an event, and (b) the seating plan of each venue. For each one, decide whether it is embedded or kept apart and justify the decision with this lesson's two criteria (read together and growth).
Exercise 3: choose guarantees
Classify these four pieces of data according to whether they need strong or eventual consistency, and justify it: (1) a session's sold, (2) the view counter on an event page, (3) the status of an order, (4) the list of "events recommended for you".
Solutions
Exercise 1. The dangerous pattern is always read-modify-write without mutual exclusion. In the purchase flow: the memoized catalog is read, session.available >= quantity is checked, session.sell() is called and the result is persisted. If two requests run the check before either of them writes, both pass it even when only one ticket is left. Memoization makes things worse: the in-memory copy can be minutes out of date with respect to the file. And because Node is single-threaded, the gap opens exactly at every await — the event loop from Module 2 yields its turn right there.
Exercise 2. (a) Reviews are kept apart: they grow without bound (a festival can accumulate thousands), they are not needed to render the catalog, they are paginated and sorted by date, and they are moderated with a life cycle of their own. (b) The seating plan is embedded in the venue (or referenced as a separate catalog if several venues share a template), because it is bounded in size, it changes almost never and it is always read together with the venue. Careful, though: the occupancy of each seat in a specific session is volatile and write-heavy, so that state must not live in the static plan.
Exercise 3. (1) sold: strong, it is the very definition of the overselling problem. (3) Order status: strong, it decides whether money is charged and whether tickets are issued; a stale read can double a charge. (2) Views: eventual, a 2% error for a few seconds harms nobody and in exchange lets you count in memory and flush in batches. (4) Recommendations: eventual, they are computed offline and nobody notices they are an hour behind.
Conclusion
We have closed the file chapter. You now know, with concrete names, what data/events.json lacks: atomicity, concurrency, queries, indexes, integrity, history and multi-process access. You know what a DBMS gives you in exchange, what separates the relational model from the document model without needing to take sides, and why capacity belongs to the territory of strong guarantees under ACID and CAP. You have seen the real landscape of engines for Node and the trade-off between dropping down to the native driver and leaning on an ORM/ODM. And, above all, you have Escena Viva's data model decided and reasoned out: Event with its sessions embedded, Order, Ticket and User as separate collections, with the sold counter denormalized on purpose.
You also have the architectural piece that holds up the whole module: src/repositories/, the boundary that will stop the database engine from leaking into the controllers and the domain.
In the next lesson we go down into the code. We will connect to MongoDB from src/config/index.js, write src/db/connection.js with its opening at startup and its closing in the graceful shutdown that already exists in src/server.js, and translate this diagram into Mongoose schemas and models in src/models/: types, validators, virtuals, schema middleware and indexes. The blueprint becomes structure.
Node.js Course: From Beginner to Advanced
Module 1: Introduction to Node.js
- What Is Node.js?
- Installing and Setting Up the Environment
- Your First Node.js Program
- The Node.js REPL
- Modern JavaScript for Node.js
- The Course Project: the Escena Viva Platform
Module 2: Core Concepts
- Node.js Architecture
- The Event Loop
- Callbacks and Asynchronous Programming
- Promises and async/await
- Events and EventEmitter
- CommonJS Modules and require()
- ES Modules and Interoperability
Module 3: File System and I/O
- Reading and Writing Files
- The fs Module in Depth
- Cross-Platform Paths with the path Module
- Working with Streams
- Transform Streams and pipeline
- Buffers and Binary Data
Module 4: HTTP and Web Servers
- Creating a Simple HTTP Server
- Handling Requests and Responses
- Manual Routing
- Serving Static Files
- Receiving Data: Request Bodies and JSON
- Consuming External APIs from Node.js
Module 5: NPM and Package Management
- Introduction to NPM and package.json
- Installing and Using Packages
- Semantic Versioning and package-lock
- npm Scripts and Project Automation
- Creating and Publishing Packages
- Dependency Security and Maintenance
Module 6: The Express.js Framework
- Introduction to Express.js
- Setting Up an Express Application
- Routing in Express
- Middleware
- Essential Third-Party Middleware
- Input Data Validation
- Error Handling
Module 7: Databases and ORMs
- Introduction to Databases
- Using MongoDB with Mongoose
- CRUD Operations
- Relationships, Population and Advanced Queries
- Using SQL Databases with Sequelize
- Migrations, Transactions and Seed Data
Module 8: Authentication and Authorization
- Introduction to Authentication
- User Registration and Password Hashing
- Sessions and Cookies with Passport.js
- Authentication with JWT
- Role-Based Access Control
- API Security Best Practices
Module 9: Testing and Debugging
- Introduction to Testing
- Unit Testing with Mocha and Chai
- Test Doubles with Sinon
- Integration Testing
- Coverage and Test Automation
- Debugging Node.js Applications
Module 10: Advanced Topics
- The Cluster Module
- Worker Threads
- Caching and Job Queues with Redis
- Performance Optimization
- Building RESTful APIs
- GraphQL with Node.js
Module 11: Deployment and DevOps
- Configuration and Environment Variables
- Logging and Monitoring in Production
- Using PM2 for Process Management
- Packaging with Docker
- Deploying to Heroku and Other PaaS
- Continuous Integration and Deployment
