The data/backlog.js module you closed the previous lesson with holds a lie inside: it returns the six tasks instantly, because they are hand-written in the file itself. In the real application that data will live on a server, and asking for it will take anywhere between fifty milliseconds and several seconds, depending on the network. The question that opens this block of the course is what the program does in the meantime. And the obvious answer —"wait"— turns out to be catastrophic in a language that runs one thing at a time: while it waits, it cannot do anything else at all, so the page freezes. In this lesson you will understand why JavaScript needs asynchrony, you will learn the oldest tools for handling it —setTimeout, setInterval and the callback pattern—, you will simulate loading the backlog with latency, and you will arrive under your own steam at the famous callback hell, the pyramid of code that motivated the invention of promises.

1. A single thread: what it means and why it matters
2. Synchronous versus asynchronous, on a timeline
3. setTimeout: scheduling for later
4. setInterval, clearTimeout and clearInterval
5. Why setTimeout(f, 0) is not immediate
6. The callback pattern
7. The error-first convention
8. Simulating the backlog load
9. Chaining three operations: the pyramid
10. The four problems with callbacks
11. try/catch does not catch asynchronous errors
12. Partial mitigations and why they are not enough
13. Common Mistakes and Tips
14. Exercises
15. Conclusion

JavaScript is a single-threaded language: there is one call stack —the one you studied in 03-05— and exactly one function runs on it at any given instant. There are never two pieces of your code running at once.

That has an enormous advantage, best appreciated if you have suffered through other languages: you never have to worry that another thread will change a variable halfway through a function. The task array cannot be modified while your reduce walks it. The entire class of bugs called "race conditions over shared memory" simply does not exist.

And it has an equally large drawback: if one function takes a long time, everything else waits. In the browser, that "everything else" includes repainting the screen, responding to clicks and animating anything at all.

During those three seconds the tab is dead: buttons do not respond, text cannot be selected and, if the browser decides enough is enough, the "page is not responding" warning appears. A user gives up long before that.

The solution is not to add threads, but to not wait: ask for the data, say what to do when it arrives, and hand control back immediately so the program stays alive. That is asynchronous programming.

Compare the two styles on the same problem: loading the backlog and printing the summary.

That change of order is the first thing that throws people. The output of the second block is:

Because loadBacklog does not wait: it registers the function you pass it, hands control back immediately and the program carries on. When the data is available —ten, a hundred or a thousand milliseconds later— the registered function runs.

In the asynchronous version the data arrives at the same instant —the network takes as long as it takes— but during those three seconds the thread is free: the interface responds, animations run, other data can be requested in parallel.

An important nuance that is often explained badly: asynchrony does not make anything faster. What it does is avoid wasting the thread while waiting for something that does not depend on it. Somebody else does the waiting: the browser's networking system, the operating system's timer, the disk. Your code merely signs up to be told.

The simplest form of asynchronous code is setTimeout, which registers a function to run after a certain time.

Its full signature:

• fn: what to run. It is a callback: a function you write but somebody else calls.
• milliseconds: the minimum delay. If omitted, it is 0.
• arg1, arg2…: arguments that will be passed to the callback.
• Returns an identifier, which is used to cancel it.

An alternative you will see a lot and should avoid: passing the arguments through an arrow that captures them.

The second line is the classic mistake: the parentheses call the function, so setTimeout receives the returned value (usually undefined) instead of a function. It is the same slip from 03-02 when passing functions as arguments.

setInterval repeats the callback every N milliseconds, indefinitely, until it is cancelled.

And clearTimeout cancels a setTimeout that has not fired yet:

A warning about setInterval that is expensive to discover in production: it does not wait for the callback to finish. If you schedule an interval of 100 ms and the callback takes 150 ms, the executions overlap and pile up. For work of variable duration —querying a server, for example— a setTimeout that reschedules itself once it is done is safer:

And an operational rule that will save you memory leaks (Module 9): always keep the identifier and cancel when the work stops making sense. A forgotten interval keeps running, draining battery and keeping alive by reference every variable its callback captures.

This is one of the most misunderstood lines in the language:

With a delay of zero milliseconds, B still comes out last. The reason is that the second argument is not "when it will run", but "the minimum time that must pass before it is a candidate to run". And to be a candidate something else is needed: the call stack has to be empty.

The complete mechanism, in three steps:

1. setTimeout hands the callback to the environment (the browser or Node), which starts a timer. This does not occupy the thread.
2. Once the time is up, the environment places the callback in a queue of pending tasks.
3. When the code currently running finishes completely and the stack is empty, the first one in the queue is taken and run.

That is why a setTimeout(f, 0) behind a heavy loop does not run until the loop is over:

There is also a historical detail: by specification, timers nested beyond a certain depth are raised to a minimum of 4 ms in browsers. So setTimeout(f, 0) really means "as soon as possible, and at the earliest in a few milliseconds, when the thread is free".

That use —"run this after what I am doing now"— is perfectly legitimate and is called yielding the thread. The whole queueing mechanism you have just glimpsed has a name of its own, a precise algorithm and several more subtleties, and it is the full subject of The Event Loop and the Microtask Queue. For now the rule is enough: all the synchronous code finishes first; then the pending callbacks.

A callback is a function you pass to another one so it can call it later. You have known the idea since 03-06: it is exactly what you did with map, filter and sort. The difference is when it gets called.

An asynchronous callback is recognized by an unmistakable sign: the function that receives it does not return the result. Compare:

From that comes the rule that governs this whole lesson:

A value produced asynchronously cannot be returned with return. It can only be handed to somebody who is waiting to receive it.

And from that also comes the most frequent beginner's mistake:

There is no way of fixing that function while keeping its signature. The value does not exist yet when the return runs. What has to change is the design, not the code.

If the result travels through the callback, errors have to travel that way too. Node.js established a convention that was adopted everywhere: the callback receives the error as its first argument and the result as its second.

• If everything went well: callback(null, data).
• If something failed: callback(new Error('…')), with no second argument.

Three details to observe religiously:

1. Check the error first. If you skip that if, task will be undefined and the error will turn into a confusing TypeError three lines further down.
2. return after calling the callback with an error. It is the same guard return you learned in 02-01: without it, the function carries on and may end up calling the callback twice, which breaks whoever consumes it.
3. Call the callback exactly once. Not zero times (the consumer waits forever, with no error at all) and not twice.

With all of the above, we can now replace the instant createBacklog() from 05-04 with a version that behaves like the real world: with latency and with the possibility of failure.

And its use from app.js:

Notice the order: the "keeps responding" message comes out before the data. That is the proof that the thread was left free. And note a design detail too: the try/catch is inside the setTimeout, wrapping the construction of the tasks, and it turns any ValidationError from the constructor into a callback call with an error. Why it has to be in there is what you will see in section 11.

Progress reminder: readSimulatedBacklog fakes the network with setTimeout. The real API —fetch, HTTP status codes, headers and everything else— arrives in 07-02, and the robust way of handling its failures and timeouts, in 07-03. What matters here is the shape of asynchronous code, not where the data comes from.

So far, callbacks look reasonable. The problem shows up when one asynchronous operation depends on the result of another. Marta asks for a weekly report that requires three steps, and each one needs what the previous one produced:

1. Load the backlog.
2. With the assignees that appear, load their contracted hours.
3. With both, generate the report and save it.

We add the two missing simulated modules:

And now the report, written with callbacks:

The output is correct:

But look at the shape of the code. Each operation nests the next one a level deeper, and the final closing is that staircase of }); that has earned itself a nickname:

This is called callback hell or the pyramid of doom. With three steps it is still readable; with six —load, validate, enrich, calculate, save, notify— it is illegible. And these are only three sequential steps: if on top of that you had to do two things in parallel and wait for both, you would have to invent a manual counter.

The indentation is what you see, but it is not the worst of it. There are four real problems.

Problem 1: repeated error handling. Count the if (error) { console.error(...); return; } blocks in the previous example: three, one per level, practically identical. There is no way of writing "if anything fails at any point in this sequence, do this". Each level defends itself alone, and it only takes forgetting one if for a failure to go unnoticed and blow up later with a message that has nothing to do with it.

Problem 2: reading order is not execution order. The code reads top to bottom but runs in a dance of jumps. What happens after saveSimulatedReport is inside it, and what happens after everything is at the deepest level. Our brains read sequences; this is a tree.

Problem 3: inversion of control. When you write saveSimulatedReport(report, myCallback), you are handing your function to a third party. From then on, you control nothing:

With a function of your own it does not happen; with a third-party library, all of that happens for real. And you have no tool to protect yourself, other than reading their code.

Problem 4: composing is impossible. With ordinary functions, combining is trivial: pipe(filter, sort, summarize), as you did in 03-06. With callbacks there is nothing equivalent. Operations as common as "do these three things at once and tell me when they have all finished" or "retry this three times if it fails" require hand-writing counters, flags and checks:

Four state variables and a helper function to express an idea that is half a line long. In the next lesson this will be Promise.all([loadA(), loadB()]).

In 02-05 we noted a warning you can now fully understand: try/catch does not catch errors that happen inside an asynchronous callback.

The reason is exactly the one from section 5, and now it fits with what you know about the call stack from 03-05. try/catch protects a region of the stack: it catches whatever is thrown while those lines are running. When the callback runs, a hundred milliseconds later, the stack that contained the try no longer exists; the callback starts on a fresh, empty stack, with no try around it.

The only way of catching it with callbacks is putting the try/catch inside the callback, which is exactly what we did in readSimulatedBacklog:

And here is the most important design consequence of the whole lesson: with callbacks, errors do not propagate by themselves. In the synchronous code of 02-05, a throw at the bottom of ten calls climbed up the stack to the first try/catch, without the intermediate functions doing anything at all. With callbacks, every level has to catch its error and hand it up to the callback above by hand. The language's automatic propagation mechanism, one of its best features, stops working the moment you cross an asynchronous boundary.

There are techniques for easing the pyramid. They are worth knowing because old code uses them, and because understanding their limits explains why a new mechanism was needed.

Mitigation 1: naming the functions and flattening. Instead of nesting anonymous arrows, you declare named functions and pass them by reference.

It is better: the indentation is gone, every function has a name and the error traces from 03-05 are readable. But look at the price:

• Shared variables have appeared (currentBoard, currentWorkload) to pass data between steps. It is global state reintroduced through the back door, with all the problems you learned to avoid in 03-04.
• The sequence is no longer readable anywhere. To know what happens after the backlog loads you have to find the last line of onBacklogLoaded, go to onTeamLoaded, and so on. The order is scattered across the file.
• Error handling is still repeated in every function, even though it now delegates to abort.

Mitigation 2: modularizing. Putting each step in its own module (05-04) improves organization, but changes none of the above: the problems are about the shape of the control flow, not about where the code lives.

Mitigation 3: flow-control libraries. Around 2012, libraries like async became popular, with functions such as waterfall, series and parallel:

They worked, but they were an external convention: you had to learn them, they were not part of the language, and the inversion of control of problem 3 was still intact, because you were still handing your functions to a third party.

No mitigation touches the two underlying problems:

What is needed is for an asynchronous operation to return something: an object representing "the result that will arrive", which can be stored in a variable, passed to a function, chained and combined, and which propagates errors automatically the way the synchronous stack did. That object exists, it is called a promise, and it is the subject of the next lesson.

• Trying to return from an asynchronous callback. The value does not exist when the return runs. If you find yourself writing let result; setTimeout(() => result = …); return result;, stop: the design has to change, not the code.
• Calling the function instead of passing it. setTimeout(notify(), 100) runs notify immediately. You pass notify or () => notify(...).
• Forgetting the return after calling the callback with an error. The function carries on and ends up calling the callback a second time with incomplete data. It is a devilish bug to debug.
• Not checking the error, or checking it after using the result. The first line of the callback must be if (error) { …; return; }.
• Wrapping an asynchronous call in try/catch expecting to catch something. It does not work: the catch belongs to a stack that no longer exists. The try goes inside the callback.
• Forgetting clearInterval. An orphaned interval runs forever, drains battery and keeps alive by reference every variable it captures: it is a textbook memory leak (Module 9).
• Trusting the exact value of the delay. setTimeout(f, 100) means "not before 100 ms", never "exactly at 100 ms". To measure real time, use marks with Date.now() rather than counting timers.
• Loops with timers inside. for (let i = 0; i < 3; i++) setTimeout(() => console.log(i), 0) prints 0 1 2 with let but 3 3 3 with var: it is exactly the block-scoping case from 03-04, now with asynchronous consequences.
• Tip: name callbacks after what happens, not after what they do: onBacklogLoaded, onSaveFailed. When they turn up in an error trace at three in the morning, you will be grateful.

Exercise 1 — Predict the output. Without running it, write the exact order in which the six lines appear and justify each one.

Exercise 2 — countTasksByAssigneeSimulated. Write a function following the error-first convention that takes (assignee, callback) and, after a simulated 250 ms, delivers { assignee, open, hours } with the data from the canonical backlog. It must fail with DataError if the assignee does not exist or is not a string. Try it with 'Iván' (it should give 3 open tasks and 25 h), with 'Nobody' and with 42.

Exercise 3 — Retries with callbacks. Write withRetries(operation, attempts, callback) that runs an error-first asynchronous operation and, if it fails, retries it up to attempts times before giving up, waiting 200 ms between attempts. Try it with an operation that fails the first two times and works on the third. Then answer: how many lines did it take you, and which part is retry logic and which is callback plumbing?

Exercise 1

The interesting point is the order between 2 and 3.2. Both have a delay of 0, and when two timers come due at the same time they run in the order they were registered, so in practice you will see 2 before 3.2:

Two lessons. The first: all the synchronous code goes first, without exception; 4 comes out before any timer even though the timers have zero delay. The second: callbacks do not come out in the order they were written, but by due time; the loop registers the decreasing delays 100, 50 and 0, so the output runs the opposite way round to the loop. And notice that i holds the right value in each one thanks to let, which creates a new variable per iteration (03-04); with var you would have got three 3.3s.

Exercise 2

Iván's 25 h are the canonical ones: 12 for the multipurpose room, 8 for the bookbinding guide and 5 for the carpentry workshop quote. Notice the three calls in a row: all three are registered immediately, without any of them waiting for the previous one, and their results arrive at around 250 ms more or less together. That is parallelism of asynchronous operations, for free, with no coordination at all. The problem —as you saw in section 10— appears when you need to know when they have all finished.

Exercise 3

Answering the question in the statement: it is about twenty lines, and only three of them express the retry idea (remaining -= 1, the exhaustion check and the setTimeout(attempt, 200)). Everything else is plumbing: the inner attempt function that exists only so it can repeat itself, the manual error check, the guard returns after every callback call and the closing of the nesting. And there is a hidden fragility: if operation called its callback twice —problem 3 from section 10—, withRetries would call its own twice, and nothing in this code prevents it.

Keep this exercise. When you rewrite it with promises and async/await in the next lesson, the retry logic will fit in a five-line for with an ordinary try/catch, and the double-call problem will disappear by construction.

You have understood the underlying problem and the first-generation tools for solving it. JavaScript has a single thread, which spares it a whole category of concurrency bugs but imposes an implacable rule: while one function is running, nothing else can be. Blocking for three seconds does not mean "taking three seconds", it means freezing the entire application for three seconds. The way out is not waiting better, but not waiting at all: ask for the data, declare what to do when it arrives, and hand back control immediately. That is why the output of an asynchronous program is disconcerting at first —'Done' appears before the data— and that is why that inversion of order is precisely the sign that the thread has been left free.

You have the basic pieces down. setTimeout to schedule a future execution, with its identifier and its clearTimeout; setInterval to repeat, with the warning that it does not wait for the callback to finish and that a forgotten interval is a memory leak; and the explanation of why setTimeout(f, 0) is not immediate: the delay is a minimum, and on top of that the callback has to wait for the call stack to empty. You can recognize an asynchronous callback by an unmistakable sign —the function that receives it does not return the result— and from there the rule that governs everything: a value produced asynchronously cannot be returned with return, only handed over. And you know Node's error-first convention, (error, result), with its three disciplines: check the error first, put a return after every callback call, and call it exactly once.

With that you have built readSimulatedBacklog, readSimulatedTeamHours and saveSimulatedReport, you have loaded the canonical backlog with faked latency —48 h, 45 open, 1 overdue, effort 124, with the application responding meanwhile— and you have chained the three steps of Marta's weekly report until you saw the pyramid with your own eyes. And you have diagnosed why it hurts, which is the important part: error handling is repeated at every level, reading order stops matching execution order, handing your function to a third party is an inversion of control with no guarantees —it may not call it, may call it twice or may swallow its exceptions—, and composing operations (two in parallel, a retry, a maximum time) demands inventing counters and flags by hand. Above all, you have confirmed the warning left pending in 02-05: try/catch does not catch asynchronous errors, because by the time the callback runs the stack that contained the try no longer exists. Automatic error propagation, one of the language's best features, stops working the moment you cross an asynchronous boundary.

The mitigations —naming the functions, flattening the pyramid, modularizing, using flow-control libraries— fix the indentation and little else, and on top of that they reintroduce shared variables for passing data between steps. The problem was not cosmetic. What is missing is for an asynchronous operation to return a value: an object representing the future result, which can be stored in a variable, passed to a function, chained without nesting, combined in parallel and, above all, which propagates errors by itself the way the synchronous stack did. That object has existed since ES2015, it is called a promise, and with the async/await syntax built on top of it, it lets you write asynchronous code that reads exactly like synchronous code —try/catch included. It is the subject of Promises and Async/Await, where you will write Marta's weekly report again and see how much code disappears.