You now command the recurrent machinery: cells with memory (LSTM/GRU), stacked and bidirectional layers. But one obstacle stands before applying it to TecnoMarket's reviews: neural networks only process numbers, and a review is text. This lesson builds that bridge — tokenization, vocabulary, padding and, above all, embeddings — and applies it to the flagship project of natural language processing (NLP): sentiment analysis. Following the course methodology, we'll first build a prototype with the public IMDB dataset (movie reviews) and then read it through the TecnoMarket lens: classifying customer reviews and feeding the customer-service dashboard. We'll close with an overview of other NLP tasks with RNNs and their limits, which motivate the evolution towards attention.

1. The problem: networks eat numbers, not words
2. Tokenization and vocabulary
3. Different lengths: padding and truncation
4. Embeddings: from indices to meaning
5. The Keras Embedding layer
6. Prototype: sentiment analysis on IMDB (Embedding + LSTM)
7. The TecnoMarket reading: the customer-service dashboard
8. Overview: other NLP tasks with RNNs and their limits

Everything you've trained so far received numbers: pixels between 0 and 255 (MNIST, 02-05), sales figures, synthetic sequences. "The battery dies fast" is not a tensor. The standard pipeline for turning text into network input has three stages:

Let's go stage by stage.

Tokenizing means chopping the text into units (tokens): most intuitively, words. Then a vocabulary is built: a dictionary assigning each token an integer index, normally ordered by frequency (low indices = the most common words).

Details that matter:

• Index 0 is reserved for padding and 1 for [UNK] (unknown): any word not in the vocabulary (typos, rare words) gets mapped to [UNK]. Capping the vocabulary (e.g., to the 10,000 most frequent words) is a trade-off: fewer parameters in exchange for losing rare words.
• Real tokenization also handles lowercasing, accents and punctuation (TextVectorization applies a default standardization). Modern models use subwords (splitting "unbreakable" into "un-break-able"), but the underlying idea is the same.

Reviews come in different lengths, but a tensor is rectangular: every row must be the same size. The solution is to fix a length L:

• Shorter sequences → get filled with zeros (padding).
• Longer sequences → get cut (truncation).

Choosing L is a trade-off: too short loses the end of long reviews (where the conclusion usually lives!); too long wastes compute on zeros. A practical rule: look at the 90th-95th percentile of the actual lengths in your corpus.

The indices [5, 214, 89, 47] are already numbers, but arbitrary numbers: the fact that "fridge" is 214 and "refrigerator" is 611 says nothing about their similarity. Two options for handing them to the network:

An embedding is a lookup table: a matrix of shape (vocabulary, dimension) where row i is the vector for word i. Those vectors start out random and are learned during training: if "broken" and "defective" appear in similar contexts and predict the same label, backpropagation pushes their vectors closer together — the blame assignment (02-03) reaches all the way down to each word's own representation.

Does this ring a bell? It's the same idea as in 03-04: there, a CNN turned each product image into a feature vector and we measured product similarity with cosine similarity. A word embedding is exactly that for language: a space where geometry encodes meaning. With well-trained embeddings, cosine(v_fridge, v_refrigerator) is high, cosine(v_fridge, v_rug) is low, and famous regularities show up, such as v_king − v_man + v_woman ≈ v_queen.

In Keras, the table is just another layer:

• Parameters: 10000 × 32 = 320,000. It's usually the biggest layer in the model — another reason to cap the vocabulary.
• Look at the output shape: (batch, steps, features) — exactly the 3D format the recurrent layers of 04-01 expect. Embedding and LSTM snap together like Lego bricks.
• mask_zero=True propagates a "mask" so the downstream layers don't process the padding zeros as if they were words.

Course methodology: prototype on a public dataset first. IMDB ships 50,000 movie reviews, labeled positive (1) or negative (0), already tokenized into indices — perfect for focusing on the model. It's a textbook many-to-one task.

How to read the results, as we did with MNIST in 02-05:

• The final sigmoid gives the probability of a positive review; with BCE as the loss, it's the standard binary-classification recipe from 02-04.
• Watch the curves: on IMDB, around epoch 3-4 the training accuracy keeps climbing but validation accuracy stalls or drops — the model is starting to memorize. Stop there (the systematic anti-overfitting techniques arrive in 05-04).
• 87% from such a small model is remarkable: chance would give 50%. The remaining errors are usually reviews with irony, complex negations or mixed sentiment — precisely long, subtle dependencies.

Second step of the methodology: carry it over to TecnoMarket's (fictional) data. The customer-service team receives hundreds of reviews and messages a day; the goal is to prioritize them automatically.

Changes with respect to the prototype:

1. Your own data: the store's real reviews with its own labels. Here the model goes from binary to three classes: positive, negative and urgent (mentions damage, fraud or legal deadlines). Only the model's head needs changing: Dense(3, activation="softmax") with a categorical_crossentropy loss — the activation/loss-per-problem-type guide from 02-02 and 02-04 tells you exactly what to touch.
2. A vocabulary of your own: learned with TextVectorization.adapt() over TecnoMarket's corpus ("doesn't cool", "cracked screen", "refund" need to be in the vocabulary).
3. Confidence threshold and human review queue, the same deployment architecture we set up for product photos in 03-04:

The principle is the same as in 03-04: the model automates the clear-cut cases and routes the doubtful ones to people. In customer service this is not optional: misclassifying an urgent review (a fridge dripping onto a power strip) costs far more than one extra manual review. That's why, beyond accuracy, what matters most here is not letting urgencies slip through, even at the cost of some false positives.

Sentiment analysis is many-to-one, but for years RNNs covered almost all of NLP:

The seq2seq scheme deserves two conceptual lines: an encoder (an LSTM) reads the entire source sentence and compresses it into its final state; a decoder (another LSTM) generates the target sentence word by word starting from that state. Elegant... and with an obvious bottleneck: the whole source sentence, whether 5 or 60 words long, must fit into a single state vector. With long sentences, the decoder "forgets" the beginning, no matter how LSTM it is. On top of that, recurrence forces word-by-word processing in order, with no parallelization.

The historical solution was to let the decoder "look back" at all the encoder positions and decide which one to attend to at each step: the attention mechanism, which ended up removing recurrence entirely and giving rise to transformers. We won't develop it here: it's the subject of lesson 05-05. Text generation with RNNs (writing character by character) isn't practiced now either: it's guided project 07-02.

• Learning the vocabulary (or the vectorizer) from test data. adapt() must run only on the training set; otherwise, information from the test set leaks into the model and the evaluation comes out inflated. It's the same data-leakage principle we'll see with scalers in 04-04.
• Truncating on the wrong side. pad_sequences truncates from the beginning by default (truncating='pre'). In reviews, the conclusion usually sits at the end, so that default is often the right one; but decide consciously, not by accident.
• An oversized vocabulary. 100,000 words × 128 dims = 12.8M parameters in the Embedding alone, most of them for words seen 2 or 3 times (impossible to learn well). Trim down to the frequent ones and let [UNK] absorb the rest.
• Forgetting mask_zero=True (or an equivalent) with heavy padding. If half of each sequence is zeros and the LSTM processes them as words, you dilute the signal. With the mask, the network skips them.
• Evaluating with accuracy alone on imbalanced classes. If 90% of reviews are positive, a model that always says "positive" scores 90% accuracy and is worthless. Look at the per-class confusion matrix, especially the recall of urgent.

1. Pipeline by hand: with the vocabulary {'': 0, '[UNK]': 1, 'product': 2, 'excellent': 3, 'broken': 4, 'arrived': 5} and fixed length L = 6, convert the review "the product arrived broken" into a sequence of indices (the word "the" is not in the vocabulary). What shape will the output have after an Embedding(6, 4)?
2. Parameter count: compute the total parameters of the lesson's IMDB model (Embedding 10,000×32, LSTM(64) with a 32-dim input, Dense(1) over 64). What percentage belongs to the Embedding?
3. TecnoMarket design: the dashboard classifies into positive/negative/urgent with a 0.80 threshold. For review X the model returns [0.42, 0.45, 0.13]. What does the route_review function do, and why is that the correct behavior?

1. Tokens: ['the', 'product', 'arrived', 'broken'] → indices [1, 2, 5, 4] ("the" → [UNK] = 1). Padded to 6 (filled at the end): [1, 2, 5, 4, 0, 0]. After Embedding(6, 4): shape (6, 4) — 6 time steps with a 4-number vector each (with batch: (1, 6, 4)).
2. Embedding: 10,000 × 32 = 320,000. LSTM: 4·64·(32+64+1) = 24,832. Dense: 64 + 1 = 65. Total: 344,897. The Embedding accounts for 320,000 / 344,897 ≈ 93% of the parameters — which is why vocabulary size dominates model size.
3. The most probable class is negative (0.45), but the maximum confidence (0.45) falls well below the 0.80 threshold, so it returns destination: human_review with suggestion: negative. That's correct: the model is essentially undecided between positive and negative (0.42 vs. 0.45) — probably a mixed or ironic review — and deciding automatically would be a coin toss; a person resolves the case in seconds.

You've closed the bridge between language and networks: tokenization and vocabulary to chop and index, padding to make things rectangular, and embeddings — learned dense vectors where geometric closeness captures semantic similarity, like the image embeddings of 03-04 — to give the indices meaning. With Embedding + LSTM you've built an 87% sentiment classifier on IMDB and carried it over to TecnoMarket's dashboard with three classes and the confidence-threshold-plus-human-review setup from 03-04. You've also seen the map of NLP tasks (NER, seq2seq for translation and summarization) and the single-vector bottleneck that motivated attention and transformers (05-05). One flagship project of the module remains: the TecnoMarket sequences that aren't words but dated numbers — forecasting daily demand, where time series, sliding windows and a classic mistake or two we'll learn to dodge are waiting for us.