The correlation in the previous lesson told us that floor area and sales go strongly hand in hand (r = 0.93), but Marta wants more: "We are considering opening a 1,500 m² store in Alicante. How much will it sell?" To move from "they are related" to "if x is this much, y will be that much" we need simple linear regression: the technique that fits the best possible line to the scatter plot and turns it into a prediction machine. In this lesson we will set up the model, compute its coefficients by hand using least squares (reusing the worksheet from the previous lesson), interpret the slope and intercept in euros, measure the quality of the fit with R², examine the residuals, make the prediction for Alicante — and learn why predicting outside the range of the data is playing with fire. We will close with inference on the slope and a preview of multiple regression.

1. The simple linear regression model
2. Least squares: the best possible line
3. Computing NovaMarket's line by hand
4. Interpreting slope and intercept in business units
5. How good is the fit? The coefficient of determination R²
6. Residuals: the model's visual check-up
7. Prediction: interpolation yes, extrapolation with great care
8. Inference on the slope: CI and hypothesis test
9. Preview: when one variable is not enough (multiple regression)

Regression takes a step that correlation did not: it designates one variable as explanatory (or independent, \( x \): floor area) and another as the response (or dependent, \( y \): sales), and proposes a model for the population:

\[ y = \beta_0 + \beta_1 x + \varepsilon \]

• \( \beta_0 \) (intercept): the mean value of \( y \) when \( x = 0 \).
• \( \beta_1 \) (slope): how much \( y \) changes, on average, for each additional unit of \( x \). It is the star parameter.
• \( \varepsilon \) (error term): everything that affects sales and is not floor area — location, competition, the manager's skill, chance. The model does not claim that the points fall exactly on the line; it claims that the line captures the average trend and that \( \varepsilon \) absorbs the deviations.

As always in inference, \( \beta_0 \) and \( \beta_1 \) are unknown population parameters; from the sample we will compute their estimates \( b_0 \) and \( b_1 \), and the fitted line will be \( \hat{y} = b_0 + b_1 x \) (the "hat" on \( \hat{y} \) marks a predicted value, not an observed one).

Infinitely many lines could pass through the cloud of eight stores. Which one is "the best"? The universal criterion is least squares: for each store, the line's error is the residual \( e_i = y_i - \hat{y}_i \) (observed minus predicted, the vertical distance from the point to the line). We choose the line that minimizes the sum of the squared residuals:

\[ \min \sum_{i=1}^{n} e_i^2 = \min \sum_{i=1}^{n} \left( y_i - b_0 - b_1 x_i \right)^2 \]

(We square for the same reason as in the variance: so that positive and negative errors do not cancel out, and to penalize large errors more heavily.) Solving that minimization problem yields two closed-form formulas:

\[ b_1 = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2} \qquad\qquad b_0 = \bar{y} - b_1 \bar{x} \]

Look at the first one: the numerator is the same as in the covariance and the correlation. In fact, slope and correlation are close relatives (\( b_1 = r \cdot s_y/s_x \)): the slope is "the correlation expressed in the units of the problem". And the second formula says something elegant: the least-squares line always passes through the point of means \( (\bar{x}, \bar{y}) \).

Here we harvest the work from the previous lesson. From that worksheet we already have everything we need (sales in € thousands):

\[ \bar{x} = 1{,}281.25 \text{ m²} \quad \bar{y} = 45.7 \quad \sum (x_i-\bar{x})(y_i-\bar{y}) = 29{,}210 \quad \sum (x_i-\bar{x})^2 = 774{,}687.5 \]

Slope:

\[ b_1 = \frac{29{,}210}{774{,}687.5} = 0.0377 ; \text{€ thousands per m²} \]

Intercept:

\[ b_0 = 45.7 - 0.0377 \times 1{,}281.25 = 45.7 - 48.30 = -2.60 ; \text{€ thousands} \]

Fitted line (with sales in thousands of euros and floor area in m²):

\[ \hat{y} = -2.60 + 0.0377,x \]

This is the part Marta wants to hear, and it must be said precisely:

• Slope \( b_1 = 0.0377 \) € thousands/m² = €37.70 per m²: for each additional square meter of sales floor, average daily sales rise by about €37.70. In decision terms: enlarging a store by 100 m² is associated, on average, with about €3,770 more in daily sales. Two mandatory cautions: it is an average effect (no individual store will obey the exact figure) and it is an association, not proven causation — remember the lurking variable "location" from the previous lesson.
• Intercept \( b_0 = -2.60 \) € thousands: literally, "a 0 m² store would sell −€2,600 a day". That is obviously absurd, and that's fine: here the intercept is merely a mathematical anchor for the line. It only carries a business interpretation when \( x = 0 \) is a possible, observed value (0 m² stores do not exist, and the smallest one in the sample has 850 m²). It is our first warning about extrapolation.

The line is the best possible one, but is it any good? The standard measure is the coefficient of determination \( R^2 \), which comes from decomposing the variability of sales:

\[ \underbrace{\sum (y_i-\bar{y})^2}_{\text{Total SS}} = \underbrace{\sum (\hat{y}i-\bar{y})^2}{\text{SS explained by the line}} + \underbrace{\sum (y_i-\hat{y}i)^2}{\text{Residual SS}} \qquad R^2 = \frac{\text{Explained SS}}{\text{Total SS}} \]

\( R^2 \) is the proportion of the variance of y that the line explains, ranging from 0 (the line adds nothing) to 1 (perfect fit). In simple linear regression there is a lovely shortcut: \( R^2 = r^2 \), the square of the Pearson correlation. With our data:

\[ R^2 = 0.93^2 = 0.865 \]

Floor area explains 86.5% of the between-store variability in sales; the remaining 13.5% is down to everything else (location, management, competition…). The sums of squares tell the same story: Total SS = 1,272.84 (the sum of \( (y_i-\bar{y})^2 \) from the worksheet), Explained SS = \( b_1 \times 29{,}210 = 1{,}101.2 \) and Residual SS = 171.6, and indeed \( 1{,}101.2/1{,}272.84 = 0.865 \). There is no universal threshold for a "good R²": industrial processes demand very high values; in customer behavior, a 0.3 can be gold. File this decomposition of variability into "explained + residual" away in your memory: it is exactly the machinery that ANOVA will reuse.

Residuals are the model's magnifying glass: if the line is adequate, they should look like structureless noise. Let's compute them:

(Least-squares residuals always sum to zero — another arithmetic check.) Two readings:

• Business reading: Cuenca sells €10,200 a day less than its floor area predicts — by far the largest residual. It is not just "the small store": it is a store that underperforms even for its size. Marta has an investigation pending there (location? assortment? local competition?). Residuals turn regression into a benchmarking tool: they compare each store against what is expectable given its characteristics, not against the raw average.
• Statistical reading (visual diagnostics). Plotting the residuals against \( \hat{y} \) (or against x) we should see a random horizontal band around zero. Warning signs:

In our case, seven moderate residuals and Cuenca's standing out: with n = 8 that is not enough for a firm verdict, but it deserves watching.

At last, Marta's question: the candidate store in Alicante would have 1,500 m².

\[ \hat{y} = -2.60 + 0.0377 \times 1{,}500 = -2.60 + 56.55 = 53.95 ; \text{€ thousands} \approx \text{€}53{,}900 ; \text{in daily sales} \]

This prediction is an interpolation: 1,500 m² lies within the observed range (850–2,000 m²), where the line was "trained" and the fit is good. Even so, communicate it with humility: it is the expected average sales for a store of that size; the actual figure will come with the model's error attached (the residual standard error is \( s_e = \sqrt{171.6/6} = 5.35 \) € thousands, so deviations of ±€5,000–10,000 a day from the prediction are perfectly normal, as Cuenca's own residual shows).

Extrapolation is another story. Senior management also wants to assess a 5,000 m² hypermarket: the line would say \( \hat{y} = -2.60 + 0.0377 \times 5{,}000 = 185.9 \) € thousands (~€185,900/day). That figure is not reliable: 5,000 m² lies 2.5 times beyond the largest store observed, and nothing guarantees the relationship stays linear out there (hypermarkets run a different commercial model, a different turnover per m², wider aisles that sell nothing…). The line only describes the world it came from. Practical rule: predict within the range of the data; outside it, your line is an untested hypothesis, not a prediction. The negative intercept (\( x = 0 \Rightarrow \hat{y} = -2.6 \)) was already the perfect caricature of this danger.

Like everything computed from a sample, \( b_1 = 0.0377 \) carries sampling error: another 8 stores would yield a different slope. Could the population slope \( \beta_1 \) be 0 (floor area contributes nothing)? The test is the usual one from Module 5:

• \( H_0: \beta_1 = 0 \) — floor area does not help predict sales
• \( H_1: \beta_1 \neq 0 \)

The statistic uses the standard error of the slope, \( \text{SE}(b_1) = s_e / \sqrt{\sum (x_i-\bar{x})^2} \), where \( s_e = \sqrt{\text{Residual SS}/(n-2)} \):

\[ \text{SE}(b_1) = \frac{5.35}{\sqrt{774{,}687.5}} = \frac{5.35}{880.2} = 0.00608 \qquad t = \frac{b_1 - 0}{\text{SE}(b_1)} = \frac{0.0377}{0.00608} = 6.20 \]

With \( n-2 = 6 \) degrees of freedom, the two-tailed 5% critical value is 2.447: we reject \( H_0 \) comfortably. (It is no coincidence that we get the same t ≈ 6.2 as in the test on r in the previous lesson: in simple regression they are the same test.) The 95% confidence interval for the slope:

\[ b_1 \pm t_{0.025;,6} \cdot \text{SE}(b_1) = 0.0377 \pm 2.447 \times 0.00608 = 0.0377 \pm 0.0149 ;\Rightarrow; [0.0228 ;; 0.0526] \]

In business units: each additional m² is associated with between €22.80 and €52.60 more in daily sales, with 95% confidence. The interval is wide — we only have 8 stores —, but it does not contain 0: the effect is real. To evaluate the Alicante investment, Marta should work with that whole range, not just the central €37.70.

Our model leaves 13.5% of the variability unexplained, and we know that location, foot traffic and the area's income live there. The natural extension is multiple regression: a model with several explanatory variables at once, \( y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \varepsilon \), where each slope measures the effect of its variable holding the others constant — precisely the tool that lets us separate the effect of floor area from that of location, attacking the lurking-variable problem head on. We will develop it in Multivariate Analysis; for now it is enough to know that it exists and what it promises.

• Swapping x and y. Unlike correlation, regression is not symmetric: the line that predicts sales from floor area is not the same as the one that would predict floor area from sales. Decide what you want to predict before you compute.
• Forcing an interpretation onto the intercept. If \( x = 0 \) is not a possible value or falls outside the range of the data, \( b_0 \) is just a mathematical anchor. Do not present "−€2,600" as a business figure.
• Extrapolating without saying so. Predictions outside the observed range (the 5,000 m² hypermarket) inherit the shape of the line, not reality. Always state the model's range of validity.
• Reading the slope as causation. "+€37.70 per m²" is an average association in observational data; only an experiment or a model that controls for confounders justifies causal language.
• Stopping at R² without looking at the residuals. A high R² can coexist with clear curvature or with a single outlier propping up the whole fit (remember Anscombe: the four datasets also share the same line and the same R²). The residual plot is the quality control.
• Forgetting the uncertainty of the prediction. \( \hat{y} \) is an expected average: accompany it with the residual standard error or an interval, especially if it backs an investment.

Exercise 1. Using the lesson's line (\( \hat{y} = -2.60 + 0.0377x \), in € thousands): (a) predict the daily sales of a 1,000 m² store and of a 1,900 m² one, and say whether each is an interpolation or an extrapolation; (b) the expansion team asks for the prediction for a 400 m² unit in an "ultra-convenience" format: give the figure the line produces and explain why you should not hand it over without reservations.

Exercise 2. Fitting the relationship between local advertising spend (x, € thousands/month) and online sales in the area (y, € thousands/month) across 10 areas, an analyst obtains \( b_0 = 12.4 \), \( b_1 = 3.2 \), \( \text{SE}(b_1) = 1.25 \) and \( R^2 = 0.45 \). (a) Interpret \( b_1 \) and \( R^2 \) in business language. (b) Test at the 5% level whether advertising adds predictive power (\( t_{0.025;,8} = 2.306 \)) and build the 95% CI for \( \beta_1 \).

Exercise 3. A colleague presents a regression with R² = 0.91 whose residual plot shows a clear funnel: small residuals for stores with low predictions and huge ones for those with high predictions. He claims: "with this R², the model is excellent and the predictions are equally reliable for every store". Point out which part of the claim is defensible and which is not.

Exercise 1. (a) For 1,000 m²: \( \hat{y} = -2.60 + 37.7 = 35.1 \) € thousands ≈ €35,100/day. For 1,900 m²: \( \hat{y} = -2.60 + 71.63 = 69.0 \) € thousands ≈ €69,000/day. Both are interpolations: 1,000 and 1,900 m² fall within the observed range (850–2,000). (b) For 400 m² the line gives \( \hat{y} = -2.60 + 15.08 = 12.5 \) € thousands ≈ €12,500/day, but it is an extrapolation: it lies well below the smallest store in the sample (850 m²), and an ultra-convenience format runs on a different commercial logic (turnover, assortment, transaction value). The figure should be presented as an unvalidated indication, ideally supplemented with data from comparable stores. Common mistake: only checking whether the prediction "sounds reasonable" instead of checking whether x lies within the range of the data.

Exercise 2. (a) \( b_1 = 3.2 \): each additional €1,000 of monthly advertising is associated, on average, with €3,200 more in online sales in the area. \( R^2 = 0.45 \): advertising explains 45% of the between-area variability in sales; more than half depends on other factors. (b) \( t = 3.2/1.25 = 2.56 > 2.306 \): we reject \( H_0: \beta_1 = 0 \) at the 5% level — the relationship is significant. 95% CI: \( 3.2 \pm 2.306 \times 1.25 = 3.2 \pm 2.88 \Rightarrow [0.32 ;; 6.08] \). Note how the interval, although it excludes 0, is extremely wide: significant does not mean precise. Common mistake: using the critical z = 1.96 instead of the t with n − 2 = 8 df, and asserting causation ("advertising generates €3,200") when the data are observational.

Exercise 3. It is defensible that the model captures a very large share of the overall variability (R² = 0.91) and that the relationship is approximately linear on average. It is not defensible that the predictions are equally reliable everywhere: the funnel indicates heteroscedasticity — the error variance grows with the predicted level —, so predictions for large stores carry far more uncertainty than for small ones, however high the overall R². Different error margins should be communicated by segment (or the variable transformed). Common mistake: treating R² as a universal certificate of quality; R² is a global average that cannot see where the errors concentrate.

We have turned a correlation into a decision tool: the model \( y = \beta_0 + \beta_1 x + \varepsilon \), fitted by least squares, gave us the line \( \hat{y} = -2.60 + 0.0377x \), a slope with direct economic meaning (between €22.80 and €52.60 of daily sales per m², with 95% confidence), an R² of 86.5%, residuals that flag Cuenca as a store to investigate, and a concrete prediction for Alicante: about €53,900 a day for 1,500 m² — within the range of the data, which is where the line holds jurisdiction. We have also seen the machinery underneath: decomposing total variability into "explained + residual". Well then: that very decomposition, applied not to a line but to the comparison of several groups at once, is exactly the Analysis of Variance (ANOVA) — the answer we left pending in Module 5 when we discovered that comparing 42 stores with pairwise t tests was a false-alarm factory.