A different kind of assignment has landed at NovaMarket's head office: the retail trade association wants to produce a major study on consumer habits in Spain, and has asked Marta's team to take part in its design and analysis. Suddenly, the object of study is no longer 42 stores or 380,000 members with perfect transactional data: it is 48 million people who have to be asked. That leap — from measuring recorded behaviour to surveying a society — is the territory of statistics in the social sciences, and it has rules and traps of its own: samples that must be weighted, opinions measured on ordinal scales, correlations between territories that do not hold for people, and causes that can almost never be established with an experiment. This lesson works through all of it with worked cases.

1. What changes when the data point is a survey respondent
2. Large-scale surveys: strata and clusters
3. Weighting and post-stratification: fixing the sample you actually got
4. The margin of error of a barometer
5. Comparing social groups: chi-square and the digital divide
6. Ordinal scales: how to handle a Likert (properly)
7. Ecological correlation and the ecological fallacy
8. Causality without a laboratory: observation, confounding and natural experiments
9. Survey ethics: anonymity, consent and the GDPR

At NovaMarket, every receipt is a fact recorded by a till. In a survey, the data point is what somebody declares, and between reality and the answer stand all the biases from Data Collection and Sampling, amplified:

• Non-response bias: those who answer are not like those who do not (remember the campaign of 2,000 invitations with 640 responses? Extrapolating that 32% to the rest was risky; in social surveys non-response often exceeds 70%).
• Social desirability: people report recycling more, drinking less and reading more than they actually do.
• Coverage bias: surveying only over the internet leaves out precisely those who use it least — fatal if the study is, of all things, about digital habits.
• Questionnaire effects: the order and wording of the questions move the results.

The consequence: in the social sciences, sample design and fieldwork weigh as much as the analysis. A brilliant analysis of a crooked sample only produces crooked conclusions with decimal places.

The sector study plans 2,500 face-to-face interviews. Simple random sampling of the whole census would be hugely expensive (2,500 interviewers scattered across 8,000 municipalities), so the two techniques you already know are combined:

• Stratification: the population is divided into homogeneous groups (region × municipality size) and the sample is allocated proportionally among them. It guarantees that every stratum is represented and reduces the estimator's variance.
• Clusters (stages): within each stratum, municipalities are drawn at random, then census tracts within each municipality, then households within each tract. It makes fieldwork cheaper (interviewers are concentrated), but increases the variance: neighbours in the same tract resemble one another, so 10 interviews on the same street carry less information than 10 scattered ones.

That extra cost is measured with the design effect (deff): if deff = 1.5, the 2,500 interviews inform like \(2,500 / 1.5 \approx 1,667\) simple random interviews (the "effective sample size"). Serious barometers publish it in their technical sheet; ignoring it makes the declared margin of error optimistic.

However good the design, the final sample never nails the structure of the population: young people are at home less and answer less. The standard fix is weighting: giving each response a weight that restores the known population proportions (from the census or official statistics). A small, complete example, with the question "do you shop online at least once a month?":

• Unweighted estimate: \((150 \cdot 0.62 + 420 \cdot 0.38 + 430 \cdot 0.12)/1,000 = (93 + 159.6 + 51.6)/1,000 = 30.4\ %\).
• Weighted estimate: each group weighs what it weighs in the population: \(0.25 \times 62 + 0.40 \times 38 + 0.35 \times 12 = 15.5 + 15.2 + 4.2 = 34.9\ %\).

Almost 5 points of difference, purely from the under-representation of the young (who shop online the most). This is post-stratification: it is done after the fieldwork, using variables whose population distribution is known (age, sex, territory, education level). Two honest limits: it only corrects the non-response tied to the weighting variables (if, among the young, it is the most digital who answer, the bias persists), and extreme weights inflate the variance — a group that is nearly empty in the sample is not fixed by multiplying it by 8.

The technical sheets of surveys like the CIS barometer (the CIS is Spain's sociological research centre) say things like: "2,500 interviews; margin of error ±2.0% for the sample as a whole, at a confidence level of 95.5% and under the assumption p = q = 0.5". You can now read every piece with what you learned in Confidence Intervals:

\[ e = z \sqrt{\frac{p(1-p)}{n}} = 2 \times \sqrt{\frac{0.5 \times 0.5}{2,500}} = 2 \times 0.01 = 0.02 = \pm 2 \text{ points} \]

• \(p = q = 0.5\) is the worst case: it maximizes \(p(1-p)\), so the actual margin for any specific percentage is equal or smaller.
• The 95.5% corresponds to \(z = 2\) exactly (survey institutes prefer it to the 1.96 of 95%).
• The margin holds for the total; for a subgroup (the 150 young people in the sample) the margin is \(2\sqrt{0.25/150} \approx \pm 8\) points. Headlines that compare small subgroups tend to forget this.
• And all of it assumes simple random sampling: with clusters, the deff widens it.

The professional reading: when a habit "rises from 31% to 32%" between two waves of \(n = 2,500\), the change is within the margin — indistinguishable from noise, exactly the logic of Hypothesis Testing.

The first substantive question in the study: does regular online shopping depend on age? Two categorical variables → contingency table and chi-square. A subsample of 1,200 people (400 per group):

Under \(H_0\) (independence), each group would have \(400 \times 492/1,200 = 164\) regular shoppers. The statistic:

\[ \chi^2 = \frac{(248-164)^2}{164} + \frac{(168-164)^2}{164} + \frac{(76-164)^2}{164} + \frac{(152-236)^2}{236} + \frac{(232-236)^2}{236} + \frac{(324-236)^2}{236} \]

\[ \chi^2 = 43.0 + 0.1 + 47.2 + 29.9 + 0.1 + 32.8 \approx 153.1 \]

With \((2-1)(3-1) = 2\) degrees of freedom, the critical value at 5% is 5.99: the association is overwhelming (\(p < 0.001\)). The effect size, with Cramér's V: \(V = \sqrt{153.1/(1,200 \times 1)} = 0.36\), a moderate-to-strong association. But the number that will go in the report is the descriptive one: 62% regular online shoppers among the young versus 19% among the over-55s. That distance is the digital divide, and it shapes very concrete decisions: for NovaMarket, that the online channel does not replace the physical store for senior customers; for the sector, where face-to-face service must not be shut down.

Half of Spain answers surveys on Likert scales: "rate from 1 (strongly disagree) to 5 (strongly agree)". They are ordinal data (Types of Data): the order means something, the distances do not — nothing guarantees that going from 4 to 5 is "the same amount" as going from 2 to 3. Practical consequences:

• Description: median and frequency distribution before the mean. "Median 4; 71% answer 4 or 5" says more — and says it more honestly — than "mean 3.84". The mean of a Likert is used (and abused) out of convenience; if it is reported, always alongside the distribution.
• Comparing groups: the rank-based methods from the Non-Parametric Methods lesson: Mann-Whitney for two groups, Kruskal-Wallis for several. An example from the study: trust in online shopping (1–5) gives a median of 4 in urban areas and 3 in rural ones; Mann-Whitney yields \(p = 0.002\) → the shift is real, and it is communicated with the medians and the percentages at each level, not with "0.6 points of mean".
• Summing items: when many Likert items are summed into an index (e.g. 10 digital-trust questions, range 10–50), the total behaves almost continuously and parametric methods become reasonable again. It is the exception that proves the rule.

The team crosses data by province: average provincial income and the provincial percentage of online shopping. The correlation comes out at \(r = 0.81\): extremely strong. The tempting headline: "the richer you are, the more you shop online". Stop right there.

That \(r = 0.81\) is an ecological correlation: between aggregates (provinces), not between people. Within each province, the individual correlation between income and online shopping can be much weaker (in the study, \(r \approx 0.25\)) or even of the opposite sign. Drawing conclusions about individuals from aggregates is the ecological fallacy, documented since Robinson's classic (1950): in the 1930s, the US states with more immigrants had higher literacy (a positive aggregate correlation), while at the individual level immigrants were less literate — they simply settled in more literate states. Aggregates mix a group's composition with its members' behaviour.

The professional rule: the unit of analysis must match the unit of the conclusion. Provincial data → conclusions about provinces ("higher-income provinces concentrate more online shopping", useful for logistics); to talk about people, you need data on people.

On NovaMarket's website you can randomize a checkout (Statistics in Business); in society you cannot randomize education level or income. Almost every social study is observational, and there correlation is a clue, not a verdict — the "correlation does not imply causation" of Correlation Analysis in its natural habitat.

The study finds: regular online shopping at 54% among those with higher education versus 31% for everyone else. Does education cause online shopping? Let's stratify by age, the usual suspect:

Within each age group, the difference by education is 3–4 points, not 23. What was going on? Graduates are, on average, much younger (the expansion of universities is recent), and age — the digital divide from the chi-square section — was doing nearly all the work. Age is a confounder: related both to the apparent "cause" and to the effect. It is the same partial-reversal phenomenon you saw as Simpson's paradox in Chi-Square, and the raison d'être of the multiple regression in Multivariate Analysis: "controlling for" age is, precisely, comparing within each stratum. A reminder from Cuenca: its sales "deficit" also vanished once foot traffic was controlled for.

• Adjusting for confounders: stratify, or put the confounders into a regression (multiple or logistic). The limit: it only controls what has been measured.
• Natural experiments: sometimes reality randomizes on its own — a regulatory change that takes effect in one region but not its neighbour, a lottery for school places, a cutoff by date of birth. Comparing right on either side of that boundary comes close to an experiment. (NovaMarket's staggered city-by-city rollout of the 2-hour delivery service worked as one: it made it possible to compare cities with and without the service, before and after.)
• A hierarchy of confidence: randomized experiment > well-defended natural experiment > adjusted observational study > raw correlation. In the social sciences you live almost always on the two middle rungs, and honesty consists of saying so.

Running a survey means processing personal data, and in Europe that triggers the GDPR. The minimum the team must guarantee in the sector study:

• Informed consent: the respondent knows who processes their data and for what purpose, and accepts freely; they can decline any question and withdraw.
• Minimization: ask only what the study's purposes require (political affiliation has no business in a consumer study).
• Genuine anonymization: keep identifiers separate from responses, and beware of re-identification by cross-referencing — "a 34-year-old woman, a pharmacist, in a village of 800 inhabitants" is identifiable even without a name. Public microdata are released with small cells aggregated.
• Special categories (health, ideology, religion, sexual orientation): reinforced protection; in general, avoid them except with justified need and explicit consent.
• Retention and security: defined retention periods, restricted access, and a right to erasure that actually works.

Warning: this section is statistical guidance, not legal advice. Before launching any survey involving personal data, the design (questionnaire, privacy notices, retention plan) must be reviewed by the DPO or a compliance professional. At NovaMarket no survey goes out without that sign-off, and the same should be true in your organization.

Ethics is not a toll: a survey people perceive as respectful gets more responses and better responses — it is also a data-quality issue.

• Analysing the raw sample when there is differential non-response. Without weighting, the groups that answer least vanish from the result (30.4% vs 34.9% in the example). Weight with known population variables — and be wary of extreme weights.
• Applying the overall margin of error to subgroups. The ±2% of \(n = 2,500\) becomes ±8% for 150 young people. Every percentage travels with the \(n\) that supports it.
• Averaging Likerts without looking at the distribution. A mean of 3.0 can be "everyone indifferent" or "half enthusiastic, half opposed": opposite decisions. Median + distribution, always.
• Jumping from aggregates to individuals. Provincial correlation says nothing about people (ecological fallacy). Unit of analysis = unit of conclusion.
• Reading causality into observational data without hunting for confounders. Before signing off on "education causes X", stratify by age, income and territory. If the effect survives, it is still a hypothesis — a stronger one, but a hypothesis.
• Treating ethics as end-of-project paperwork. Consent, anonymity and minimization are designed in at the start; patching them on afterwards is usually impossible (and sanctionable).

In a municipal survey (\(n = 500\)), 40% of the sample is under 40 when the municipal register says they are 30%. Support for extending the pedestrianization of the town centre: 60% among the under-40s, 40% among the rest. Calculate the unweighted estimate and the post-stratified one.

From the sector study: use of the chain's app by habitat. Urban: 260 users out of 500; rural: 96 out of 300. Test independence with chi-square (\(\alpha = 0.05\), critical value 3.84) and interpret in terms of the digital divide.

A report concludes: "provinces with more supermarkets per inhabitant have a lower obesity rate (\(r = -0.6\)); therefore, opening supermarkets reduces people's obesity". Point out the two distinct problems with this sentence.

Exercise 1.

Unweighted: \(0.40 \times 60 + 0.60 \times 40 = 24 + 24 = 48.0\ %\). Weighted: \(0.30 \times 60 + 0.70 \times 40 = 18 + 28 = 46.0\ %\). The over-representation of the young inflated support by 2 points — enough to cross the psychological 50% in tight headlines. Common mistake: weighting with the sample sizes instead of the population ones; the correct weights restore the population, not the sample.

Exercise 2.

Totals: 356 users, 444 non-users; \(n = 800\). Expected: urban-yes \(500 \times 356/800 = 222.5\); urban-no 277.5; rural-yes 133.5; rural-no 166.5. \(\chi^2 = \frac{(260-222.5)^2}{222.5} + \frac{(240-277.5)^2}{277.5} + \frac{(96-133.5)^2}{133.5} + \frac{(204-166.5)^2}{166.5} = 6.3 + 5.1 + 10.5 + 8.4 = 30.3 > 3.84\): independence is rejected (\(p < 0.001\)). The descriptive figure for the report: 52% urban use versus 32% rural — a 20-point territorial digital divide, relevant for deciding where the app can replace channels and where it cannot. Tip: as always, the p-value says "there is an association"; the percentages say how much and for whom.

Exercise 3.

(1) Ecological fallacy: the correlation is between provinces; it says nothing about whether the people who live near a supermarket are less obese. (2) Causality from observational data: even at the provincial level, income and urban habitat are obvious confounders (rich, urban provinces have more supermarkets and less obesity through many other channels). The sentence commits both sins at once; a defensible version would be: "at the provincial level, retail density and obesity are negatively associated; the individual-level relationship and its causal direction require individual data and adjustment for confounders". Tip: training the reflex of asking "what is the unit?" and "what third variable could explain this?" is worth more than any formula in this lesson.

The sector assignment has taken you from perfect transactional data to imperfect self-reported data, and you have watched the toolkit adapt: strata and clusters with their design effect, the post-stratification that gave 5 points back to the missing young people, a barometer's margin of error read with the eyes of the intervals lesson, the chi-square that quantified the digital divide, Likerts treated as what they are (ordinal), Robinson's ecological fallacy, and age unmasked as the confounder behind the "education" effect. And one constant: in social matters, causality is argued with design and humility — it is not computed.

The next assignment opens another door: the insurer that manages the wellness programme for NovaMarket's 8,000 employees proposes analysing its results together, and the reports come packed with relative risks, odds ratios, sensitivities and survival curves. Time to learn to read — critically, and without playing doctor — the language of Statistics in Health Sciences.