Welcome to the Fundamentals of Artificial Intelligence course. In this first lesson we will do two things: introduce the running example that will accompany us throughout the course (a fictional online store called NovaMarket) and walk through the history of artificial intelligence, from its philosophical roots to the era of generative AI. Understanding this history is not a general-knowledge exercise: the ups and downs of the field (the famous "AI winters") explain why the things that work today work, why ideas that seemed promising failed, and what expectations are reasonable when a company like NovaMarket decides to invest in AI.

1. Introducing the course and NovaMarket
2. Philosophical and mathematical precedents: from logic to Turing
3. Dartmouth 1956: the discipline is born
4. The golden age (1956–1974): Logic Theorist, ELIZA and the perceptron
5. The first winter (1974–1980)
6. Expert systems and the second winter (1980–1993)
7. The statistical revival (1993–2011): Deep Blue and machine learning
8. Big data and deep learning (2012–2016): ImageNet and AlphaGo
9. Transformers and the generative era (2017–today)
10. Summary timeline
11. The lessons of the winters

This course is a broad introduction to AI aimed at technical profiles who are just starting out (future AI engineers, junior developers, analysts). Over ten modules we will look at what AI is, how problems are represented, which classical algorithms exist (search, optimisation, logic), how machines learn from data (machine learning, neural networks, language models) and which tools are used in practice. You do not need advanced mathematics: whenever a numerical concept is needed, we will introduce it with small examples. It does help to have basic notions of Python, because the code examples are written in that language.

So that the concepts do not stay abstract, the whole course revolves around a fictional company:

• NovaMarket is a medium-sized Spanish online store selling electronics and home goods: around 180 employees, two warehouses (Zaragoza and Getafe), its own delivery fleet in several cities and courier shipping to the rest of Spain.
• It sells around 12,000 different products and receives on the order of 3,000 orders a day in normal season (considerably more during campaigns such as Black Friday or Christmas).
• Marta is the head of data. She comes from the world of business analytics and wants the company to stop making decisions "by eye".
• Diego is the head of operations: warehouses, delivery, returns and customer service. He is sceptical of technology fads, but the costs hurt.

NovaMarket has five datasets that we will be using in examples and exercises:

Twenty years ago, a company like NovaMarket could hardly have considered anything we will see in this course. Today it can, for three reasons that will come up again and again in this history lesson:

1. Data: every day NovaMarket generates thousands of records of orders, clicks, reviews and incidents. Those data used to be lost or stored unused; today they sit in accessible databases.
2. Cheap compute: training a model that in the 1990s would have required a supercomputer costs a few euros of cloud server time today.
3. Mature tools: there are libraries, services and pre-trained models that let a small team build in weeks what used to require years of research.

The problems Marta and Diego want to tackle (recommending products, forecasting demand, detecting fraud, classifying reviews, planning routes, assigning orders to delivery drivers, serving customers, automating return rules) are each an example of a different AI technique. Over the coming lessons we will place them on a map. But first, the history.

The question "can a machine think?" is far older than computers.

• Aristotle (4th century BC) formalised syllogisms: rules for drawing valid conclusions from premises ("all men are mortal; Socrates is a man; therefore Socrates is mortal"). It is the first time reasoning is treated as something mechanical.
• Leibniz (17th century) dreamed of a calculus ratiocinator: a formal language in which arguing would be "calculating".
• George Boole (1854) published The Laws of Thought, in which logic becomes algebra: true/false become 1/0 and the operations AND, OR and NOT behave like sums and products. All of digital electronics rests on this.
• Frege, Russell and Whitehead (1879–1913) developed predicate logic and tried to rebuild mathematics from logic.
• Gödel (1931) proved that any sufficiently powerful formal system contains truths it cannot prove: there are limits to what can be mechanised.
• Alan Turing (1936) defined the Turing machine, an abstract model of what it means "to compute". Anything a computer can do, a Turing machine can do. It is the theoretical foundation of computer science.
• McCulloch and Pitts (1943) proposed the first mathematical model of an artificial neuron: a unit that adds up input signals and "fires" if it exceeds a threshold. It is the direct ancestor of the neural networks we will see in module 5.
• Norbert Wiener (1948) founded cybernetics, the study of control and communication in machines and living beings (systems with feedback).

In his paper Computing Machinery and Intelligence, Turing sidestepped the philosophical question "can a machine think?" and replaced it with a practical experiment, the imitation game:

1. A human judge converses in writing with two interlocutors it cannot see: a person and a machine.
2. If the judge cannot tell which is which any better than chance, the machine "passes the test".

The idea is powerful because it defines intelligence by its observable behaviour, not by what happens "inside". It has been criticised (a machine could deceive without understanding anything) but it remains a cultural reference. Interestingly, today's conversational systems, which would fool a judge in many short conversations, have shown that passing the test is not the same as "being intelligent" in the general sense.

And for NovaMarket? The customer-service assistant the company wants to build is, in a way, an heir of the imitation game: what matters is not so much whether it "understands", but whether it resolves the customer's query in a way that is indistinguishable from (or better than) a human agent.

In the summer of 1956, John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon organised a workshop at Dartmouth College (New Hampshire, USA). In the proposal, McCarthy coined the term "Artificial Intelligence" and put forward a very optimistic conjecture:

"Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it."

Few concrete conclusions came out of the workshop, but something decisive did: a community with a name, a shared goal and funding (mostly military, through the DARPA agency). From this point on, AI exists as an academic discipline.

The first twenty years were marked by enormous enthusiasm. Some milestones:

• Logic Theorist (1956), by Newell, Simon and Shaw, presented at Dartmouth: a program capable of proving theorems from Russell and Whitehead's Principia Mathematica. One of its proofs was more elegant than the original. It is considered the first AI program.
• General Problem Solver (1957), by the same authors: it tried to solve any problem formalised as "initial state → goal state" by means of search (an idea we will see in module 3).
• Perceptron (1958), by Frank Rosenblatt: the first artificial neuron that learned from examples by adjusting weights. The press spoke of a machine that would soon "walk, talk, see and be conscious of its existence".
• LISP (1958), by McCarthy: the programming language of AI for decades.
• ELIZA (1966), by Joseph Weizenbaum: a program that simulated a psychotherapist by rephrasing what the user said. It worked with very simple substitution rules, yet many people felt it "understood them". Weizenbaum was alarmed by that effect (today we call it the ELIZA effect).
• SHRDLU (1970), by Terry Winograd: it understood English commands about a simulated "blocks world".
• Shakey (1966–1972): the first mobile robot that reasoned about its actions; its development produced the A* search algorithm, which we will study in module 3.

To get a feel for the spirit of ELIZA, here is a minimal version in Python. It is not "intelligent": it just looks for keywords and answers with templates.

Notice the detail: the program walks through the dictionary in order and returns the first match. With "My order is arriving late" it answers about the order and never reaches the "late" rule. This kind of system learns nothing: everything it knows was written by a person. Hold on to this idea, because in lesson 01-02 we will contrast "hand-written rules" with "rules learned from data".

By the early 1970s the promises were not being kept and funding was cut. This is known as the first AI winter. Main causes:

• Limits of the perceptron: in 1969 Minsky and Papert published Perceptrons, showing that a single layer of perceptrons cannot learn functions as simple as XOR (exclusive or). Although the problem is solved with several layers, nobody knew how to train them at the time, and neural network research was almost abandoned for 15 years.
• Combinatorial explosion: search methods worked on toy problems (the blocks world) but, as the problem grew, the number of combinations shot up and the computers of the day could not cope.
• Lack of data and compute: a 1970 computer had less memory than a modern musical greeting card.
• The Lighthill report (1973): the mathematician James Lighthill evaluated AI research for the British government and concluded that it had fulfilled almost none of its "grandiose promises". The United Kingdom dismantled a large part of its funding. In the USA, DARPA did something similar after the failure of speech-understanding projects.

AI was reborn in the 1980s with a more pragmatic approach: instead of a general intelligence, build programs that captured the knowledge of a human expert in a specific domain in the form of "IF... THEN..." rules.

• MYCIN (1970s, Stanford): it diagnosed bacterial infections and recommended antibiotics using around 600 rules. In evaluations it was as accurate as or more accurate than doctors, although it was never used in clinical practice (because of legal liability and the tedium of entering the data). It introduced certainty factors for reasoning under uncertainty.
• XCON / R1 (1980, Digital Equipment Corporation): it configured orders for VAX computers (which cables, boards and cabinets are needed). It saved DEC tens of millions of dollars a year. It was the first expert system with clear commercial success.
• Companies devoted to LISP machines and expert-system "shells" were founded; Japan launched the ambitious Fifth Generation project (1982).

We will look at expert systems in detail in module 6. For NovaMarket, its rules for returns and warranties ("if the product is less than 14 days old and unopened, the return is accepted with no questions asked") are a miniature expert system.

• Expert systems were expensive to maintain: each new rule could conflict with the previous ones and only a knowledge engineer knew how to touch them.
• They were brittle: outside their domain they knew nothing, and they did not learn from experience.
• The LISP-machine market collapsed in 1987 when generic PCs overtook them.
• The Fifth Generation project did not achieve its goals and funding was cut once again.

In the 1990s the field changed its philosophy. Instead of writing knowledge by hand, it bet on statistical and probabilistic methods that learn from data, and on evaluating results rigorously:

• Bayesian networks (Judea Pearl, late 1980s): rigorous probabilistic reasoning (module 6).
• Backpropagation (rediscovered and popularised in 1986 by Rumelhart, Hinton and Williams): at last multi-layer neural networks could be trained, albeit slowly.
• Support vector machines, decision trees, boosting: machine learning algorithms that worked well with the data and computers of the time (module 4).
• Deep Blue (1997): IBM's computer defeated the world chess champion Garry Kasparov. Its strength did not come from "understanding" chess, but from a highly optimised adversarial search (minimax with pruning, module 3) that evaluated around 200 million positions per second, plus a knowledge base of openings.
• The internet and the first big data: Bayesian spam filters, search engines, the recommenders of Amazon (2003) and Netflix (the Netflix Prize competition, 2006–2009).
• Watson (2011): IBM's system beat the human champions of the quiz show Jeopardy! by answering natural-language questions.

This is the period when the AI "that works" stopped being called AI: it was called machine learning, data mining or analytics. Many of the methods Marta will use at NovaMarket (demand forecasting, fraud detection) were born here.

Three ingredients came together in the early 2010s:

1. Massive data: the internet, smartphones and sensors generated gigantic labelled datasets. ImageNet (2009) contained millions of images classified into thousands of categories.
2. GPUs: graphics cards, designed for video games, turned out to be ideal for the matrix multiplications neural networks need.
3. Algorithmic improvements: new activation functions (ReLU), regularisation (dropout) and convolutional architectures.

The milestones:

• AlexNet (2012): the deep neural network by Krizhevsky, Sutskever and Hinton cut the error in the ImageNet competition from 26% to 16%, an unheard-of improvement. From that year on every winning entry was a deep network: the deep learning era is born.
• Speech recognition (2012–2014): voice assistants go from almost useless to usable.
• Neural translation (2016): Google Translate switches to neural networks and improves overnight.
• AlphaGo (2016), by DeepMind, defeats Lee Sedol, one of the world's best players of Go, a game considered beyond the reach of AI for decades because of its enormous number of positions. It combined deep neural networks with tree search and reinforcement learning (playing millions of games against itself).

• "Attention Is All You Need" (2017): Google researchers present the transformer architecture, based on an attention mechanism that allows sequences (text, above all) to be processed in parallel and at scale. We will study it in lesson 05-05.
• BERT (2018) and GPT-2 (2019): language models pre-trained on huge amounts of text and then adapted to specific tasks.
• GPT-3 (2020): 175 billion parameters; it shows that, as size and data scale up, capabilities appear that were not explicitly programmed (solving tasks from a handful of examples in the text itself).
• Diffusion models (2021–2022): DALL·E 2, Stable Diffusion and Midjourney generate images from text descriptions.
• ChatGPT (November 2022): OpenAI offers a conversational model to the general public; it reaches 100 million users in two months. AI becomes a topic of everyday conversation and of the political agenda.
• 2023–today: multimodal models (text, image, audio, video), open models, agents that execute tools, regulation (the EU AI Act, 2024) and an unprecedented investment race.

For a company like NovaMarket, this latest stage changes the rules of the game: things that five years ago required a research team (an assistant that answers in natural language about the status of an order, summarising thousands of reviews) are built today on pre-trained models accessible via API. We will look at their limits and risks in module 2 (ethics) and in module 5.

The two winters share a pattern worth committing to memory, because it repeats in every technology cycle:

1. A success on a small problem gets extrapolated to general intelligence. ELIZA "conversed", therefore "it will soon understand language"; the perceptron "learned", therefore "it will soon be conscious". Every impressive demonstration came with an outsized promise.
2. Compute and data did not keep up. The ideas of 1960 (neural networks, search) were essentially right; what was missing was millions of times more computing power and data, which only arrived 50 years later. Many recent "revolutions" are old ideas with new hardware.
3. Brittle systems do not scale. Hand-written knowledge (rules) works in closed domains and breaks outside them. Methods that learn from data generalise better, at the cost of being less explainable.
4. When funding is cut, it is cut all at once. Investors who overestimated the short term underestimated the long term.

What does this mean for NovaMarket? Marta and Diego must avoid two symmetrical mistakes:

• Over-expectation: believing that "AI" will solve any problem just by plugging it in. An assistant that answers frequently asked questions well will not know how to handle a complex complaint without proper design and data.
• Premature dismissal: thinking that, because a previous project failed (or because expert systems failed in the 1980s), the technique is worthless. Very often the technique was right and the data were missing.

The practical rule we will follow throughout the course: start with concrete, measurable problems for which data are available, and scale up only when something works.

• Thinking AI was born with ChatGPT. The discipline is seventy years old and most of today's techniques (neural networks, search, probability) are decades old. Knowing the history avoids "reinventing the wheel" and helps you assess what is new.
• Confusing "passing the Turing test" with "being intelligent". The test measures conversational behaviour; a system can be convincing without reasoning (the ELIZA effect). When evaluating an assistant for NovaMarket, measure whether it solves real problems, not whether it "sounds human".
• Believing rule-based systems are dead. They are still used enormously (validations, regulations, business rule engines). What was learned is where they work: stable, well-delimited domains.
• Ignoring the cost of data and compute. Every historical leap arrived when there were enough data and hardware. Before proposing a project, ask yourself what data you actually have.
• Tip: when you read a news item about AI, place it on the timeline: is it a new idea, or an old idea that has now become viable? That question will give you a far more realistic perspective than the headline.

Place the following milestones in chronological order and assign each one to the stage it belongs to (precedents, golden age, first winter, expert systems, second winter, statistical approach, deep learning, generative era):

a) AlphaGo beats Lee Sedol · b) Lighthill report · c) Turing test · d) XCON goes into production at DEC · e) ChatGPT opens to the public · f) Dartmouth workshop · g) Deep Blue beats Kasparov · h) AlexNet wins ImageNet

Diego has read that "an AI can serve customers without human intervention" and proposes laying off half the customer-service team within six months. Marta, on the other hand, remembers that in 2015 the company hired an "intelligent chatbot" that turned out to be a rule-based ELIZA and was a failure, and proposes changing nothing. Using the lessons of the winters, write three or four reasoned sentences on which mistake each of them is making and what approach would be more sensible.

Modify the lesson's mini-ELIZA program so that:

1. When the sentence contains several keywords, it picks the rule for the keyword that appears latest in the sentence (instead of the first one in the dictionary).
2. Add a rule for "warranty" with the reply "When did you buy the product?".

Test it with the sentence "My order is arriving late and I want to use the warranty". Reflect: does the program "understand" anything more after the change?

Solution 1

Solution 2 (indicative)

Diego is making the mistake of over-expectation: he extrapolates from a demonstration (an assistant that answers simple questions well) to solving the whole of customer service, just as in the 1960s ELIZA was extrapolated to language understanding. Marta is making the mistake of premature dismissal: she judges today's technique by a 2015 failure with a technology (hand-written rules) that has nothing to do with today's language models; it is like dismissing neural networks in 2010 because the perceptron failed in 1969. The sensible thing is an incremental approach: choose a measurable subset (for example, "where is my order?" queries), check with real data what percentage the assistant resolves well, keep people for the complex cases and expand only if the metrics back it up.

Solution 3

Explanation: find returns the position of the keyword in the sentence; we keep the keyword with the highest position. The program still "understands" nothing: we have only changed the criterion for choosing a template. It is a good reminder of why rule systems fell short and why the field ended up betting on learning from data.

In this lesson we have introduced the course and NovaMarket, the company that will serve as our laboratory, and we have walked through seventy years of AI history: from logic and Turing to the Dartmouth workshop; from the golden age (Logic Theorist, ELIZA, the perceptron) to the first winter; from expert systems (MYCIN, XCON) to the second winter; from the statistical revival (Deep Blue) to deep learning (ImageNet, AlphaGo) and the generative era of transformers and ChatGPT. The underlying lesson is that ideas usually arrive before the data and compute that make them viable, and that outsized expectations end in winters.

Now that we know where the field comes from, it is time to pin down what we are talking about: in the next lesson, Definition and Areas of AI, we will see what artificial intelligence is (and is not), how it relates to machine learning and deep learning, and what its major areas are, each with what it would bring to NovaMarket.