The Project | About Us | Contribute | Donations | License

HOME

The previous lesson ended with two loose ends. The first: the EasyTask v0.15 agenda is a bare list to which any part of the program can add whatever it likes —a Task object, a stray dictionary or a number—, and the operations that handle it are still spread around the file. The second: save_tasks is broken, because json.dump cannot write objects. This lesson resolves both, and along the way teaches you to work comfortably with many objects at once, which is what a real program does almost all the time. You will see how to iterate, filter, count, sort and search objects using everything learnt in modules 5 and 6, and then you will take the same leap as in the previous lesson but one level up: creating an Agenda class that contains the tasks and offers the operations that today wander loose. At the end, inheritance and polymorphism will appear in their most basic form.

List of objects versus list of dictionaries
Iterating, filtering and counting objects
Sorting objects
Searching objects and building indexes
Composition: the Agenda class
Why the agenda must protect its internal list
Serialising and rebuilding: to_dict and from_dict
Inheritance and polymorphism, the basics
EasyTask v0.16
Common mistakes and tips
Exercises
Conclusion

List of objects versus list of dictionaries

The structure is the same as in module 5 —a list— but what is inside has changed:

agenda = [
    Task("Book fair poster", "Luis", "high", 3),
    Task("Sole Bakery logo", "Nuria", "high", 5),
    Task("Vidal quote", "Marta", "medium", 2),
]
print(len(agenda), agenda[0].title)      # 3 Book fair poster

Everything you know about lists still applies: indexes, slices, append, len, in, comprehensions and sorted. The only thing that changes is how each field is reached, and that is where the gain lies:

	List of dictionaries	List of objects
Reaching a field	`t["title"]`	`t.title`
Misspelt name	`t["titel"]` → `KeyError` at run time	`t.titel` → the editor underlines it as you type
Editor autocompletion	None: the keys are strings	Yes: pressing `.` shows the attributes
Guaranteed fields and behaviour	No guarantee; loose functions	The constructor's; the object's methods

The second row is the one you notice most day to day: with dictionaries, task["assigne"] is a perfectly valid string as far as Python is concerned and the failure does not appear until that line runs; with objects, task.assigne is a name the editor does not recognise and it marks it in red while you type. It is the difference between finding a bug in two seconds or in two weeks.

Iterating, filtering and counting objects

The three patterns from module 5 translate directly: iterate with for, filter with a comprehension and count with sum.

for task in agenda:                                        # iterate
    print(task)                                            # uses __str__
pending = [t for t in agenda if not t.completed]           # filter
how_many = sum(1 for t in agenda if not t.completed)       # count
days = sum(t.days for t in agenda if not t.completed)      # add up
print(f"{how_many} pending tasks, {days} days of work.")

Three notes on these lines, because they condense half the course:

sum(1 for t in agenda if ...) counts without building any intermediate list: the generator produces ones and sum accumulates them. It is equivalent to len([t for t in agenda if ...]) without spending memory, and it is the idiomatic way to count with a condition.
The filter can use methods, not just attributes: [t for t in agenda if t.is_overdue()] is valid and far more expressive than repeating the day comparison in every place. That is the advantage of having put the behaviour inside the class.
The filtered objects are not copied. pending holds references to the very same tasks, so pending[0].complete() also affects the task sitting in agenda: it is the aliasing of 05-01, now with objects.

Sorting objects

In 06-02 you learnt to sort with sorted(..., key=...). With objects, the key is written with dot notation:

from operator import attrgetter

by_days = sorted(agenda, key=lambda t: t.days)                    # a lambda
by_days = sorted(agenda, key=attrgetter("days"))                  # equivalent
by_person = sorted(agenda, key=attrgetter("assignee", "days"))    # key tuple

PRIORITY_ORDER = {"high": 0, "medium": 1, "low": 2}
by_urgency = sorted(agenda, key=lambda t: (PRIORITY_ORDER[t.priority], t.days))

operator.attrgetter is to an attribute what itemgetter was to a dictionary key: attrgetter("days") returns a function that, given an object, extracts its days attribute. With attrgetter("assignee", "days") you get the two-criteria key tuple straight away. And the last line brings back the technique of 06-02: a tuple sorts first by its first element and, in the event of a tie, by the second. There is one more alternative: teaching the class itself how to compare itself, by defining the special method __lt__ (from less than).

    def __lt__(self, other):
        """Natural order: most urgent first; at equal urgency, the shortest."""
        return ((PRIORITY_ORDER[self.priority], self.days)
                < (PRIORITY_ORDER[other.priority], other.days))

print(sorted(agenda)[0].title)         # no key: it uses __lt__
print(min(agenda).title)               # min and max use it too

Way of sorting	When to use it
`key=lambda t: ...` or `key=attrgetter("field")`	One-off sorts; the second is more readable
`__lt__` in the class	When the type has one obvious natural order

The practical rule is this: define __lt__ only if there is an order anybody would take for granted for that object (a date, an invoice number, the urgency of a task); if there are several equally reasonable orders, key is better, because it makes explicit in each call which criterion is being used.

Searching objects and building indexes

The linear search of 06-01 has a very convenient shorthand in Python: next() over a generator.

def find(agenda, title):
    """Return the first task with that title, or None if there is none."""
    return next((t for t in agenda if t.title.lower() == title.lower()), None)

found = find(agenda, "sole bakery logo")
print(found.assignee if found else "Not found")     # Nuria

next(generator, default_value) asks for the first element the generator produces and, if there is none, returns the default value instead of failing. Since the generator is lazy, it stops iterating as soon as it finds the first match: it is exactly linear search with early exit, written in one line. The second argument is not optional in practice: without it, an empty generator raises StopIteration. And when a query is repeated a lot, you build an index, just as in 06-01, but now from assignee to a list of objects:

index = {}
for task in agenda:
    index.setdefault(task.assignee, []).append(task)     # assignee -> [Task, ...]
print([t.title for t in index.get("Nuria", [])])         # get: [] if she has none

The trade-off is the same as back then: building the index costs O(n) once and afterwards each query is O(1), but the index goes stale as soon as a task is added or deleted. In the Agenda class of the next section that stops being a problem, because the agenda itself can rebuild it when it changes.

Composition: the `Agenda` class

All the previous functions receive agenda as their first parameter, and that is exactly the signal we recognised in 07-02: a pile of functions revolving around the same piece of data wants to be a class. The Agenda is not a list: it has a list. That relationship —an object that contains others— is called composition, and it is the commonest and healthiest way of combining classes.

classDiagram
    class Agenda {
        -_tasks: list
        +add(task)
        +remove(title)
        +find(title)
        +filter_by(criterion)
        +sorted_tasks(key)
    }
    class Task {
        +title
        +complete()
        +to_dict()
    }
    Agenda "1" o-- "0..*" Task : contains

class Agenda:
    """Collection of studio tasks, with the operations that belong to it."""
    def __init__(self, tasks=None):
        self._tasks = list(tasks) if tasks else []         # copies what it receives
    def __len__(self):
        return len(self._tasks)                            # enables len(agenda)

    def __iter__(self):
        return iter(self._tasks)                           # enables: for t in agenda

    def add(self, task):                          # rejects anything that is not a Task
        if not isinstance(task, Task) or task in self._tasks:       # and duplicates
            return False
        self._tasks.append(task)
        return True

    def remove(self, title):                      # reports whether it deleted anything
        task = self.find(title)
        if task is not None:
            self._tasks.remove(task)              # remove uses __eq__ under the bonnet
        return task is not None

    def find(self, title):                        # first one with that title, or None
        return next((t for t in self._tasks if t.title.lower() == title.lower()), None)

    def filter_by(self, criterion):               # criterion: function returning a bool
        return [t for t in self._tasks if criterion(t)]

    def sorted_tasks(self, key=None):
        """Sorted copy; in natural order (__lt__) if no key is given."""
        return sorted(self._tasks) if key is None else sorted(self._tasks, key=key)

    def summary_by_assignee(self):
        """Dictionary assignee -> {'tasks': n, 'days': d} of what is pending."""
        summary = {}
        for t in self._tasks:
            if not t.completed:
                data = summary.setdefault(t.assignee, {"tasks": 0, "days": 0})
                data["tasks"], data["days"] = data["tasks"] + 1, data["days"] + t.days
        return summary

What is gained is enormous, and it is worth seeing it in use:

agenda = Agenda()
agenda.add(Task("Book fair poster", "Luis", "high", 3))
agenda.add(Task("Sole Bakery logo", "Nuria", "high", 5))
agenda.add("just some string")                      # False: it is not a Task
print(len(agenda))                                  # 2, thanks to __len__
for task in agenda: print(task)                     # thanks to __iter__
print(agenda.summary_by_assignee())
# {'Luis': {'tasks': 1, 'days': 3}, 'Nuria': {'tasks': 1, 'days': 5}}

The two special methods are what make the agenda behave like a Python collection: __len__ enables len(agenda) and also makes if agenda: false when it is empty; __iter__ enables for task in agenda, and with it you get in, list(agenda), sum(...) and comprehensions for free. Notice too that add validates, just as the Task constructor did: it rejects anything that is not a task and avoids duplicates by taking advantage of the __eq__ we defined in 07-02, because the in operator uses it internally. The hole through which a stray dictionary used to slip is closed.

Why the agenda must protect its internal list

The attribute is called _tasks, with a leading underscore. It is a Python convention, not a prohibition: it means "this is internal to the class, do not touch it from outside". Python does not prevent it, but any programmer reading it knows not to meddle there. Why does it matter? Because if the agenda hands over its real list, anyone can bypass all of its rules:

    def tasks(self):
        return self._tasks           # WRONG: hands over the real list

items = agenda.tasks()
items.append("this is not a task")   # the control in add() has evaporated
items.clear()                        # and the agenda has been left empty

It is the aliasing of 05-01 in its most dangerous form: items and agenda._tasks are the same object, and whoever holds one holds the other. The solution is to return a copy, with return list(self._tasks) instead of return self._tasks. It is worth being precise about how far that protection goes, because it is a shallow copy (05-04): the list is new, but the tasks inside it are the same ones. Adding or removing elements from the copy no longer affects the agenda; on the other hand, copy[0].complete() does modify the original task. And it is right that it should: normally you want to protect the structure, not to stop anyone working with the tasks. If you needed total isolation, that is where copy.deepcopy would come in, with its cost. Note that sorted_tasks and filter_by already obey the rule effortlessly: sorted and comprehensions always build new lists.

Serialising and rebuilding: `to_dict` and `from_dict`

We reach the second loose end. When you try json.dump(agenda, file) with objects inside, Python answers TypeError: Object of type Task is not JSON serializable. The reason is that json only knows how to write basic types —dictionaries, lists, strings, numbers, booleans and None— and has no way of guessing which part of an object should be saved. So we do the conversion ourselves, with a method on the way out and an alternative constructor on the way back:

    def to_dict(self):                            # in the Task class
        """Return the task as a dictionary, ready for the JSON file."""
        return {"title": self.title, "assignee": self.assignee,
                "priority": self.priority, "days": self.days,
                "done": self.done, "completed": self.completed}

    @classmethod
    def from_dict(cls, data):
        """Create a Task from a dictionary read out of the JSON file."""
        ...            # the from_dictionary of 07-02, with its final name

to_dict walks the attributes that need persisting and returns the dictionary; from_dict makes the return journey through the constructor. With those two pieces, the agenda recovers its persistence:

    def save(self, path=JSON_PATH):               # in the Agenda class
        """Save every task to the JSON file."""
        with open(path, "w", encoding="utf-8") as f:
            json.dump([t.to_dict() for t in self._tasks], f, indent=2, ensure_ascii=False)

    @classmethod
    def load(cls, path=JSON_PATH):
        """Return the saved Agenda, or an empty one if there is no file."""
        if not path.exists():
            return cls()                          # empty agenda: first run
        with open(path, "r", encoding="utf-8") as f:
            return cls(Task.from_dict(d) for d in json.load(f))

Three details deserve attention. The comprehension [t.to_dict() for t in self._tasks] converts the whole list of objects into a list of dictionaries in one line; the resulting JSON is identical to v0.14's, so the old files still work. load is a @classmethod because it manufactures an agenda, just as from_dict manufactures a task: it is called as Agenda.load(), without having an agenda beforehand. And the rebuild passes each dictionary through Task.from_dict, that is, through the constructor: any dirty data in the file —a "High" priority, an invented field that is ignored— is normalised on the way in. The program stops inheriting the rubbish of its own history.

Inheritance and polymorphism, the basics

Marta needs something new: tasks that repeat at regular intervals, such as the monthly maintenance of the Solé Bakery website. They are ordinary tasks plus a period. Inheritance lets you say exactly that: create a class from another one, keeping everything the first already knows how to do.

class RecurringTask(Task):                        # in brackets, the base class
    """A task that repeats every so many days."""
    def __init__(self, title, assignee, priority="medium", days=1, every_days=30):
        super().__init__(title, assignee, priority, days)        # Task's constructor
        self.every_days = every_days if every_days > 0 else 30

    def __str__(self):
        return f"{super().__str__()}  (every {self.every_days}d)"  # reuse and extend

maintenance = RecurringTask("Sole website maintenance", "Luis", "medium", 1, 30)
print(maintenance.complete())           # True: inherited method, we never wrote it
print(maintenance.assignee, isinstance(maintenance, Task))   # Luis True

The three essential ideas:

class Child(Parent) inherits every attribute and method of the base class: complete, reassign, to_dict or __eq__ work without writing a single line.
super() gives access to the base class. super().__init__(...) runs Task's constructor —with all its validation— and then the child adds its own bits; forgetting that call is the classic mistake, and it leaves the task with no title, no assignee and nothing else. Overriding a method means defining it again in the child, as __str__ does, which even so reuses the parent's with super().__str__().

And now polymorphism, which sounds grandiose and is disarmingly simple:

mixed = [Task("Book fair poster", "Luis", "high", 3),
         RecurringTask("Sole website maintenance", "Luis", "medium", 1, 30)]
for t in mixed:
    print(t)                            # each object uses ITS own version of __str__

The loop does not ask which class each element belongs to, nor does it have a single if: each object knows how to print itself its own way. That is polymorphism, it works the same with to_dict or complete, and it is what will let us add an UrgentTask tomorrow without touching the code that walks the agenda. That said, inheritance is used far less than it seems:

	Inheritance (`RecurringTask(Task)`)	Composition (`Agenda` has tasks)
Relationship	"is a": a recurring task is a task	"has a": an agenda has tasks
Coupling	Strong: a change in the parent affects the children	Weak: each class evolves on its own
Real frequency	Rare	Almost always

The practical rule is to prefer composition, and reserve inheritance for when the sentence "an X is a Y" is true without forcing it and the child does not need to undo anything the parent does. Agenda does not inherit from list for precisely that reason: an agenda is not a list —agenda.sort(reverse=True) and agenda + [3, 4] make no sense—, it has one.

EasyTask v0.16

Everything comes together. The program no longer handles a loose list but an Agenda object, and the functions that walked it have moved inside:

# easytask.py - Alba Studio / Version 0.16: the agenda is an object
# --- Task (v0.15) with to_dict/from_dict, RecurringTask and Agenda: sections 5 to 8 ---

def show_list(agenda):
    """Show the agenda sorted by urgency."""
    if not agenda:                                    # __len__ makes this possible
        print("The agenda is empty.")
        return
    for number, task in enumerate(agenda.sorted_tasks(), start=1):
        print(f"{number:>2}. {task}")
    completed = sum(1 for t in agenda if t.completed)  # __iter__ makes this possible
    print(f"Total: {len(agenda)} tasks, {completed} completed.")

def main():
    """Run the main loop of the application."""
    agenda = Agenda.load()                            # @classmethod: builds the agenda
    while True:
        show_menu()
        option = ask_option("Choose an option (1-10): ", OPTIONS)
        if option == "10" and confirm("Are you sure you want to quit?"):
            agenda.save()
            break
        # ... the other options call agenda.add(...), agenda.find(...),
        #     agenda.filter_by(...), agenda.remove(...) and the Task methods

Compare the before and after of any given operation:

In v0.15	In v0.16
`agenda.append(task)` with no checks at all	`agenda.add(task)`, which validates and avoids duplicates
`find_if(agenda, criterion)`	`agenda.find(title)` / `agenda.filter_by(criterion)`
`sorted(agenda, key=sort_key)` in three places	`agenda.sorted_tasks()`
`save_tasks(agenda)` broken by the objects	`agenda.save()` with `to_dict`

The main program has slimmed down to what actually belongs to it: talking to the user. All the agenda logic lives in Agenda, and all the task logic in Task.

Common Mistakes and Tips

Returning the internal list instead of a copy. It is the mistake in section 6 and it cancels the class's validation: return list(self._tasks), always.
Mistaking _tasks for something genuinely private. The underscore is a signal for humans: agenda._tasks.append(3) works just the same, and the discipline comes from the team, not from the language.
Inheriting from list or dict to "save yourself" the composition. It drags in dozens of methods you do not want (sort, pop, extend) that bypass your rules. Contain the list inside your class.
Forgetting super().__init__(...) in the child's constructor: the parent's attributes are never created and everything fails later with AttributeError.
Using next(...) with no default value. next(t for t in agenda if ...) raises StopIteration when there are no matches: always add the , None. And define __iter__ and __len__ as a pair: with both, your class behaves like a Python collection in almost any context.
Tip: measure the index before building it. An index_by_assignee over an agenda of twenty tasks saves nothing and has to be kept up to date; with twenty thousand, it changes the program. It is the rule of 06-04: measure before optimising.

Exercises

Exercise 1: Queries over a list of objects

Given a list of Task objects, write one-line expressions that return: (a) the titles of Nuria's pending tasks; (b) how many high-priority ones are still not completed; (c) the task with the most days of work; (d) the list sorted by assignee and, within each person, by days descending.

Exercise 2: Extending the `Agenda` class

Add three methods to Agenda: pending(), with the uncompleted tasks; pending_days(), with the total days of work left; and __contains__(self, title), which lets you write "Book fair poster" in agenda. Justify why pending() must not return self._tasks.

Exercise 3: A subclass with `super()`

Create TaskWithClient(Task), which adds the attribute client (mandatory text), overrides __str__ to show it in brackets at the end, and extends to_dict() so that the dictionary also includes the client. Then check that a mixed list of Task and TaskWithClient is printed and saved correctly without a single if.

Solutions

Solution 1.

[t.title for t in agenda if t.assignee == "Nuria" and not t.completed]    # (a)
sum(1 for t in agenda if t.priority == "high" and not t.completed)        # (b)
max(agenda, key=attrgetter("days"))                  # (c) max with key, like sorted
sorted(agenda, key=lambda t: (t.assignee, -t.days))                       # (d)

Part (d) uses the trick from 06-02: since reverse=True would invert both criteria, only the numeric one is negated with -t.days. You cannot do that with strings, and you would have to sort twice, taking advantage of Timsort being stable: first by the secondary criterion and then by the main one.

Solution 2.

    def pending(self):
        """New list with the uncompleted tasks."""
        return [t for t in self._tasks if not t.completed]

    def pending_days(self):
        """Days of work still to be done across the whole agenda."""
        return sum(t.days for t in self.pending())

    def __contains__(self, title):                # 'Book fair poster' in agenda
        return self.find(title) is not None

pending() cannot return self._tasks for two reasons: it would be incorrect —it would include the completed ones— and, more fundamentally, it would hand over the internal list, so whoever received it could empty it or fill it with rubbish, bypassing add. The comprehension solves both, because it builds a new list. And __contains__ is another special method from the __len__ and __iter__ family: without it, in would work by walking __iter__ and comparing objects, but this way we can search by title, which is what feels natural here.

Solution 3.

class TaskWithClient(Task):
    """A task tied to a specific client of the studio."""
    def __init__(self, title, assignee, client, priority="medium", days=1):
        super().__init__(title, assignee, priority, days)
        self.client = client.strip() if client.strip() else "No client"

    def __str__(self):
        return f"{super().__str__()}  [{self.client}]"

    def to_dict(self):
        data = super().to_dict()             # the base class's dictionary
        data["client"] = self.client         # and we add our own bit
        return data

mixed = [Task("Book fair poster", "Luis", "high", 3),
         TaskWithClient("Logo", "Nuria", "Sole Bakery", "high", 5)]
for t in mixed: print(t)                     # each one uses its own __str__

The data = super().to_dict() and then add pattern is the same as __str__'s: reuse what the base class already does well and extend it, instead of copying its six fields and risking them going out of sync the day Task gains a new one. And the final loop demonstrates polymorphism: there is not one isinstance nor one if, and each object behaves as it should.

Conclusion

Working with lists of objects is just like working with lists of dictionaries, except that access is t.title instead of t["title"], the editor autocompletes and warns about misspelt names, and the fields are guaranteed by the constructor. The usual patterns translate effortlessly: iterate with for, filter with comprehensions —even calling methods such as t.is_overdue()—, count with sum(1 for t in ... if ...), sort with key=lambda t: t.days or attrgetter, combine criteria with a key tuple and, if the type has one obvious natural order, define __lt__ so that sorted, min and max work without a key. Searching is next((t for t in agenda if ...), None), and when a query is repeated a lot, an index from assignee to a list of objects. All of that, gathered around the same piece of data, was crying out for a class: the Agenda, which by composition contains the list and exposes add —which validates and avoids duplicates using __eq__—, remove, find, filter_by, sorted_tasks and summary_by_assignee, plus __len__ and __iter__ so that it behaves like a Python collection. The rule that holds all that protection together is never hand over the internal list, but a copy of it. And since json.dump cannot write objects, the round trip is done by hand with to_dict() and the @classmethod from_dict(), which also cleans up the old data by passing it through the constructor. Finally you have seen inheritance in its minimal form —RecurringTask(Task), super().__init__(...), overriding __str__— and polymorphism, which lets you walk a mixed list without a single if; with the warning that in practice composition is preferred, and that inheritance is reserved for when "an X is a Y" is true without forcing it.

EasyTask is now v0.16: Agenda.load() at start-up, agenda.save() on the way out, and a main() that now only deals with talking to the user. But the project now has a size problem: easytask.py piles up three classes, a dozen input and output functions, the constants, the persistence and the menu, all in a single file that is already hard to scan when you need to find where to make a change. Real programs are not written that way. In Modules, packages and imports we will see how to split the project across several files that import each other, what exactly happens when Python imports something —and at last, the full explanation of the if __name__ == "__main__": we have been carrying since 04-04—, how a package is organised, what the standard library brings and how third-party libraries are installed. EasyTask will stop being a file and become the easytask/ package.

Collections of Objects

Contents

List of objects versus list of dictionaries

Iterating, filtering and counting objects

Sorting objects

Searching objects and building indexes

Composition: the `Agenda` class

Why the agenda must protect its internal list

Serialising and rebuilding: `to_dict` and `from_dict`

Inheritance and polymorphism, the basics

EasyTask v0.16

Common Mistakes and Tips

Exercises

Exercise 1: Queries over a list of objects

Exercise 2: Extending the `Agenda` class

Exercise 3: A subclass with `super()`

Solutions

Conclusion

Fundamentals of Programming

Module 1: Introduction to Programming

Module 2: Core Concepts

Module 3: Control Structures

Module 4: Functions and Procedures

Module 5: Data Structures

Module 6: Basic Algorithms

Module 7: Objects and Code Organisation

Module 8: Good Practices and Tools

Module 9: Final Project and Course Wrap-up

Collections of Objects

Contents

List of objects versus list of dictionaries

Iterating, filtering and counting objects

Sorting objects

Searching objects and building indexes

Composition: the Agenda class

Why the agenda must protect its internal list

Serialising and rebuilding: to_dict and from_dict

Inheritance and polymorphism, the basics

EasyTask v0.16

Common Mistakes and Tips

Exercises

Exercise 1: Queries over a list of objects

Exercise 2: Extending the Agenda class

Exercise 3: A subclass with super()

Solutions

Conclusion

Fundamentals of Programming

Module 1: Introduction to Programming

Module 2: Core Concepts

Module 3: Control Structures

Module 4: Functions and Procedures

Module 5: Data Structures

Module 6: Basic Algorithms

Module 7: Objects and Code Organisation

Module 8: Good Practices and Tools

Module 9: Final Project and Course Wrap-up

Composition: the `Agenda` class

Serialising and rebuilding: `to_dict` and `from_dict`

Exercise 2: Extending the `Agenda` class

Exercise 3: A subclass with `super()`