We left the PersistentVolume pv-bookings-postgres-10gi in the Available phase, with an empty CLAIM column and 10 GiB waiting for someone. A PV does not mount itself: the application has to claim it, and that claim is a namespaced object called a PersistentVolumeClaim (PVC). This is the lesson where the module's debt is genuinely settled: you will write the PVC, understand precisely the algorithm by which the controller matches demand and supply, mount the volume in the bookings-postgres pod and run the test we have been promising since module 2 — create a booking, delete the pod and find it intact. You will also see what happens when you delete a PVC, why it sometimes gets stuck in Terminating, and why a Deployment with a ReadWriteOnce PVC cannot scale.

1. What a PersistentVolumeClaim is, exactly
2. Anatomy of the object, field by field
3. The matching algorithm
4. Why you can get more space than you asked for
5. The Pending state and its diagnosis
6. Consuming the PVC from the pod
7. The definitive bookings-postgres manifest
8. The demonstration: the data survives the pod
9. The 1:1 relationship and the danger of two pods on one RWO
10. Deleting a PVC: finalizers and Terminating
11. Why a Deployment with a PVC does not scale

A PersistentVolumeClaim is the storage request an application makes. The analogy that works best is the one with the compute resources you already know from 03-04:

Its two defining properties:

1. It is namespaced. It lives next to the application using it, it is affected by the namespace's ResourceQuota (03-04) and it disappears if the namespace is deleted. A pod can only mount PVCs from its own namespace: there is no way of referencing a PVC from another one.
2. It is the only storage piece the application team writes. On a cluster with dynamic provisioning properly configured, a developer never writes a PersistentVolume; they write PVCs.

The fields, with their exact meaning:

Two fields deserve a separate comment.

It works just like the selectors of 02-07, with matchLabels and matchExpressions, and it applies to the PV's labels. It serves to express requirements that capacity does not capture: matchExpressions: [{ key: disk-type, operator: In, values: ["ssd", "nvme"] }] means "I want a fast disk, I do not care which".

Careful: if the PVC carries a selector, dynamic provisioning cannot satisfy it (a freshly created volume will not have those labels), so the PVC will stay Pending forever on a cluster that only uses StorageClasses. It is a field from the static world.

It ties the PVC to a specific PV by name, skipping the search. It is useful in a recovery — "I want exactly that volume, the one with the data" — and it is the counterpart of the claimRef you saw in 05-02: if both objects name each other, the binding is deterministic and nobody else can sneak in.

When you create a PVC without volumeName, the PersistentVolume controller (part of kube-controller-manager) looks for a candidate PV. The process, in order:

The five criteria, with no exceptions:

1. The same storageClassName, compared as an exact string. "" and absent are not the same thing.
2. The PV's capacity.storage ≥ the PVC's requests.storage. Greater or equal, never less.
3. The PVC's accessModes must be contained in the PV's. If the PVC asks for ["ReadWriteMany"] and the PV offers ["ReadWriteOnce"], there is no deal.
4. Identical volumeMode.
5. The PVC's selector, if there is one, must match the PV's labels.

Among all the candidates, the controller picks the one with the smallest capacity, so as to waste as little as possible. And once decided, the binding is bidirectional and exclusive: spec.claimRef is written on the PV and spec.volumeName on the PVC. From then on both objects are married and no other PVC can use that PV, even if there is space to spare.

From rule 2 follows a behaviour that takes people by surprise: requests.storage is a minimum, not an exact size. If you ask for 8 GiB and the only available PV has 10 GiB, you bind to it and kubectl get pvc reports CAPACITY: 10Gi. Practical consequences:

• There is no refund. The 2 GiB of difference are lost to the rest of the cluster: nobody else can use that PV.
• The ResourceQuota counts what was requested, not what was obtained. If the namespace has a quota of requests.storage: 20Gi (03-04), your 8 GiB PVC consumes 8 GiB of quota even though you enjoy 10.
• In dynamic provisioning this does not happen: the volume is created at exactly the requested size (rounded up to the provider's minimum). That is one more reason to prefer it.

What never happens is the opposite: a PVC is never bound to a PV smaller than it asked for.

Pending is the state you will see and diagnose most. It means one thing only: the PVC has not found a volume. The causes are few and the method for telling them apart is always the same.

The events at the end are the key:

And the diagnostic procedure, in order, when the event is not enough:

A Pending PVC drags the pod with it: it stays Pending too, with the scheduler event pod has unbound immediate PersistentVolumeClaims. Always diagnose the PVC first; the pod is only the symptom.

In the pod, the PVC is used through a volume of type persistentVolumeClaim. The volumes/volumeMounts mechanics of 05-01 do not change at all:

That is all. The pod mentions the PersistentVolume nowhere, nor the provider, nor the disk: only the PVC's name. That indirection is what makes the same manifest work in minikube and in production.

The time has come to delete the "PROVISIONAL" comment we have been dragging along since 02-05. First the PVC:

Bound on both sides. The PV's CLAIM column, empty in the previous lesson, now names the PVC with its namespace. And the complete Deployment:

When a block volume is formatted with ext4, its root contains a lost+found directory. And PostgreSQL's initdb refuses to initialise a data directory that is not empty:

There are two ways of solving it, and the manifest has both, which is not redundancy but belt and braces:

At Rutas Norte we adopt PGDATA as the main mechanism (it already came from module 3) because subPath has a serious drawback: a volume mounted with subPath is not resized automatically on some drivers, which complicates the expansion of 05-05. If you pick only one, pick PGDATA for PostgreSQL and keep subPath for the general case of other applications.

The PostgreSQL process does not run as root, but as the postgres user (UID 999 in the official image). A freshly created volume usually belongs to root:root, so the process cannot write:

securityContext.fsGroup: 999 makes the kubelet change the volume's owning group to 999 and add write permission to it on mounting. It is the standard solution and it is covered in depth in 08-02.

This is the module's moment. We apply and repeat exactly the experiment of 05-01, which back then lost all the data.

We create a fictitious booking:

And now the test. We delete the pod, just as a rollout, an eviction or a node failure would:

The bookings are still there, in a different pod, with another name and another IP. The module 2 debt is settled. And you can verify where the bytes really live with minikube ssh -p rutas-norte -- "sudo ls -l /data/bookings-postgres/pgdata": the files belong to UID 999 thanks to fsGroup, and they are in the node's directory, outside the container.

Raise the stakes if you like: delete the whole Deployment and apply it again. The data is still there, because the PVC and the PV do not depend on the Deployment.

The relationship between PVC and PV is strictly 1:1. A bound PV does not accept a second PVC even if it has 9 GiB out of 10 to spare. Check it:

Pending: the only PV of that class is Bound. Delete it and let us carry on.

Now the dangerous part. Several pods can indeed mount the same PVC, and that is where the trap of 05-02 reappears:

The error in the second case, which you will see as soon as you work with a multi-node cluster:

That message is a protection, not a breakdown: the attach controller prevents a block disk from being attached to two machines. The serious case is the first one, where nobody protects you. That is why, for any stateful workload on ReadWriteOnce, the mandatory combination at Rutas Norte is:

• replicas: 1
• strategy: Recreate (terminates the old pod before creating the new one)
• and, if the driver supports it, ReadWriteOncePod on both PV and PVC.

Deleting a PVC in use is an operation that almost never goes as the beginner expects, and for good reasons.

The command hangs. And in another terminal, the explanation:

kubernetes.io/pvc-protection is the storage-object-in-use protection. While an active pod is using the PVC, the finalizer is not removed and the object is not deleted. It is the finalizer mechanism of 01-06 applied to preventing someone from blowing away the storage of a running application.

The fix is to remove the consumer first, never to force the finalizer:

Never run kubectl patch pvc ... -p '{"metadata":{"finalizers":null}}'. It is the recipe doing the rounds on the internet for "unsticking" a PVC and what it achieves is leaving the real volume attached to a node with no object representing it: an invisible orphan that has to be cleaned up by hand at the provider.

When the PVC disappears, the PV's fate is decided by its reclaim policy (05-02):

In our case the PV is left Released, with the CLAIM column still showing rutas-norte-dev/bookings-postgres-data, and the bookings inside. To get back to the operational situation, you apply the rescue you already know and recreate the PVC and the Deployment:

That is the whole point of Retain on a database holding personal data. With Delete, this exercise would have ended with the loss of every Rutas Norte booking.

The last piece, and it is a fundamental limitation, not a detail. Try scaling:

On a multi-node cluster you would see two pods stuck with Multi-Attach error. In minikube, which has a single node, you would see something worse: all three pods start, mount the same data directory and PostgreSQL starts complaining that there is already an instance using that directory (or, with another, less protected engine, it corrupts the files silently).

The reason is structural: all the pods in a Deployment share exactly the same template, including the volumes list. If the template says claimName: bookings-postgres-data, all three replicas ask for that same PVC. A Deployment does not know how to give each replica its own volume.

The correct solution is the StatefulSet, which incorporates a volumeClaimTemplate: it creates one PVC per replica, with a stable name tied to the pod's identity, so that bookings-postgres-0 always finds its volume again. It is the object that opens module 6 in 06-01, and now you know exactly what problem it comes to solve.

In the meantime, the operational rule for Rutas Norte is clear and sufficient: bookings-postgres is a single-replica Deployment with strategy: Recreate and a ReadWriteOnce PVC. Put it back where it belongs:

Tips:

1. Go-to command for seeing supply and demand at once:

   kubectl get pvc -A -o custom-columns=\
   NS:.metadata.namespace,PVC:.metadata.name,STATE:.status.phase,\
   PV:.spec.volumeName,REQUESTED:.spec.resources.requests.storage,CLASS:.spec.storageClassName
   

2. Find out who is using a PVC before deleting it, filtering kubectl get pods -o json with jq on .spec.volumes[]?.persistentVolumeClaim.claimName.
3. Name PVCs by their function, not their size: bookings-postgres-data, not pvc-10gi. The size changes with expansion (05-05); the function does not.
4. Ask with room to spare but without exaggerating. In static provisioning you bind to the smallest PV that serves you; in dynamic you create exactly what you asked for and you will be able to expand it later, but never shrink it.

Apply the definitive bookings-postgres PVC and Deployment from section 7 in rutas-norte-dev and demonstrate persistence at three increasing levels:

1. Insert three fictitious bookings and remove them from the pod with kubectl delete pod.
2. Delete the whole Deployment and apply it again.
3. Delete the PVC (having deleted the Deployment first) and recover the data by rescuing the PV.

Document the state of the PVC and of the PV at each step.

Create these four PVCs in rutas-norte-dev knowing that the only available PV is pv-bookings-postgres-10gi (10Gi, RWO, class rutasnorte-fast, label disk-type: ssd) and that it is already Bound. For each one, predict whether it will end up Bound or Pending and why; then check it.

The team wants redis-cache to keep its RDB dump between restarts so as not to lose the seat-availability cache on every deployment. Write the PVC (redis-cache-data, 2 GiB) and the Deployment fragment that mounts it at /data, with the Rutas Norte labels.

Then answer, with reasoning: is it a good decision? Bear in mind what was decided in 03-05 about the nature of redis-cache.

Exercise 1

Summary of states:

Had the policy been Delete, the fourth step would have destroyed the data for good.

Exercise 2

All four stay Pending, each for a different reason, and for x in a b c d; do kubectl describe pvc test-$x -n rutas-norte-dev | tail -3; done confirms it with the same generic event FailedBinding: no persistent volumes available for this claim and no storage class is set — a reminder that the event says that there is no candidate, but not why: that you have to reason out yourself with the five criteria.

For A to bind, the PV would first have to be freed (delete the bookings-postgres PVC and clean up the claimRef), which illustrates the central limitation of static provisioning that 05-04 solves.

Exercise 3

Is it a good decision? It is acceptable, but it is not a priority and it has trade-offs. In 03-05 we classified redis-cache as Burstable precisely because it is expendable: what it holds — the cached seat availability — is recalculated by querying bookings-postgres. What is lost on a restart is not information, it is a spell of higher latency.

In favour: after a deployment, the cache starts warm and bookings-postgres does not get an avalanche of queries; on a bank-holiday weekend, that recalculation spike can be considerable.

Against, three things: the PVC turns it into a stateful workload, with replicas: 1 and Recreate compulsory; a persisted cache can serve stale data if the RDB is hours old and seat availability has changed, which is worse than having no cache; and if Redis holds any personal customer data, that volume falls within the scope of data protection and of the backups of 05-06.

Rutas Norte's decision: do not persist redis-cache. If the real problem is the query spike after a deployment, the correct solution is a controlled cache warm-up at start-up, not keeping potentially obsolete availability data.

The debt Rutas Norte had been dragging since module 2 is settled. You have written the PersistentVolumeClaim — the namespaced object that expresses the application's demand and the only storage piece an application team writes —, you have bound it to the previous lesson's PV and you have watched the CLAIM column fill up with Bound on both sides. And above all you have run the test: three bookings inserted, the pod deleted, the Deployment deleted, and the bookings intact in a new pod with another name and another IP. The bytes, verifiable on the node, live outside the container.

You know how that binding comes about. The matching algorithm demands five conditions — the same exact class, sufficient capacity, the PVC's access modes contained in the PV's, an identical volumeMode and a selector that matches — and, among the candidates, it picks the smallest, from which comes the behaviour that surprises people: requests.storage is a minimum and you can get more space than you asked for, with no refund possible and without the quota reflecting it. When there is no candidate, the PVC stays Pending, and you no longer diagnose that blindly: kubectl describe pvc and its events tell apart in seconds "there is no compatible PV", "the class does not exist", "waiting for the first consumer" — which is not an error — and "quota exceeded".

You have mastered the definitive database manifest and the three traps around it: the right mountPath (/var/lib/postgresql/data, not one level higher), the subdirectory via PGDATA or subPath that avoids the lost+found failure, and the fsGroup without which the non-root process cannot write to its own volume. You know the 1:1 relationship between PVC and PV — a bound PV does not accept a second claimant even with space to spare — and the difference between the two ways of breaking with ReadWriteOnce: on different nodes the Multi-Attach error protects you; on the same node nobody protects you, and that is why replicas: 1 with strategy: Recreate is not a preference but a requirement. And you know how to delete without destroying: the kubernetes.io/pvc-protection finalizer leaves the PVC in Terminating while a pod is using it, it is resolved by removing the consumer and never by forcing the finalizer; and what happens to the PV afterwards is decided by its reclaim policy, where Retain has let you recover the bookings from a deletion that with Delete would have been final.

The ceiling of this architecture has been noted: a Deployment has a single template, so all its replicas ask for the same PVC and therefore it does not scale; the answer is the volumeClaimTemplate of StatefulSets (06-01). And there is the other ceiling, the one that opened this lesson: we are still creating PersistentVolumes by hand, one by one, with the waste, the manual work and the orphans that entails. The solution is for the cluster to create the exact volume at the moment someone asks for it, and the object that makes it possible is the one we have spent three lessons naming in storageClassName without explaining: Storage Classes.