The previous lesson ended with a precise diagnosis: every volume we knew — emptyDir, configMap, secret, projected — dies with the pod, and hostPath survives but ties the data to one specific node and opens an unacceptable security hole. For bookings-postgres to keep the bookings, something different is needed: a storage object with a life of its own in the cluster, independent of any pod. That object is the PersistentVolume (PV). In this lesson you will see why Kubernetes separated it from consumption, take its specification apart field by field — including the access modes trap, which applies per node and not per pod —, walk through its phases with a state diagram, and create a real 10 GiB PersistentVolume for the Rutas Norte database in your practice cluster.

1. The real problem: who administers and who consumes
2. What a PersistentVolume is
3. Anatomy of the object: capacity and volumeMode
4. The access modes and the node trap
5. The reclaim policy: Retain, Delete and the obsolete Recycle
6. storageClassName, mountOptions and nodeAffinity
7. The possible backends and which modes each family supports
8. Lifecycle: the phases of a PV
9. What to do with a PV in Released
10. Hands-on: a 10 GiB PV for bookings-postgres
11. Why static provisioning does not scale

Before looking at a single YAML field, you need to understand what organisational problem this abstraction solves, because otherwise the PV/PVC pair looks like unnecessary bureaucracy.

At Rutas Norte there are two distinct roles and they do not overlap:

If the bookings-postgres manifest had to name an AWS disk directly by its identifier, three bad things would happen: the manifest would stop being portable between minikube, pre-production and production; the developer would need credentials and knowledge of the infrastructure; and any change of provider would force a rewrite of every manifest on the platform.

Kubernetes' solution is a two-sided contract:

The sentence that sums up the whole lesson: the PVC is the demand, the PV is the supply, and the controller matches them. Whoever deploys the application writes only the demand.

A PersistentVolume is an API object representing a specific piece of storage that already exists in the cluster, provisioned by an administrator or created dynamically by a StorageClass.

Three properties define its nature and it is worth nailing them down from the outset:

1. It has no namespace. It is a cluster-scoped resource, like nodes or StorageClasses. You can check it with what you learned in 02-06:

   kubectl api-resources | grep -E "^NAME|persistentvolume"
   

   NAME                    SHORTNAMES   APIVERSION   NAMESPACED   KIND
   persistentvolumeclaims  pvc          v1           true         PersistentVolumeClaim
   persistentvolumes       pv           v1           false        PersistentVolume
   

   The PVC is namespaced: it belongs to the application. The PV is not.
2. Its lifecycle is independent of the pod. You can delete the whole Deployment, the pod, even the PVC, and the PV may still exist with the data inside. That independence is exactly what was missing in 05-01.
3. It is a declarative object like any other. It has apiVersion: v1, kind: PersistentVolume, metadata, spec and status, exactly the model of 01-06. Explore it:

   kubectl explain pv.spec
   kubectl explain pv.spec.accessModes
   

A complete PV, with every field that matters, commented:

The size of the volume, in Kubernetes units (Gi = 2^30 bytes, G = 10^9 bytes; always use Gi). It is a declared figure, not a measured one: Kubernetes does not check that the disk really is that size. With a hostPath test backend you can write capacity: 10Gi over a directory on a 4 GiB disk and nobody will complain until it fills up. Its real function is to serve as a criterion in the matching with the PVC.

Today storage is the only resource supported in capacity; attributes such as IOPS are expressed through the StorageClass parameters (05-04).

In block mode the container does not use volumeMounts but volumeDevices, with devicePath: /dev/xvda instead of mountPath: the device appears raw, with no filesystem.

At Rutas Norte we use Filesystem for everything. Block only makes sense if the application knows how to write straight to the device, and PostgreSQL does not.

The accessModes declare in what ways the volume can be mounted. There are four:

The trap is in ReadWriteOnce, and almost everyone falls into it at least once. The "Once" refers to one node, not one pod. This means that two, three or twenty pods can mount the same RWO volume simultaneously in read-write mode if the scheduler places them on the same node. No layer of Kubernetes will prevent it.

The consequence for a database is direct and serious: if a rollout with the RollingUpdate strategy brings up a new bookings-postgres pod on the same node before terminating the old one, you will have two PostgreSQL processes over the same data directory. That is why bookings-postgres has had strategy: Recreate since module 2, and that is why accessModes: ["ReadWriteOncePod"] exists.

ReadWriteOncePod is the only guarantee that a single pod can use the volume; the second one stays Pending with an explicit event. It is a stable mode from Kubernetes 1.29 onwards and requires the CSI driver to support it. When the driver allows it, it is the right choice for a database with a single replica.

Two more warnings about the modes:

• The list is a declaration of capabilities, not an active restriction. A PV can declare ["ReadWriteOnce", "ReadOnlyMany"] to say it supports both uses; the PVC will pick one.
• What a backend actually supports does not depend on what you write. You can put ReadWriteMany on a PV backed by an AWS block disk, and Kubernetes will accept it without a murmur; the failure will show up when trying to mount it from the second node. The table in section 7 records what each family really supports.

persistentVolumeReclaimPolicy decides what happens to the volume and to the data when the PVC that was using it is deleted. It is the field with the greatest consequences in the object.

Recycle is obsolete and must not be used: it only worked with hostPath and NFS, it deleted through a helper pod with no guarantees whatsoever, and it does not fit the CSI model. Its replacement is dynamic provisioning (05-04): instead of recycling a used volume, a new, clean one is created.

The practical decision at Rutas Norte:

• bookings-postgres → Retain. The scenario to avoid is that someone deletes the rutas-norte-pro namespace by mistake — remember from 02-06 that kubectl delete ns drags the PVCs with it — and that this destroys every customer's data. With Retain, the PVC disappears but the volume is still there and can be recovered.
• Rebuildable volumes (caches, working spaces) → Delete. Recreating is cheaper than cleaning by hand, and Delete avoids being billed for orphaned disks.

An important detail that causes a lot of confusion: in dynamic provisioning the PV inherits the policy from the StorageClass, and most default cloud classes use Delete. That is, if you do nothing, deleting a production PVC deletes the disk. Check it with kubectl get storageclass -o custom-columns=NAME:.metadata.name,POLICY:.reclaimPolicy.

And on an existing PV it can be changed on the fly, which is the standard defensive manoeuvre before touching anything:

It is a matching label: a PVC only binds to a PV whose class matches exactly. Three situations to tell apart carefully (and which we will see again in 05-04):

A static PV can carry a storageClassName that does not correspond to any existing StorageClass (for example manual). That is common practice and perfectly valid: the string is only used to match supply and demand.

Options passed as-is to the filesystem mount — noatime and nodiratime to reduce writes, or hard and nfsvers=4.1 on NFS. Kubernetes does not validate them: if the backend does not support them, the mount fails and the pod stays in ContainerCreating with a FailedMount event.

It restricts from which nodes the volume can be mounted. It is mandatory on local volumes, because the disk is physically on one machine and the scheduler must know it so as not to place the pod where the volume does not exist.

This is a case of storage conditioning scheduling, a topic explored in depth in 06-05. In the cloud the mechanism is identical but the key is usually topology.kubernetes.io/zone: a disk created in eu-west-1a can only be mounted from nodes in that zone, and from there comes the volumeBindingMode problem that 05-04 will solve.

The PV spec contains exactly one backend key, which defines where the real storage comes from.

hostPath as the backend of a PV has the same dangers described in 05-01 and must only be used on single-node clusters such as your minikube. local is its serious cousin: it also uses a disk on the machine, but it is a type designed for production, it respects scheduling thanks to the mandatory nodeAffinity and it does not allow arbitrary system paths. Even so, a local volume does not survive the loss of the node: redundancy is the application's responsibility (PostgreSQL replication, for instance).

NFS is the simplest route to ReadWriteMany, useful for instance so that the three web-store replicas share a directory of route images. It is not suitable for PostgreSQL's data files: file locking over NFS is a classic source of corruption.

On a managed cluster you will hardly ever write a PV by hand; the provider's CSI driver will create them. When you inspect them, you will see a csi key with driver: ebs.csi.aws.com, the volumeHandle (the identifier of the real disk at the provider), the fsType and some volumeAttributes.

The CSI architecture is studied in depth in 05-05.

An indicative table; the final truth is set by the driver and its version.

The rule that follows: if you need ReadWriteMany, you need a shared filesystem, and that is an architecture and cost decision, not a YAML field. For bookings-postgres we do not need it: a database with a single replica wants ReadWriteOnce (or better, ReadWriteOncePod) over a fast block disk.

The PV has a status.phase field with four possible values:

Check on the cluster:

The CLAIM column tells you, for each PV, which PVC in which namespace has it taken. It is the at-a-glance view most used day to day.

This is an operational situation that always turns up and baffles people the first time. Someone deletes the bookings-postgres PVC; since the policy is Retain, the PV goes to Released. You create the same PVC again, with the same name and the same request… and it stays Pending forever, even though the PV it wants is right there with 10Gi free.

The cause is that the PV keeps in spec.claimRef the reference to the previous PVC, including its uid:

Even though the new PVC has the same name, its uid is different, so it does not match. And that block is deliberate: it is the protection that prevents one tenant's data from ending up mounted by mistake in another's pod.

The correct procedure has three routes, in order of preference:

a) Reuse the volume for the same purpose (the case of a recovery after an accidental deletion). The stale reference is removed and the PV returns to Available:

Now any compatible PVC can take it. The data is still inside.

b) Pre-assign it to a specific PVC, which is safer than leaving it loose. The claimRef is edited leaving namespace and name but without the uid: the PV will only accept that exact PVC.

c) Discard it. If the data is no longer worth anything, the PV is deleted and the real volume is cleaned up by hand (with Retain, deleting the PV object does not delete the data on the backend: in the cloud, the disk keeps being billed).

Operational tip: before reusing a Released PV that contains data, take a copy. Section 10 creates the volume; backups are studied in 05-06, but the rule applies from today.

Let us create the volume that settles the module's debt. In minikube the backing will be a node directory, and here hostPath is acceptable because it is a single-node test cluster; the manifest carries a comment making it explicit so that nobody copies it into production.

First we prepare the directory on the node:

In minikube, /data is a directory that persists across restarts of the virtual machine, unlike most of the filesystem. That is why it is the right place for this exercise.

Now the PersistentVolume:

Available: it exists, it is free and it is waiting for a PVC. Notice that there is no namespace at all in the output: it is a cluster resource.

Detailed view:

Two details worth reading:

• Finalizers: [kubernetes.io/pv-protection]. It is the mechanism from 01-06: it prevents the PV from being deleted while it is bound to a PVC. If you delete a PV in use, the object stays in Terminating until it stops being so.
• Empty Claim:. Nobody has claimed it yet. That gap will be filled by the PVC of the next lesson.

Now check what still does not happen: the bookings-postgres pod is still not using it. A PV does not mount itself; a PVC is needed to claim it. That is exactly the content of 05-03.

What you have just done is called static provisioning: a human creates the PV by hand, in advance, and waits for someone to claim it. It works, and for a directory in minikube or for a fixed storage array in your own data centre it is perfectly reasonable. But as a general model it fails in four places:

The solution is to reverse the order: instead of creating volumes just in case, create the exact volume at the moment someone asks for it. That is dynamic provisioning, and the object that makes it possible is the StorageClass, which we have already named in storageClassName without explaining it. You will see it in 05-04 and the machinery behind it, CSI, in 05-05.

The good news is that what you are learning does not change: in dynamic provisioning there are still PVs with exactly these fields — capacity, access modes, reclaim policy, node affinity —; the only difference is who writes them.

Operational tips:

1. Label your PVs. Without labels you cannot use them with a selector from a PVC nor filter them in an audit. Use the same conventions as the rest of Rutas Norte, including app.kubernetes.io/part-of.
2. Quick risk audit — which volumes would be destroyed on deleting their PVC:

   kubectl get pv -o custom-columns=\
   NAME:.metadata.name,POLICY:.spec.persistentVolumeReclaimPolicy,\
   STATE:.status.phase,CLAIM:.spec.claimRef.name | grep Delete
   

3. Before any delicate operation, change the policy to Retain with kubectl patch. It is reversible and it has saved many databases.
4. kubectl get pv,pvc -A in a single command gives you the full picture of the cluster's supply and demand.

Create the pv-bookings-postgres-10gi PersistentVolume from section 10 in your minikube. Then:

1. Check that it is a resource without a namespace and that its phase is Available.
2. Write a file inside the node directory and verify it is there.
3. Change its reclaim policy to Delete with kubectl patch and set it back to Retain, checking the change at each step.
4. Try to delete the PV and explain which finalizer appears in kubectl describe.

The platform team has to prepare storage for three needs. For each one, decide capacity, accessModes, persistentVolumeReclaimPolicy, volumeMode and the backend type, and justify every choice:

• A. The bookings-postgres data in production: 200 GiB, a single instance, containing personal customer data, performance-critical.
• B. A directory of route images that the three replicas of web-store must read simultaneously, and that an administration process updates once a day: 20 GiB.
• C. A 500 GiB working space for occupancy-reports to generate the nightly report and delete it when finished.

Write the manifest for case B.

Reproduce and solve the classic incident:

1. Create a 1 GiB PV pv-rescue-practice with Retain, class manual and a hostPath backing at /data/rescue.
2. Create a PVC called practice-data in rutas-norte-dev that binds it and write a witness file from a pod.
3. Delete the PVC and observe the PV's phase.
4. Create the same PVC again. What happens and why?
5. Fix it without losing the witness file.

Exercise 1

While the PV is Available the deletion is immediate: the finalizer is removed on its own. If it were Bound to a PVC, the object would stay in Terminating until the PVC disappeared. It is the volume-in-use protection, sister to the pvc-protection you will see in 05-03.

Exercise 2

Justifications:

• A. A single PostgreSQL instance: ReadWriteOncePod guarantees that no second pod can mount it, not even on the same node, which removes the risk of corruption from two processes. Retain because it contains personal customer data and an accidental deletion of the PVC must not destroy it. A block disk for performance: shared files add latency that is unacceptable for a database.
• B. Several readers on different nodes require ReadWriteMany, and that forces a shared filesystem: a block disk is no use here however much you write RWX in the YAML. Retain because the images are business content that is recoverable but expensive to regenerate.
• C. Completely rebuildable data: Delete avoids paying for 500 orphaned GiB every night. In fact, case C is the perfect candidate for a generic ephemeral volume (05-01), which does not even require creating the PV.

Exercise 3

It stays Pending. Why: the PV is in Released, not Available, because it keeps a spec.claimRef pointing at the previous PVC with its uid. The new PVC has the same name but a different uid, so the controller does not consider it a candidate. Besides, a PV in the Released phase is never offered for binding.

The witness file is still intact: Retain did its job. To clean up, delete the PVC, repeat the claimRef patch and delete the PV.

You have taken on board the abstraction that underpins all Kubernetes storage, and the first thing you have understood is why it exists: it separates whoever administers the storage — the platform team, which writes PersistentVolumes and StorageClasses, cluster resources — from whoever consumes it — the application team, which writes a PersistentVolumeClaim in its namespace without knowing what lies underneath. Thanks to that separation, the bookings-postgres manifest is identical in minikube and in production even though underneath there may be a node directory or a cloud SSD.

You know the PV specification in full. The capacity, which is declared and not measured. The volumeMode, with Filesystem for 99% of cases and Block for whoever manages their own I/O. The four access modes, and above all the trap that costs the most dearly: ReadWriteOnce means one node, not one pod, so two pods on the same machine can write at the same time over PostgreSQL's files; the real guarantee is ReadWriteOncePod, stable since 1.29. The reclaim policy, with Retain for anything holding business data, Delete for the rebuildable — and the warning that default cloud classes come with Delete, so deleting a PVC deletes the disk — and Recycle obsolete and replaced by dynamic provisioning. And storageClassName as a matching label, mountOptions with no prior validation and nodeAffinity mandatory on local volumes, the first example of storage conditioning scheduling.

You know which backends exist and what each family really supports — hostPath and local for testing and local disks, NFS as the simple route to ReadWriteMany, and CSI drivers in production — and you are clear that writing ReadWriteMany over a block disk does not make it shared. You have mastered the phases (Available, Bound, Released, Failed) and, very particularly, the operation that always shows up in a real incident: a PV trapped in Released by its stale claimRef, rescued by removing that field with kubectl patch --type=json without losing a single byte.

And you have created the volume: pv-bookings-postgres-10gi, Retain, ReadWriteOnce, class rutasnorte-fast, waiting in the Available phase with an empty CLAIM column. That empty gap is the subject of the next lesson. A PersistentVolume does not mount itself: someone has to claim it from the application's namespace, and that someone is the object that finally makes the Rutas Norte database persistent. We build it in Persistent Volume Claims, where we will also run the test the whole module has been promising: create a booking, delete the pod and find it intact.