An anomaly was flagged at the close of the previous lesson. In module 4 we installed cert-manager and started writing manifests with kind: Certificate and kind: ClusterIssuer. In module 5 we created kind: VolumeSnapshot objects. None of those types exists in Kubernetes: they do not come in the apiserver binary, they do not appear in the original API specification. And yet the cluster treats them exactly like a Deployment: kubectl get, kubectl describe, kubectl explain, field validation, RBAC access control, storage in etcd, kubectl edit.
That is not magic. It is the most important feature of Kubernetes from the ecosystem's point of view: the API can be extended. Anyone can add new object types and, from that moment on, the cluster manages them as if they had always been there.
In this lesson we will learn how to do it. We will define the RutaProgramada type for Rutas Norte — origin, destination, timetables, seats, vehicle — with real validation, and we will use it with the same tools as ever. And we will finish with the most important warning on the topic: a CRD with no controller behind it is just a database with a validation form. What puts the software behind it is the operator, and that is the subject of the next lesson.
Contents
- What extending the Kubernetes API means
- The three ways of extending Kubernetes
- Anatomy of the
CustomResourceDefinitionobject - The OpenAPI v3 schema: declarative validation
- Subresources:
statusandscale additionalPrinterColumns: a usefulkubectl get- A complete example: the
RutaProgramadaCRD - Using the resource like any native object
- Versioning and conversion
- When NOT to create a CRD
- What extending the Kubernetes API means
It helps to be clear about what the apiserver really is. It is not "the brain of Kubernetes": it is a REST server with storage in etcd, authentication, authorization, validation and change notification. The objects it serves — Pods, Services, Deployments — are data with a schema. All the intelligence lives in the controllers that watch that data and act.
Extending the API means teaching it a new object type. From that moment on, that type gets the whole apiserver infrastructure for free:
| What you get for free | What it means |
|---|---|
| REST endpoints | /apis/<group>/<version>/namespaces/<ns>/<plural> with GET, POST, PUT, PATCH, DELETE, WATCH |
| Persistence in etcd | High availability, transactions, revision history |
| Validation | Malformed manifests rejected before being stored |
| RBAC | Roles and permissions over your type, just like any other (08-01) |
Full kubectl |
get, describe, edit, apply, delete, explain, label, patch |
| Auditing | Every change is recorded in the audit log |
| Watch | Real-time notification of changes: the basis of controllers |
Standard metadata |
labels, annotations, ownerReferences, finalizers, resourceVersion |
That list is why the whole Kubernetes ecosystem is built this way. When cert-manager wanted to model "a certificate that must exist and be renewed", it did not invent a configuration format of its own or a separate server: it defined a Certificate type and let Kubernetes do the rest.
Custom resources you have already used in this course, probably without noticing:
| Resource | API group | Who provides it | Where it appeared |
|---|---|---|---|
Certificate, Issuer, ClusterIssuer |
cert-manager.io |
cert-manager | 04-05 |
VolumeSnapshot, VolumeSnapshotClass |
snapshot.storage.k8s.io |
snapshot-controller | 05-05 |
Backup, Restore, Schedule |
velero.io |
Velero | 05-06 |
ServiceMonitor, PrometheusRule |
monitoring.coreos.com |
Prometheus Operator | 07-03 |
IPPool, NetworkSet |
crd.projectcalico.org |
Calico | 04-01 |
You can see them in your own cluster:
NAME CREATED AT
certificates.cert-manager.io 2026-07-12T09:14:22Z
challenges.acme.cert-manager.io 2026-07-12T09:14:22Z
clusterissuers.cert-manager.io 2026-07-12T09:14:22Z
issuers.cert-manager.io 2026-07-12T09:14:23Z
volumesnapshotclasses.snapshot.storage.k8s.io 2026-07-19T11:02:41Z
volumesnapshotcontents.snapshot.storage.k8s.io 2026-07-19T11:02:41Z
volumesnapshots.snapshot.storage.k8s.io 2026-07-19T11:02:41ZAnd check which of the available types are native and which were added:
NAME SHORTNAMES APIVERSION NAMESPACED KIND
certificaterequests cr,crs cert-manager.io/v1 true CertificateRequest
certificates cert,certs cert-manager.io/v1 true Certificate
clusterissuers cert-manager.io/v1 false ClusterIssuer
issuers cert-manager.io/v1 true Issuer
- The three ways of extending Kubernetes
There are three mechanisms, with different purposes. It is worth telling them apart because they are often confused.
| Mechanism | What it adds | Complexity | When to use it |
|---|---|---|---|
| CustomResourceDefinition (CRD) | New object types, served by the apiserver itself | Low: one YAML | 95 % of cases |
| API aggregation layer | An API server of your own that the apiserver delegates to | High: you have to write and operate a server | Data that does not belong in etcd, or special read logic |
| Admission webhooks | Intercepting and modifying or rejecting existing objects | Medium: an HTTPS server | Validating or injecting over native or custom types |
CRD
You declare the schema in a YAML and the apiserver starts serving the type. The objects are stored in etcd like any other. It is what cert-manager, Velero, Prometheus Operator and practically the whole ecosystem use.
Limitations: the data lives in etcd (not the place for large data volumes or very frequent writes) and you cannot customise how it is read or stored.
Aggregation layer
You register an APIService object that tells the apiserver "for the group metrics.k8s.io, delegate to this Service". The apiserver acts as a proxy towards your server.
The canonical example is the metrics-server we enabled as an addon in 01-04: its metrics are volatile, high-frequency data that must not go into etcd, so they are served from memory through aggregation. We will look at it in 07-02.
It requires writing a complete API server (authentication, authorization, versioning) and operating it with high availability. The rule is simple: when in doubt, use a CRD.
Admission webhooks
They add no types: they intercept requests for types that already exist, on the way between the apiserver's validation and storage.
- Mutating webhook: it modifies the object. It is what a service mesh does when injecting a sidecar into every pod.
- Validating webhook: it accepts or rejects. It is what enforces complex security policies.
A CRD can have an associated validating webhook for rules the OpenAPI schema cannot express, such as "the arrival time must be later than the departure time" or "there cannot be two routes with the same code".
- Anatomy of the
CustomResourceDefinition object
CustomResourceDefinition objectA CRD is itself a Kubernetes object, from the apiextensions.k8s.io/v1 group. This is its structure, with the RutaProgramada example we will build up:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
# MANDATORY: the name must be exactly <plural>.<group>
name: rutasprogramadas.rutasnorte.example
spec:
group: rutasnorte.example
scope: Namespaced
names:
kind: RutaProgramada
listKind: RutaProgramadaList
plural: rutasprogramadas
singular: rutaprogramada
shortNames: ["route", "routes"]
categories: ["rutasnorte", "all"]
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
# ... the schema, section 4 ...group
The API group the type lives under. It must be a domain name you control or that is clearly yours, to avoid collisions. The usual convention: <product>.<your-domain>.
The group is combined with the version to form your objects' apiVersion: rutasnorte.example/v1.
names
| Field | What it is | Example |
|---|---|---|
kind |
The manifest's kind:, in singular CamelCase |
RutaProgramada |
listKind |
The list's kind; by convention <kind>List |
RutaProgramadaList |
plural |
The name in the API URL and in kubectl, in lower case | rutasprogramadas |
singular |
Singular alias for kubectl | rutaprogramada |
shortNames |
Abbreviations: kubectl get routes |
["route", "routes"] |
categories |
Groups for kubectl get <category> |
["rutasnorte", "all"] |
The categories are more useful than they look. With categories: ["rutasnorte"], a single command lists every custom resource on your platform:
Be careful about including all: it makes your objects show up in kubectl get all, which is already a noisy command. Use it only if your resource really is a first-class thing for the cluster's users.
scope
| Value | Meaning | Real examples |
|---|---|---|
Namespaced |
The object lives in a namespace | Certificate, VolumeSnapshot, RutaProgramada |
Cluster |
It is global to the cluster, with no namespace | ClusterIssuer, VolumeSnapshotClass, StorageClass |
The decision is important and cannot be changed afterwards without deleting the CRD (and with it all of its objects). The criterion: if different teams or environments must have separate, isolated versions of the resource, it is Namespaced. If it represents global infrastructure configuration, it is Cluster.
For RutaProgramada we choose Namespaced: the routes in rutas-norte-dev are test data and must not get mixed up with those in rutas-norte-pro.
versions
A list, because a CRD can serve several versions at once:
versions:
- name: v1alpha1
served: false # no longer served: old clients get an error
storage: false
schema: {...}
- name: v1beta1
served: true # served, for clients still using it
storage: false
schema: {...}
- name: v1
served: true
storage: true # EXACTLY ONE version may have storage: true
schema: {...}| Field | Meaning |
|---|---|
served |
Whether the apiserver accepts requests in that version |
storage |
Whether objects are stored in etcd with that schema. Only one version may have it |
The distinction is subtle but crucial. An object created as v1beta1 is converted to the storage version before being saved, and converted back when read as v1beta1. The data in etcd has a single shape; the versions are views over it. We will come back to this in section 9.
- The OpenAPI v3 schema: declarative validation
Without a schema, a custom resource would accept any YAML. With a schema, the apiserver validates before storing and rejects whatever does not fit, with a specific message. It is what makes a CRD feel like a native type.
The schema goes in versions[].schema.openAPIV3Schema and is a subset of OpenAPI v3.
Types and basic structure
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required: ["origin", "destination", "timetables", "seats"]
properties:
origin:
type: string
minLength: 2
maxLength: 60
seats:
type: integer
minimum: 1
maximum: 90
active:
type: boolean
default: true
timetables:
type: array
minItems: 1
maxItems: 24
items:
type: stringConstraints available per type:
| Type | Constraints | Typical use |
|---|---|---|
string |
minLength, maxLength, pattern, enum, format |
Codes, names, number plates |
integer |
minimum, maximum, exclusiveMinimum, multipleOf |
Seats, prices in cents |
number |
Same as integer, with decimals | Coordinates |
boolean |
— | Switches |
array |
minItems, maxItems, uniqueItems, items |
Timetables, stops |
object |
properties, required, additionalProperties |
Nested structures |
required and default
origin:
type: string
# no default: if it is in required, it is mandatory
active:
type: boolean
default: true # if not given, the apiserver fills it in
class:
type: string
enum: ["standard", "supra", "night"]
default: "standard"Default values are applied by the apiserver when storing, not by kubectl. Practical consequence: if you read the object after creating it, you will see the defaults already written. That makes them reliable for any consumer.
Validation by pattern and by enumeration
code:
type: string
# Rutas Norte format: two letters, hyphen, three digits (RN-041)
pattern: '^[A-Z]{2}-[0-9]{3}$'
vehiclePlate:
type: string
pattern: '^[0-9]{4}[A-Z]{3}$'
departureTime:
type: string
pattern: '^([01][0-9]|2[0-3]):[0-5][0-9]$'
operatingDays:
type: array
minItems: 1
maxItems: 7
uniqueItems: true
items:
type: string
enum: ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]The patterns use RE2 syntax (Go's), not PCRE. They allow no lookahead, no lookbehind and no back references. For more complex rules there are two options: CEL validation rules (x-kubernetes-validations) or a validating webhook.
x-kubernetes-preserve-unknown-fields
By default, the apiserver silently removes any field that is not in the schema. This is called pruning and it is one of the most baffling causes of "my field disappeared": you apply a manifest with a misspelt field, kubectl does not complain, and when you read the object the field is not there.
To allow arbitrary fields in one specific place:
externalMetadata:
type: object
x-kubernetes-preserve-unknown-fields: true
description: "Free-form data from the legacy sales system"Use it sparingly: every point where you put it is a point where you lose validation and where a typo goes unnoticed.
Descriptions
seats:
type: integer
minimum: 1
maximum: 90
description: "Total seats on the vehicle assigned to this route"The descriptions are not decorative: they are what kubectl explain returns. Writing them turns your CRD into a self-documenting type, and it is the difference between a resource people know how to use and one that requires reading the source code.
- Subresources:
status and scale
status and scale - name: v1
served: true
storage: true
subresources:
status: {}
scale:
specReplicasPath: .spec.assignedVehicles
statusReplicasPath: .status.activeVehicles
labelSelectorPath: .status.selector
schema:
openAPIV3Schema: {...}The status subresource
Enabling status: {} has three concrete effects:
- The
/statusendpoint is enabled, and it is updated independently of the main object. - Writes to the main object ignore changes to
.status. - Writes to
/statusignore changes to.spec.
That separation is not bureaucracy: it reflects the fundamental division of the declarative model we studied in 01-06.
spec |
status |
|
|---|---|---|
| Who writes it | The user or the deployment system | The controller |
| What it expresses | The desired state | The observed state |
| Versioned in Git | Yes | No |
| Can it be rebuilt | No: it is the intent | Yes: it is an observation |
Without this subresource, a controller wanting to update the status would have to update the whole object, and would risk overwriting a spec change the user had just made. And conversely: a kubectl apply by the user would wipe out the status the controller had just written.
Always enable it on any CRD that is going to have a controller.
A universal Kubernetes pattern for status is conditions:
status:
type: object
properties:
phase:
type: string
enum: ["Pending", "Scheduled", "Active", "Cancelled"]
seatsSold:
type: integer
conditions:
type: array
items:
type: object
required: ["type", "status"]
properties:
type:
type: string
status:
type: string
enum: ["True", "False", "Unknown"]
lastTransitionTime:
type: string
format: date-time
reason:
type: string
message:
type: stringIt is the same structure you have seen in kubectl describe pod (Ready, PodScheduled, Initialized) and in kubectl describe node. Following the convention makes your objects read like the native ones and makes generic tools such as kubectl wait --for=condition=... work on them.
The scale subresource
It enables kubectl scale on your own resource, and with it the possibility of a HorizontalPodAutoscaler (09-01) acting on it.
scale:
specReplicasPath: .spec.assignedVehicles # where the desired number is
statusReplicasPath: .status.activeVehicles # where the observed one is
labelSelectorPath: .status.selector # selector for the managed objectsWhat the command does is write a 3 into .spec.assignedVehicles. Whether that translates into three buses is the controller's job; the subresource only standardises the interface.
additionalPrinterColumns: a useful kubectl get
additionalPrinterColumns: a useful kubectl getWith nothing configured, kubectl get on a custom resource shows two useless columns:
With additionalPrinterColumns you declare what to show, through JSONPath expressions over the object:
additionalPrinterColumns:
- name: Code
type: string
jsonPath: .spec.code
- name: Origin
type: string
jsonPath: .spec.origin
- name: Destination
type: string
jsonPath: .spec.destination
- name: Seats
type: integer
jsonPath: .spec.seats
- name: Phase
type: string
jsonPath: .status.phase
- name: Sold
type: integer
jsonPath: .status.seatsSold
priority: 1 # only with -o wide
- name: Age
type: date
jsonPath: .metadata.creationTimestampThe result:
NAME CODE ORIGIN DESTINATION SEATS PHASE AGE
rn-041-bilbao-santander RN-041 Bilbao Santander 55 Active 3m
rn-088-oviedo-leon RN-088 Oviedo Leon 38 Active 3mDetails:
typeacceptsstring,integer,number,booleananddate. Withdate, kubectl formats it as a relative age ("3m", "2d").priority: 0(the default) always shows the column;priority: 1only with-o wide.- The
NAMEcolumn is always there and is not configurable.
It is a small detail with an enormous impact on usability. Mentally compare kubectl get pods — which shows READY, STATUS, RESTARTS and AGE — with what it would be if it only showed name and age.
- A complete example: the
RutaProgramada CRD
RutaProgramada CRDLet us put it all together. Rutas Norte wants to model its routes as Kubernetes objects, so that the operations team manages them with the same tools and the same GitOps flow as the rest of the platform.
# k8s/base/crd-rutaprogramada.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: rutasprogramadas.rutasnorte.example
labels:
app.kubernetes.io/part-of: rutas-norte
spec:
group: rutasnorte.example
scope: Namespaced
names:
kind: RutaProgramada
listKind: RutaProgramadaList
plural: rutasprogramadas
singular: rutaprogramada
shortNames: ["route", "routes"]
categories: ["rutasnorte"]
versions:
- name: v1
served: true
storage: true
subresources:
status: {}
additionalPrinterColumns:
- name: Code
type: string
jsonPath: .spec.code
- name: Origin
type: string
jsonPath: .spec.origin
- name: Destination
type: string
jsonPath: .spec.destination
- name: Seats
type: integer
jsonPath: .spec.seats
- name: Phase
type: string
jsonPath: .status.phase
- name: Sold
type: integer
jsonPath: .status.seatsSold
priority: 1
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
schema:
openAPIV3Schema:
type: object
description: "A scheduled bus route of Rutas Norte S.L."
required: ["spec"]
properties:
spec:
type: object
description: "Desired state of the route"
required: ["code", "origin", "destination", "timetables", "seats"]
properties:
code:
type: string
description: "Commercial code of the route, format XX-999 (e.g. RN-041)"
pattern: '^[A-Z]{2}-[0-9]{3}$'
origin:
type: string
description: "Town the journey starts from"
minLength: 2
maxLength: 60
destination:
type: string
description: "Town the journey ends at"
minLength: 2
maxLength: 60
durationMinutes:
type: integer
description: "Estimated journey time in minutes"
minimum: 10
maximum: 1440
default: 120
seats:
type: integer
description: "Total seats offered on each departure"
minimum: 1
maximum: 90
class:
type: string
description: "Commercial category of the service"
enum: ["standard", "supra", "night"]
default: "standard"
active:
type: boolean
description: "Whether the route currently accepts sales"
default: true
timetables:
type: array
description: "Daily departure times in HH:MM format"
minItems: 1
maxItems: 24
uniqueItems: true
items:
type: string
pattern: '^([01][0-9]|2[0-3]):[0-5][0-9]$'
operatingDays:
type: array
description: "Days of the week on which the route operates"
minItems: 1
maxItems: 7
uniqueItems: true
default: ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
items:
type: string
enum: ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
intermediateStops:
type: array
description: "Stops between origin and destination, in order"
maxItems: 20
items:
type: object
required: ["town", "minuteFromDeparture"]
properties:
town:
type: string
minLength: 2
maxLength: 60
minuteFromDeparture:
type: integer
minimum: 1
maximum: 1439
vehicle:
type: object
description: "Vehicle assigned to the route"
required: ["plate"]
properties:
plate:
type: string
description: "Spanish number plate with no separators (e.g. 4471BCD)"
pattern: '^[0-9]{4}[A-Z]{3}$'
model:
type: string
maxLength: 80
accessible:
type: boolean
default: true
basePriceCents:
type: integer
description: "Base ticket price in euro cents"
minimum: 0
maximum: 100000
legacySalesMetadata:
type: object
description: "Free-form fields from the legacy sales system"
x-kubernetes-preserve-unknown-fields: true
status:
type: object
description: "Observed state, written by the controller"
properties:
phase:
type: string
enum: ["Pending", "Scheduled", "Active", "Cancelled"]
seatsSold:
type: integer
minimum: 0
lastScheduledDeparture:
type: string
format: date-time
observedGeneration:
type: integer
description: "metadata.generation the controller last processed"
conditions:
type: array
items:
type: object
required: ["type", "status"]
properties:
type:
type: string
status:
type: string
enum: ["True", "False", "Unknown"]
lastTransitionTime:
type: string
format: date-time
reason:
type: string
maxLength: 128
message:
type: string
maxLength: 512From this instant on, RutaProgramada is a first-class type in the cluster. Nobody has restarted anything, and every kubectl in the world pointing at this cluster already knows about it.
Instances
# k8s/environments/pro/scheduled-routes.yaml
apiVersion: rutasnorte.example/v1
kind: RutaProgramada
metadata:
name: rn-041-bilbao-santander
namespace: rutas-norte-pro
labels:
app.kubernetes.io/part-of: rutas-norte
environment: pro
corridor: cantabrian
spec:
code: "RN-041"
origin: "Bilbao"
destination: "Santander"
durationMinutes: 95
seats: 55
class: "standard"
timetables: ["07:00", "09:30", "12:00", "15:30", "18:00", "20:30"]
operatingDays: ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
intermediateStops:
- town: "Castro Urdiales"
minuteFromDeparture: 40
- town: "Laredo"
minuteFromDeparture: 58
vehicle:
plate: "4471BCD"
model: "Setra S 415 (fictitious)"
accessible: true
basePriceCents: 1150
---
apiVersion: rutasnorte.example/v1
kind: RutaProgramada
metadata:
name: rn-088-oviedo-leon
namespace: rutas-norte-pro
labels:
app.kubernetes.io/part-of: rutas-norte
environment: pro
corridor: northwest
spec:
code: "RN-088"
origin: "Oviedo"
destination: "Leon"
durationMinutes: 130
seats: 38
class: "supra"
timetables: ["06:45", "14:15", "19:45"]
operatingDays: ["Mon", "Tue", "Wed", "Thu", "Fri"]
vehicle:
plate: "9902XKL"
accessible: true
basePriceCents: 1490rutaprogramada.rutasnorte.example/rn-041-bilbao-santander created
rutaprogramada.rutasnorte.example/rn-088-oviedo-leon createdValidation in action
This is the part that shows the schema is not decorative. A manifest with several errors:
apiVersion: rutasnorte.example/v1
kind: RutaProgramada
metadata:
name: invalid-route
namespace: rutas-norte-dev
spec:
code: "rn41" # does not match ^[A-Z]{2}-[0-9]{3}$
origin: "A" # minLength is 2
destination: "Gijon"
seats: 250 # maximum is 90
class: "premium" # not in the enum
timetables: ["25:00"] # invalid time
vehicle:
plate: "4471-BCD" # the hyphen is not allowedThe RutaProgramada "invalid-route" is invalid:
* spec.code: Invalid value: "rn41": spec.code in body should match '^[A-Z]{2}-[0-9]{3}$'
* spec.origin: Invalid value: "A": spec.origin in body should be at least 2 chars long
* spec.seats: Invalid value: 250: spec.seats in body should be less than or equal to 90
* spec.class: Unsupported value: "premium": supported values: "standard", "supra", "night"
* spec.timetables[0]: Invalid value: "25:00": spec.timetables[0] in body should match '^([01][0-9]|2[0-3]):[0-5][0-9]$'
* spec.vehicle.plate: Invalid value: "4471-BCD": spec.vehicle.plate in body should match '^[0-9]{4}[A-Z]{3}$'Six errors, all caught before anything is stored, with the exact path of the field and the rule broken. And the omissions too:
kubectl apply -f - <<'EOF'
apiVersion: rutasnorte.example/v1
kind: RutaProgramada
metadata:
name: incomplete-route
namespace: rutas-norte-dev
spec:
code: "RN-999"
origin: "Burgos"
EOFThe RutaProgramada "incomplete-route" is invalid:
* spec.destination: Required value
* spec.timetables: Required value
* spec.seats: Required valueThis free validation, without writing a line of code, is why it is worth investing time in a good schema.
- Using the resource like any native object
Everything you know about kubectl already works on RutaProgramada.
NAME CODE ORIGIN DESTINATION SEATS PHASE AGE
rn-041-bilbao-santander RN-041 Bilbao Santander 55 2m
rn-088-oviedo-leon RN-088 Oviedo Leon 38 2mThe PHASE column is empty because nobody writes .status: there is no controller. That emptiness is the lesson of section 10.
kubectl get routes -n rutas-norte-pro -l corridor=cantabrian
kubectl get rutaprogramada rn-041-bilbao-santander -n rutas-norte-pro -o yaml | head -30apiVersion: rutasnorte.example/v1
kind: RutaProgramada
metadata:
creationTimestamp: "2026-08-05T20:14:02Z"
generation: 1
labels:
app.kubernetes.io/part-of: rutas-norte
corridor: cantabrian
environment: pro
name: rn-041-bilbao-santander
namespace: rutas-norte-pro
resourceVersion: "184722"
uid: 3f1a9c04-8e2b-4c71-9a55-71b0e2d8c4f3
spec:
active: true # <- default applied by the apiserver
basePriceCents: 1150
class: standard
code: RN-041
destination: Santander
durationMinutes: 95Note active: true: it was not in the manifest and the apiserver wrote it from the schema's default.
kubectl explain
GROUP: rutasnorte.example
KIND: RutaProgramada
VERSION: v1
FIELD: vehicle <Object>
DESCRIPTION:
Vehicle assigned to the route
FIELDS:
accessible <boolean>
model <string>
plate <string> -required-
Spanish number plate with no separators (e.g. 4471BCD)The documentation comes straight from the schema. A colleague who has never seen this CRD can discover it on their own, with no code to read and no wiki to hunt down.
describe, edit, patch, label
Name: rn-088-oviedo-leon
Namespace: rutas-norte-pro
Labels: app.kubernetes.io/part-of=rutas-norte
corridor=northwest
environment=pro
API Version: rutasnorte.example/v1
Kind: RutaProgramada
Spec:
Active: true
Base Price Cents: 1490
Class: supra
Code: RN-088
Destination: Leon
Duration Minutes: 130
Operating Days: Mon, Tue, Wed, Thu, Fri
Origin: Oviedo
Seats: 38
Timetables: 06:45, 14:15, 19:45
Vehicle:
Accessible: true
Plate: 9902XKL
Events: <none># Edit interactively, with validation on save
kubectl edit rutaprogramada rn-041-bilbao-santander -n rutas-norte-pro
# Patch a field
kubectl patch rutaprogramada rn-041-bilbao-santander -n rutas-norte-pro \
--type=merge -p '{"spec":{"basePriceCents":1250}}'
# Label it
kubectl label rutaprogramada rn-041-bilbao-santander -n rutas-norte-pro season=summer
# Watch changes in real time: the basis of any controller
kubectl get routes -n rutas-norte-pro --watchWriting the status
Since we enabled the subresource, the status is written through its own endpoint. A controller would do this through the API; by hand, with kubectl:
kubectl patch rutaprogramada rn-041-bilbao-santander -n rutas-norte-pro \
--subresource=status --type=merge -p '{
"status": {
"phase": "Active",
"seatsSold": 37,
"observedGeneration": 1,
"conditions": [{
"type": "VehicleAssigned",
"status": "True",
"reason": "ValidPlate",
"message": "Vehicle 4471BCD assigned and available",
"lastTransitionTime": "2026-08-05T20:20:00Z"
}]
}
}'
kubectl get routes -n rutas-norte-pro -o wideNAME CODE ORIGIN DESTINATION SEATS PHASE SOLD AGE
rn-041-bilbao-santander RN-041 Bilbao Santander 55 Active 37 8m
rn-088-oviedo-leon RN-088 Oviedo Leon 38 8mAnd now even kubectl wait works on a condition of your own:
kubectl wait --for=condition=VehicleAssigned \
rutaprogramada/rn-041-bilbao-santander -n rutas-norte-pro --timeout=30sThat is what following the conventions buys you: generic tools that never knew anything about bus routes work on your type.
RBAC over your own resource
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: routes-manager
namespace: rutas-norte-pro
rules:
- apiGroups: ["rutasnorte.example"]
resources: ["rutasprogramadas"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: ["rutasnorte.example"]
resources: ["rutasprogramadas/status"]
verbs: ["get", "update", "patch"]Note that rutasprogramadas/status is a separate resource for permission purposes: you can grant write access to the spec without allowing anyone to fake the status. That granularity is enabled by the subresource, and it is another reason to turn it on. RBAC in depth is lesson 08-01.
- Versioning and conversion
Versioning a CRD is the hardest part, and it is worth knowing about before publishing the first version.
The usual progression
| Version | Stability | Commitments |
|---|---|---|
v1alpha1 |
Experimental | May change or disappear without notice; disabled by default in many projects |
v1beta1 |
In testing | Incompatible changes possible but announced; there is usually a migration |
v1 |
Stable | Backwards compatibility guaranteed; existing fields do not change meaning |
Once at v1, you cannot delete a field or change its semantics. You can add optional fields with a default value, and little else. That is why it is worth starting at v1alpha1 and not promoting to v1 until the model has settled.
The problem that makes migration hard
Let us recall the mechanism: only one version has storage: true. All the objects are stored with that version's schema, and the other versions are views converted on the fly.
Suppose v1alpha1 had a single field timetable and v1 replaces it with a list timetables. How is an object stored with the old schema converted?
conversion: None (the default)
Nothing is converted: the object is returned as it is, only the apiVersion changing. It works only if the schemas are compatible field by field, that is, if between versions you have only added optional fields. For any structural change it is not enough.
conversion: Webhook
spec:
conversion:
strategy: Webhook
webhook:
conversionReviewVersions: ["v1"]
clientConfig:
service:
namespace: rutas-norte-sistema
name: routes-conversion-webhook
path: /convert
port: 443
caBundle: <certificate in base64>The apiserver calls your HTTPS server every time somebody reads or writes an object in a version other than the storage one. Your server receives the object in one version and returns it in another.
What that means in practice:
- You have to write, deploy and operate an HTTPS server with a valid certificate (here the cert-manager of 04-05 is the natural ally).
- It must be fast and highly available: if the webhook does not answer, nobody can read or write those objects. A conversion webhook that is down blocks the whole resource.
- The conversion must be bidirectional and lossless. If
v1alpha1.timetable(a string) is converted tov1.timetables(a list of one) and back again, you have to decide what happens when the list has three elements. The convention is to store what does not fit in an annotation.
The complete migration
# 1. Publish the new version and serve it, without changing storage
# versions: v1alpha1 (served, storage) + v1 (served)
# 2. Move storage to v1
# versions: v1alpha1 (served) + v1 (served, storage)
# 3. Rewrite every existing object so that it is stored with the new schema
kubectl get rutasprogramadas -A -o json | kubectl replace -f -
# 4. Check which versions are still registered as stored
kubectl get crd rutasprogramadas.rutasnorte.example \
-o jsonpath='{.status.storedVersions}{"\n"}'# 5. Once every object is on v1, remove v1alpha1 from storedVersions
# by editing .status.storedVersions, and only then stop serving it.Step 3 is the one people forget. While storedVersions contains the old version, you cannot remove it from the CRD: the apiserver refuses, because there would be objects in etcd it no longer knew how to interpret.
Practical conclusion: design the schema carefully from the start. It is far cheaper to spend two days thinking about the data model than to operate a conversion webhook for years.
- When NOT to create a CRD
CRDs are easy to create and that is why too many get created. Before writing one, put it through this filter.
Do not create one if a ConfigMap will do
If all you need is to store configuration that somebody reads, a ConfigMap (03-01) does the job with zero infrastructure.
| Sign | Tool |
|---|---|
| Configuration data an application reads at start-up | ConfigMap |
| Data a controller must continuously reconcile | CRD |
| A simple structure, one consumer, no validation | ConfigMap |
| A complex structure several teams write and that must be validated | CRD |
| You need granular RBAC per data type | CRD |
You need watch, status and conditions |
CRD |
Do not create one if nobody is going to reconcile anything
This is the central warning of the lesson, and it deserves stating bluntly:
A CRD with no controller is a database with a validation form.
Our RutaProgramada is created. The two routes exist in etcd. They are validated, labelled, versioned in Git, they show up in kubectl get. And absolutely nothing happens. No bus sets off. No Deployment is created. The PHASE column stays empty except when we fill it in by hand.
The value of a custom resource is not in the resource: it is in the reconciliation loop that watches it and acts. Without it you have a validated YAML, and you get that more cheaply with a JSON schema in your repository and a check in the CI pipeline.
Sanity-check questions:
- Who is going to watch this resource and what will they do? If there is no concrete answer, do not create the CRD.
- What will it write into
.status? If nothing, you probably wanted a ConfigMap. - What happens if somebody deletes it? If the answer is "nothing", it is not a Kubernetes resource: it is a document.
Other warning signs
- High-frequency data. etcd is not a time-series database. A CRD updated every second will degrade it. That is what the aggregation layer (metrics-server) or an external system (Prometheus, 07-03) is for.
- Too many objects. Thousands of instances of a CRD take up etcd and slow down
listcalls. If you expect tens of thousands, rethink the model. - Large objects. The practical limit for an object in etcd is roughly 1 MiB. A CRD is no place to attach files.
- Data edited by hand constantly. If a person is going to edit the resource ten times a day, perhaps what you need is an application with an interface, not a CRD.
- Modelling pure business domain. Here it is worth being honest about our own example: modelling bus routes as Kubernetes objects makes sense if the routes translate into cluster resources (a sales Deployment per route, a reports CronJob per corridor). If they are just rows in a table, their place is
bookings-postgres, not etcd.
Alternatives before you decide
| Need | Alternative to a CRD |
|---|---|
| Per-environment configuration | ConfigMap + Kustomize (10-04) |
| Manifest templating | Helm (10-03) |
| Manifest validation in the pipeline | JSON schemas + kubeconform in CI |
| Policies over existing objects | An admission webhook or Kyverno |
| Business data | A database |
Common Mistakes and Tips
Naming the CRD wrongly. metadata.name must be exactly <plural>.<group>. If it is not, the apiserver rejects the object with a fairly misleading message.
Fields that disappear without warning. The apiserver prunes anything not in the schema, silently. If a field "does not get saved", it is almost always misspelt or missing from the schema. Compare with kubectl get ... -o yaml.
Forgetting the status subresource. Without it, the controller and the user overwrite each other's writes. Enable it from day one: adding it later means reviewing all of the controller's code.
Starting straight at v1. It ties you to backwards compatibility forever with a model you have not tested yet. Start at v1alpha1 while the design settles.
x-kubernetes-preserve-unknown-fields: true at the root. It disables validation for the entire object. If you need it, confine it to the specific subtree.
Regexes with PCRE syntax. The patterns use RE2. A (?=...) causes an error when creating the CRD, not when using it.
Deleting a CRD without thinking. kubectl delete crd <name> deletes every instance of that type in every namespace, with no confirmation. It is irreversible short of a backup.
Orphan objects after uninstalling an operator. If you delete the operator but not its CRDs, you are left with registered types nobody attends to. And if you delete the CRDs before the controller removes its finalizers, the objects hang in Terminating forever.
Tip: write descriptions for every field. They feed kubectl explain and make the CRD self-documenting. It is the best return on effort in the whole lesson.
Tip: add additionalPrinterColumns from the start. A kubectl get that only shows name and age makes an otherwise well-designed resource unusable.
Tip: test the validation with deliberately broken manifests. It is the only way to check the schema does what you think it does. Keep those manifests as regression tests for the CRD.
Tip: use kubectl explain --recursive to see the whole schema tree at a glance, very useful when reviewing somebody else's CRD.
Exercises
Exercise 1: create a CRD with validation
Create a CRD ParadaAutobus in the rutasnorte.example group, Namespaced scope, version v1, with short name stop and category rutasnorte. Its spec must have:
code(string, mandatory, pattern^P-[0-9]{4}$)town(string, mandatory, between 2 and 60 characters)platforms(integer, between 1 and 20, default 1)accessible(boolean, defaulttrue)services(an array of strings from the enumticket-office,cafeteria,left-luggage,wc, with no repetitions)
Add columns for code, town, platforms and age. Create a valid stop and an invalid one and check the error messages.
Exercise 2: the status subresource and conditions
Extend the previous CRD with the status subresource, with the fields operational (boolean), dailyPassengers (integer) and conditions (an array with the standard structure). Add an Operational column. Write the status through --subresource=status and check that a later kubectl apply of the spec does not wipe it.
Exercise 3: choosing between a CRD and a ConfigMap
For each of these four Rutas Norte cases, decide whether a CRD or a ConfigMap is appropriate and justify the answer in one or two sentences:
- The
web-storeinterface messages in Spanish, Catalan and English. - A definition of a "commercial corridor" which, on being created, must automatically trigger the deployment of a reports CronJob and a NetworkPolicy of its own.
- The list of domains allowed for CORS in
bookings-api. - An "ephemeral test environment" that a developer requests and that must create a namespace, a copy of
bookings-postgresfrom a snapshot, and delete itself after 7 days.
Solutions
Solution 1
# /tmp/crd-paradaautobus.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: paradasautobus.rutasnorte.example
labels:
app.kubernetes.io/part-of: rutas-norte
spec:
group: rutasnorte.example
scope: Namespaced
names:
kind: ParadaAutobus
listKind: ParadaAutobusList
plural: paradasautobus
singular: paradaautobus
shortNames: ["stop", "stops"]
categories: ["rutasnorte"]
versions:
- name: v1
served: true
storage: true
additionalPrinterColumns:
- name: Code
type: string
jsonPath: .spec.code
- name: Town
type: string
jsonPath: .spec.town
- name: Platforms
type: integer
jsonPath: .spec.platforms
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
schema:
openAPIV3Schema:
type: object
description: "A physical stop on the Rutas Norte network"
required: ["spec"]
properties:
spec:
type: object
required: ["code", "town"]
properties:
code:
type: string
description: "Stop code, format P-9999"
pattern: '^P-[0-9]{4}$'
town:
type: string
description: "Town where the stop is located"
minLength: 2
maxLength: 60
platforms:
type: integer
description: "Number of platforms available"
minimum: 1
maximum: 20
default: 1
accessible:
type: boolean
description: "Whether the stop is fully accessible"
default: true
services:
type: array
description: "Services available at the stop"
uniqueItems: true
maxItems: 4
items:
type: string
enum: ["ticket-office", "cafeteria", "left-luggage", "wc"]NAME SHORTNAMES APIVERSION NAMESPACED KIND
paradasautobus stop,stops rutasnorte.example/v1 true ParadaAutobusA valid stop:
kubectl apply -f - <<'EOF'
apiVersion: rutasnorte.example/v1
kind: ParadaAutobus
metadata:
name: p-0041-bilbao-termibus
namespace: rutas-norte-dev
labels:
app.kubernetes.io/part-of: rutas-norte
environment: dev
spec:
code: "P-0041"
town: "Bilbao"
platforms: 12
services: ["ticket-office", "cafeteria", "wc"]
EOF
kubectl get stops -n rutas-norte-devparadaautobus.rutasnorte.example/p-0041-bilbao-termibus created
NAME CODE TOWN PLATFORMS AGE
p-0041-bilbao-termibus P-0041 Bilbao 12 9sAn invalid stop:
kubectl apply -f - <<'EOF'
apiVersion: rutasnorte.example/v1
kind: ParadaAutobus
metadata:
name: bad-stop
namespace: rutas-norte-dev
spec:
code: "STOP41"
town: "X"
platforms: 50
services: ["ticket-office", "ticket-office", "parking"]
EOFThe ParadaAutobus "bad-stop" is invalid:
* spec.code: Invalid value: "STOP41": spec.code in body should match '^P-[0-9]{4}$'
* spec.town: Invalid value: "X": spec.town in body should be at least 2 chars long
* spec.platforms: Invalid value: 50: spec.platforms in body should be less than or equal to 20
* spec.services: Invalid value: ["ticket-office","ticket-office","parking"]: spec.services in body should have unique items
* spec.services[2]: Unsupported value: "parking": supported values: "ticket-office", "cafeteria", "left-luggage", "wc"Five different rules checked — pattern, length, range, uniqueness and enumeration — without a line of code.
# The default was applied too
kubectl get stop p-0041-bilbao-termibus -n rutas-norte-dev \
-o jsonpath='{.spec.accessible}{"\n"}'Solution 2
You add the subresources block, the status schema and the column to the CRD:
- name: v1
served: true
storage: true
subresources:
status: {}
additionalPrinterColumns:
- name: Code
type: string
jsonPath: .spec.code
- name: Town
type: string
jsonPath: .spec.town
- name: Platforms
type: integer
jsonPath: .spec.platforms
- name: Operational
type: boolean
jsonPath: .status.operational
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
schema:
openAPIV3Schema:
type: object
required: ["spec"]
properties:
spec:
# ... the same as in solution 1 ...
status:
type: object
description: "Observed state of the stop"
properties:
operational:
type: boolean
dailyPassengers:
type: integer
minimum: 0
conditions:
type: array
items:
type: object
required: ["type", "status"]
properties:
type:
type: string
status:
type: string
enum: ["True", "False", "Unknown"]
lastTransitionTime:
type: string
format: date-time
reason:
type: string
message:
type: stringkubectl apply -f /tmp/crd-paradaautobus.yaml
kubectl patch stop p-0041-bilbao-termibus -n rutas-norte-dev \
--subresource=status --type=merge -p '{
"status": {
"operational": true,
"dailyPassengers": 3184,
"conditions": [{
"type": "PlatformsAvailable",
"status": "True",
"reason": "AllFree",
"message": "12 of 12 platforms operational",
"lastTransitionTime": "2026-08-05T21:02:00Z"
}]
}
}'
kubectl get stops -n rutas-norte-devChecking the independence of spec and status:
# Reapply the original spec, which does NOT contain a status
kubectl apply -f - <<'EOF'
apiVersion: rutasnorte.example/v1
kind: ParadaAutobus
metadata:
name: p-0041-bilbao-termibus
namespace: rutas-norte-dev
labels:
app.kubernetes.io/part-of: rutas-norte
environment: dev
spec:
code: "P-0041"
town: "Bilbao"
platforms: 14
services: ["ticket-office", "cafeteria", "wc", "left-luggage"]
EOF
kubectl get stops -n rutas-norte-devThe platforms were updated to 14 and OPERATIONAL is still true: the apply did not touch the status, because the subresource isolates it. Without it, that apply would have wiped out all the state the controller had just calculated. This is exactly why it has to be enabled.
kubectl wait --for=condition=PlatformsAvailable \
stop/p-0041-bilbao-termibus -n rutas-norte-dev --timeout=10s# Clean-up (deletes the CRD and all of its instances)
kubectl delete crd paradasautobus.rutasnorte.exampleSolution 3
1. Interface messages in three languages → ConfigMap.
They are configuration data that web-store reads at start-up. Nobody has to reconcile anything: there is no desired state to chase, just text to mount as a volume or inject as variables. One ConfigMap per language, versioned in Git, solves the case with no extra infrastructure.
2. A commercial corridor that deploys a CronJob and a NetworkPolicy → CRD. This is the canonical case. There is a desired state ("the Cantabrian corridor exists") that must translate into real cluster resources, and something has to create them, keep them there if somebody deletes them by hand, and clean them up when the corridor disappears. That is a reconciliation loop, which means a CRD plus a controller. Without the controller the CRD would be worthless.
3. Domains allowed for CORS → ConfigMap.
It is a list of strings the application reads. If you also want to validate it, a JSON schema in the CI pipeline is cheaper than a CRD. The proof that it needs no CRD: nobody would write anything in its .status.
4. An ephemeral test environment with an expiry → CRD.
It requires active, continuous reconciliation: creating a namespace, restoring bookings-postgres from a snapshot (05-05), watching the clock and deleting everything after 7 days. The status would have meaningful fields (phase, expiryDate, namespaceCreated) and cascading deletion would be handled with ownerReferences. It is a textbook operator, and the next lesson explains how one is written.
Conclusion
Extending the Kubernetes API means teaching it a new object type, and in exchange that type inherits, for free, the REST endpoints, the persistence in etcd, the validation, the RBAC, the auditing, the watch and the whole of kubectl. It is the feature that explains the entire ecosystem: cert-manager's Certificates, module 5's VolumeSnapshots and Velero's Backups that you have already used are custom resources.
Of the three ways of extending — CRD, aggregation layer and admission webhooks —, the CRD covers the vast majority of cases and only requires a YAML. We have gone through its anatomy: group, names with its plurals and shortcuts, scope, and versions with the crucial distinction between served (it can be requested) and storage (it is stored that way, and only one version may have it).
The OpenAPI v3 schema is what turns a CRD into a real type: types, required, default, enum, pattern, ranges and x-kubernetes-preserve-unknown-fields when free-form fields have to be allowed. The subresources status — which separates the desired from the observed and stops the user and the controller from overwriting each other — and scale — which enables kubectl scale — and the additionalPrinterColumns complete the experience.
We have built the RutaProgramada CRD with real validation, checked that it rejects six different errors with precise messages, and handled it with get, describe, explain, edit, patch, label, wait and RBAC exactly as we would a Deployment. We have also seen why versioning is the hard part, and why a conversion webhook that is down blocks the whole resource.
And we have ended up where we had to end up: our two routes exist, they are validated and they show up in kubectl get, but nothing happens. The PHASE column is empty because nobody writes it. A CRD with no controller is a database with a form.
What is missing is the software that watches those objects, compares the desired with the observed and acts: the reconciliation loop we have known since 01-02. A custom resource plus a controller that reconciles it is, precisely, an operator. It is what cert-manager does with Certificates, what a RutaProgramada controller would do, and what will finally solve the gap we left open in 06-01: that bookings-postgres is still a hand-rolled StatefulSet that cannot replicate itself or fail over. That is the subject of the next lesson, the last of the module: Operators and the Controller Pattern.
Kubernetes Course
Module 1: Introduction to Kubernetes
- What Is Kubernetes?
- Kubernetes Architecture
- Key Concepts and Terminology
- Setting Up a Kubernetes Cluster
- The Kubernetes CLI: kubectl
- Objects, YAML Manifests and the Declarative Model
- The Course Project: the Rutas Norte Platform
Module 2: Core Kubernetes Components
- Pods
- ReplicaSets
- Deployments
- Updates, Rollbacks and Deployment Strategies
- Services
- Namespaces
- Labels, Selectors and Annotations
Module 3: Configuration and Secret Management
- ConfigMaps
- Secrets
- Environment Variables
- Resource Quotas and Limits
- LimitRanges and Quality of Service (QoS) Classes
- ServiceAccounts and API Access from Pods
Module 4: Networking in Kubernetes
- Cluster Networking
- Service Types
- Internal DNS and Service Discovery
- Ingress Controllers
- TLS and Certificate Management with cert-manager
- Network Policies
Module 5: Storage in Kubernetes
- Volumes
- Persistent Volumes
- Persistent Volume Claims
- Storage Classes
- Dynamic Provisioning, Expansion and Snapshots
- Backup and Restore of Persistent Data
Module 6: Advanced Kubernetes Concepts
- StatefulSets
- DaemonSets
- Jobs and CronJobs
- Init Containers, Sidecars and Multi-Container Patterns
- Scheduling: Affinity, Taints and Tolerations
- Custom Resource Definitions (CRDs)
- Operators and the Controller Pattern
Module 7: Monitoring and Logging
- Health Checks and Probes
- Metrics Server and kubectl top
- Monitoring with Prometheus
- Visualization and Alerting with Grafana and Alertmanager
- Centralized Logging with Elasticsearch, Fluentd and Kibana (EFK)
- Application Debugging and Cluster Events
Module 8: Kubernetes Security
- Role-Based Access Control (RBAC)
- Security Contexts and Container Hardening
- Pod Security Policies and Pod Security Standards
- Network Security
- Image Security
- Auditing, Scanning and Vulnerability Management
Module 9: Scaling and Performance
- Horizontal Pod Autoscaling
- Vertical Pod Autoscaling
- Cluster Autoscaling
- Event-Driven and Custom-Metric Scaling with KEDA
- High Availability: PodDisruptionBudgets and Topology
- Performance Tuning
Module 10: Kubernetes Ecosystem and Tooling
- Minikube and Local Environments with kind
- Kubeadm
- Helm
- Kustomize
- GitOps with Argo CD and Flux
- Managed Kubernetes: EKS, AKS and GKE
Module 11: Case Studies and Real-World Applications
- Deploying a Web Application
- Running Stateful Applications
- CI/CD with Kubernetes
- Deployment Strategies: Blue-Green and Canary
- Multi-Cluster Management
- Production Operations: Incidents, Runbooks and Costs
