In the previous lesson we used kubectl apply as if it were an obvious command and we saw YAML manifests in passing, without breaking them down. It is time to settle that debt, because the manifest is Kubernetes' real unit of work: everything you deploy across the eleven remaining modules will be a YAML file describing an object. This lesson teaches you the exact anatomy of any API object —the four fields that are always there, whatever you do—, how to find out which apiVersion corresponds to each type, the subset of YAML you need to master, the real difference between apply, create and replace and why it matters, how to validate a manifest before applying it, and how to organise the Rutas Norte project files in the k8s/ directory so that the repository is the source of truth.

1. Anatomy of a Kubernetes object
2. API groups, versions and stability levels
3. The YAML you need to know
4. Desired state and observed state
5. apply versus create and replace
6. Server-side apply and field ownership
7. Validating before applying
8. Organising the project manifests

Absolutely every Kubernetes object —a pod, a secret, an RBAC permission, an operator's CRD— shares the same structure of four top-level fields. Learning that structure once serves you forever.

It states the API group and version that must interpret the object. It determines which fields are valid and how they behave. Examples: v1 (core group), apps/v1, networking.k8s.io/v1, batch/v1. It is the field that causes the most errors when old manifests are copied from the internet.

The specific type: Pod, Deployment, Service, Ingress, ConfigMap. It is always written in CamelCase, whereas the resource in the API URL goes in lowercase and plural (pods, deployments). The apiVersion + kind pair uniquely identifies the object's schema.

The object's identity. Its most relevant fields:

One detail that explains a baffling behaviour: if a namespace or a PVC gets "stuck" in Terminating, it is almost always because of a finalizer whose owner has not finished its work.

The desired state. It is what you describe and its content depends entirely on the kind: a Pod's spec has containers, a Service's has selector and ports, a PVC's has resources.requests.storage. The whole course consists, at bottom, of learning to write spec for different types.

It does not appear in the manifest above because you do not write it: the cluster does. It is the observed state. If you query the object once created you will see it:

Note two things. First: conditions is the standard format with which Kubernetes expresses "how an object is doing", and it is what kubectl wait and deployment tools consult. Second: imageID contains the real digest of the image being run, whereas spec.image contains the tag you asked for. When somebody reuses a tag, those two values stop matching; we will come back to this when we talk about immutable tags in the next lesson.

The Kubernetes API is split into groups so it can evolve in parts and allow extensions.

The core group is the historical one and that is why its apiVersion carries no prefix: it is v1, not core/v1. All the others follow the group/version format.

There is no need to memorise this table: you query it.

This is the reflex to acquire: before copying an apiVersion from a blog, check it with kubectl api-resources. The right version is always the one your cluster states.

Kubernetes retires old versions with each upgrade, and that is why manifests copied from old tutorials fail:

The extensions/v1beta1 group disappeared many versions ago. Today a Deployment is apps/v1 and an Ingress is networking.k8s.io/v1, with no exceptions. Before upgrading a cluster, tools such as kubent (kube-no-trouble) scan your manifests for APIs about to be retired.

YAML is simple but it has traps. This is the subset you need.

Non-negotiable rules:

• Spaces, never tabs. A tab is a syntax error in YAML. Configure your editor to expand tabs to 2 spaces in .yaml files.
• Indentation defines the hierarchy. Two spaces per level is the Kubernetes convention.
• The hyphen - marks a list element and its content is indented to the same level as the hyphen or deeper.

The NO case is famous (it is known as the Norway problem): YAML 1.1 interprets yes, no, on, off, y, n as booleans. Practical rule: in a ConfigMap or Secret, always quote the values, because those fields demand strings and a badly typed value produces an error as clear as this one:

Essential for embedding configuration files in a ConfigMap:

The --- separator lets you put several objects in the same file, and kubectl apply processes them in order:

Recommended criterion: group in one file the objects that form a deployable unit (a component with its ConfigMap and its Service) and split different components into different files. A single giant file with the whole platform is hard to review and hard to apply partially.

This is where the declarative model becomes concrete. You already know the reconciliation loop from the lesson Kubernetes Architecture; let us look at it from the file's point of view.

Four practical consequences worth being very clear about:

1. You write spec; the system writes status. Editing a status by hand achieves nothing: the controller will overwrite it on its next cycle.
2. Applying is not executing. kubectl apply finishes when the object is stored, not when the application is ready. To really wait:

3. It is idempotent. Applying the same file ten times gives the same result as applying it once. This is what makes it possible for a CI/CD pipeline to run kubectl apply on every deployment without checking first what exists.

4. The file is the source of truth, not the cluster. If somebody changes something with kubectl edit, the next apply from the repository reverts it. That is exactly the property GitOps is built on (module 10).

Three commands that look interchangeable and are nothing of the sort.

The operational conclusion is simple: always use apply. create is for quick imperative commands (kubectl create namespace) and replace for very specific cases of total replacement.

Imagine you apply a pod with two labels, then edit the file leaving only one, and apply again. How does kubectl know it must remove the second label, rather than leaving it because "it is not in the file but I have not asked for it to be deleted either"?

The classic answer is that apply stores a copy of what you last applied in an annotation:

With that information, kubectl performs a three-way merge: it compares (a) what you applied last time, (b) what you are applying now and (c) what is in the cluster. Whatever was in (a) and disappears in (b) is deleted; whatever is only in (c) —because a controller or a webhook put it there— is respected.

This explains a frequent error: if you created an object with kubectl create and later modify it with apply, the reference annotation does not exist and kubectl warns that the merge may be incorrect. For consistency, use apply from the very beginning of each object's life.

Since Kubernetes 1.22, the merge can be done by the server rather than the client, and it is the mechanism the whole ecosystem is migrating towards.

The difference is conceptual: instead of an annotation with the last state, the server records which manager owns each field in metadata.managedFields.

Advantages over client-side merging:

• It detects conflicts. If you try to modify a field belonging to another manager (an HPA that controls replicas, for example), the request fails with a clear warning instead of causing a silent fight between tools:

If you really do want to take ownership of the field, you force it explicitly:

• It does not depend on an annotation that on large objects can be enormous.
• Several managers can coexist on the same object in an orderly way: your pipeline, an operator and an autoscaler, each owning its own fields.

For this course we will keep using plain kubectl apply, which is the usual choice, but it pays to recognise managedFields when it turns up in a -o yaml and to know what a conflict error means.

Applying without checking is a recipe for an incident. Kubernetes offers four levels of verification, from cheapest to most complete.

kubectl builds the object and validates the schema without sending anything. It catches YAML syntax errors and non-existent fields. It does not catch permission problems, quotas or webhooks. It is also the flag we used in the previous lesson to generate manifests:

The request does reach the apiserver and travels the whole path from lesson 01-02 —authentication, RBAC authorization, admission, validation— but it is not written to etcd. It catches what the client cannot see: missing permissions, exceeded quotas, pod security policies, your own admission webhooks. It is the validation your CI pipeline should run before deploying.

This is the command that turns a deployment into something reviewable: it tells you exactly what is going to change in the cluster before you change it. It should be a mandatory step in any manual deployment to production. Its exit code is non-zero when there are differences, which makes it usable in scripts.

We saw it in the previous lesson; here it is worth stressing that it is the right answer to "does this field exist and what type is it?" while you are writing a manifest.

The Rutas Norte manifests live in the project repository's k8s/ directory, versioned in Git. This is the structure we will use throughout the course:

Conventions we will follow:

About the order: applying a whole directory works because the model is reconciling —a pod that cannot find its ConfigMap waits and starts when it appears— but to avoid transient errors it is best to apply the namespace first:

An important warning, so as not to trespass on later modules: this base/ and environments/ structure is deliberately plain YAML, with no templating. When duplication between environments becomes annoying, there are two standard solutions —Helm (templating and packaging) and Kustomize (overlays with no templating)— covered in the lessons Helm and Kustomize. Learning plain YAML first is not wasted time: it is what will let you understand what those tools generate.

• Using tabs in YAML. A pure syntax error. Configure the editor: expandtab, 2 spaces, and a YAML extension that validates on save.
• Copying apiVersion from old tutorials. extensions/v1beta1 and apps/v1beta2 do not exist. Always check with kubectl api-resources.
• Confusing the kind name with the resource name. kind: Deployment (CamelCase, singular) in the YAML; kubectl get deployments (lowercase, plural) in the CLI.
• Unquoted values in ConfigMaps and Secrets. "3", "true", "NO". Those fields demand strings and YAML converts types cheerfully.
• Editing status by hand. It has no effect whatsoever: the controller writes it.
• Mixing create and apply on the same object. Without the last-applied annotation, the merge can behave unexpectedly. Always start with apply.
• Applying without diff in production. kubectl diff costs two seconds and prevents surprise deployments.
• Saving a -o yaml as a manifest. The cluster's output includes status, uid, resourceVersion, managedFields and creationTimestamp, which must not be versioned. Clean it (by hand or with the kubectl neat plugin) before saving it into k8s/.
• Tip: treat the k8s/ directory as code. Pull request review, CI validation with --dry-run=server, and the golden rule: if a change is not in Git, it does not exist.

1. Write the manifest of a ConfigMap called web-store-config in the rutas-norte-dev namespace, with the project labels and two keys: api_url with value http://bookings-api and maintenance_mode with value "NO". Explain why the second value needs quotes.
2. Use kubectl, without searching the internet, to find out which apiVersion CronJob, Ingress and HorizontalPodAutoscaler belong to, and which of them are namespace-scoped.
3. Check with kubectl explain whether the spec.containers.imagePullPolicy field exists on a Pod and which values it accepts.

Starting from the bookings-api pod manifest in section 1:

1. Validate it locally without touching the cluster.
2. Apply it, and apply it again without changing anything. What does kubectl say the second time and why?
3. Change the image to version 2.5.0 in the file and, before applying, show exactly what would change in the cluster.
4. Show the pod's status.phase and IP using jsonpath.
5. Explain what the difference would have been if in step 2 you had used kubectl create instead of apply.

Create the file k8s/base/redis-cache.yaml containing, in a single file and in the right order, three things:

1. A ConfigMap redis-cache-config with a redis.conf key containing several configuration lines (use the literal indicator).
2. A Pod redis-cache with the image redis:7.2-alpine.
3. All of it in rutas-norte-dev, with the project labels.

Then state which command would apply the whole directory and why it is best to apply namespace.yaml first.

Solution 1

"NO" needs quotes because YAML 1.1 interprets NO (just like no, yes, on, off) as the boolean false. A ConfigMap's data field only accepts strings, so without quotes the apiserver would reject the object with a type error.

All three are namespace-scoped (NAMESPACED = true).

Solution 2

In step 2, the second run says unchanged because apply is idempotent: kubectl compares the last-applied-configuration annotation, the current file and the live object, finds no differences and sends no modification. That property is what allows apply to run on every deployment of a pipeline with no side effects.

With kubectl create in step 2, the first time would have worked and the second would have failed with Error from server (AlreadyExists), because create does not merge: it only knows how to create new objects.

Solution 3

The ConfigMap goes first because the pod will consume it, which avoids a transient error. It is best to apply namespace.yaml separately and first because every other object declares namespace: rutas-norte-dev and, if that namespace does not exist yet, their creation fails with namespaces "rutas-norte-dev" not found. The namespace is the set's only hard dependency.

You now have command of Kubernetes' unit of work. Every object has the same anatomy —apiVersion, kind, metadata, the spec you write and the status the cluster writes—, belongs to an API group whose version you must verify in your own cluster with kubectl api-resources, and is expressed in YAML with strict rules about indentation, types and multiline strings. You have seen why apply is superior to create and replace, how the three-way merge and its last-applied annotation work, where the model is heading with server-side apply and field ownership, and which validation sequence —--dry-run=client, --dry-run=server, kubectl diff, kubectl explain— turns a deployment into something predictable. And you have the k8s/ directory structure that will hold the project together for the rest of the course.

You have a cluster, you have the CLI and you have the language of manifests. All that is missing is the patient. In the next lesson, The Course Project: the Rutas Norte Platform, you will get to know the company in detail, each of the platform's six components, the target architecture and the project conventions; and you will finish by really deploying your first piece of Rutas Norte into the cluster, seeing it work in your browser.