We begin Chapter 24: Observability, one of the skills that most distinguishes a professional who knows how to operate in production. Having an application running is not enough: you need to know what's happening inside it at all times. Is it going well? Are there errors? Is it overloaded? That's what observability is for, and in AWS the central tool is CloudWatch. Let's start with its three pillars: logs, metrics, and alarms.
The Problem: Operating Blind
Imagine you have your application deployed in production. Users are using it. And suddenly... it starts to slow down, or some users report errors. Without observability, you are blind:
- How many errors are occurring? You don't know.
- Is the server overloaded with CPU? No idea.
- What exactly happened when it failed? There's no trace.
- When did the problem start? Impossible to know.
Operating like this is like driving with your eyes closed. Observability is the dashboard instruments of your application: they tell you what's happening, so you can react.
What is CloudWatch
CloudWatch is the AWS observability service: it collects and displays information about the operation of your resources and applications. It's where you look to see if everything is going well. It has several components; in this subchapter we look at the three fundamental ones.
Pillar 1: Logs
Logs are the text messages that your applications and services write about what they are doing. They are your application's journal:
[10:32:01] User 4521 logged in [10:32:05] Processing order #8890 [10:32:06] ERROR: could not connect to the database [10:32:07] Retrying connection...
CloudWatch Logs collects and stores these messages centrally. Instead of having logs scattered on each server (and losing them if the server shuts down), they all go to CloudWatch, where you can search, filter, and query them.
Analogy: logs are like the ship's logbook: the captain writes down everything that happens ("10:00 set sail", "12:00 storm in sight", "12:30 leak repaired"). If something goes wrong, you check the logbook to understand what happened and when. CloudWatch Logs is the place where all those logbooks are kept together, ready to consult.
Logs are your first tool when something fails: you go to the logs from the time of the failure and read what happened.
Pillar 2: Metrics
Metrics are numerical values measured over time: CPU usage, amount of memory, number of requests per second, errors per minute... While logs are text ("what happened"), metrics are numbers ("how much"):
Metric "CPU Usage" of the server throughout the day: 10:00 → 20 % 11:00 → 35 % 12:00 → 85 % ← peak! 13:00 → 40 %
CloudWatch automatically collects many metrics from your resources (EC2 CPU, ALB requests, Lambda invocations...) and you can send your own (business metrics, like "orders completed per minute"). With metrics you see trends and detect when something is out of the ordinary.
Analogy: metrics are like the dashboard indicators in a car: speed, RPM, engine temperature, fuel level. They are numbers you glance at to know if everything is okay. If the temperature needle rises too much, you know there's a problem before the engine breaks down.
Pillar 3: Alarms
Here's the piece that makes observability proactive. You can't be watching metrics 24 hours a day. An alarm monitors a metric for you and automatically notifies you when it crosses a threshold you define:
Alarm: "if CPU usage exceeds 80% for 5 minutes → NOTIFY" Alarm: "if errors exceed 10 per minute → NOTIFY" Alarm: "if the database runs out of space → NOTIFY"
When an alarm is triggered, it can notify you (by email, Slack, etc., using SNS, remember subchapter 15.2) or even trigger an automatic action (like adding more servers with an Auto Scaling Group, remember subchapter 13.3).
Analogy: an alarm is like the red warning light on the dashboard that comes on when the engine temperature is dangerous, accompanied by a beep. You don't have to constantly watch the needle: the car alerts you when something important needs your attention.
How the Three Work Together
The three pillars work as a team so you operate with your eyes open:
METRICS → tell you WHAT is happening (numbers, trends) ALARMS → ALERT you when a metric goes out of range LOGS → tell you WHY it happened (the details, for investigation)
The typical incident flow: an alarm goes off ("high errors!"), you look at the metrics to see the scope and when it started, and you go to the logs from that moment to understand the exact cause and fix it.
Real-world example: an online store has an alarm on the "HTTP 500 errors" metric. On a Sunday night, a change introduces a bug and errors spike. The alarm goes off and notifies the on-call team via Slack in one minute. The engineer checks the metrics: errors started right after the last deployment, at 22:14. They go to the logs from 22:14 and see: "ERROR: 'price' field null in cart". In 10 minutes they identify and revert the change. Without observability, they would have found out the next morning from customer complaints and lost sales.
What You Should Remember
- Operating without observability is driving with your eyes closed; you need to know what's happening inside your application at all times.
- CloudWatch is AWS's observability service, with three fundamental pillars:
- Logs: the text messages your apps write ("what happened"), collected and queryable centrally. They are the logbook.
- Metrics: numerical values over time ("how much": CPU, requests, errors...). They are the dashboard indicators; they reveal trends.
- Alarms: monitor a metric and automatically alert you (or trigger actions) when it crosses a threshold. They are the red warning light that alerts you without having to look.
- They work as a team: alarms alert, metrics show the scope, logs explain the cause.
In the next subchapter, we'll see how to bring all these metrics together in visual panels with CloudWatch Dashboards.
Cloud, AWS & Terraform — From Zero to Expert
Chapter 1 · What is cloud computing
- 1.1 The traditional client-server model
- 1.2 Problems the cloud came to solve
- 1.3 On-premise vs cloud vs hybrid
- 1.4 The three service models: IaaS, PaaS, SaaS
- 1.5 The five pillars of cloud (according to NIST)
- 1.6 Real advantages: elasticity, pay-as-you-go, global availability
Chapter 2 · The cloud market and major providers
- 2.1 AWS, Azure and GCP: differences and market share
- 2.2 Why learn AWS first
- 2.3 Concepts that are universal among providers
Chapter 3 · Regions, availability zones and edge
- 3.1 What is an AWS region and how to choose it
- 3.2 Availability Zones: high availability by design
- 3.3 Edge locations and CloudFront
- 3.4 Latency, resilience and data sovereignty
Chapter 4 · Compute: EC2
- 4.1 Instances: types, families and when to choose each
- 4.2 AMIs, key pairs and Security Groups
- 4.3 Instance lifecycle
- 4.4 Elastic IPs and Placement Groups
- 4.5 Savings Plans vs Reserved vs On-Demand vs Spot
Chapter 5 · Storage: S3
- 5.1 Buckets, objects and keys
- 5.2 Storage classes (Standard, IA, Glacier…)
- 5.3 Versioning and object lifecycle
- 5.4 Bucket policies and ACLs
- 5.5 Static website hosting
Chapter 6 · Networking: VPC
- 6.1 What is a VPC and why you need it
- 6.2 Public and private subnets
- 6.3 Internet Gateway and NAT Gateway
- 6.4 Route Tables and Network ACLs
- 6.5 VPC Peering and endpoints
Chapter 7 · Identity and access: IAM
- 7.1 Users, groups, roles and policies
- 7.2 The principle of least privilege
- 7.3 Identity-based vs resource-based policies
- 7.4 MFA and temporary credentials (STS)
- 7.5 IAM security best practices
Chapter 8 · Managed databases
- 8.1 RDS: engines, Multi-AZ and read replicas
- 8.2 Aurora and its advantages over vanilla RDS
- 8.3 DynamoDB: key-value / document model
- 8.4 ElastiCache for in-memory cache
- 8.5 When to use each type of database
Chapter 9 · Why Infrastructure as Code
- 9.1 Problems with manual provisioning
- 9.2 Declarative vs imperative IaC
- 9.3 Terraform vs CloudFormation vs Pulumi vs CDK
- 9.4 The plan → apply → destroy cycle
Chapter 10 · HCL: the Terraform language
- 10.1 Resource, variable, output, locals blocks
- 10.2 Data types: string, number, bool, list, map, object
- 10.3 Expressions, references and built-in functions
- 10.4 Conditionals and loops (count, for_each, for)
Chapter 11 · Providers and state
- 11.1 How the AWS provider works
- 11.2 The terraform.tfstate file and its importance
- 11.3 Local state vs remote state (S3 + DynamoDB)
- 11.4 Essential commands: init, plan, apply, destroy, fmt, validate
Chapter 12 · Your first real infrastructure in Terraform
- 12.1 Create a VPC with subnets from scratch
- 12.2 Launch a public EC2 instance
- 12.3 Associate a Security Group and an Elastic IP
- 12.4 Outputs and references between resources
- 12.5 Team workflow: PR review of plans
Chapter 13 · Load balancing and auto scaling
- 13.1 Application Load Balancer vs Network Load Balancer
- 13.2 Target Groups, listeners and rules
- 13.3 Auto Scaling Groups: policies and metrics
- 13.4 Warm pools and lifecycle hooks
Chapter 14 · Serverless with Lambda
- 14.1 The Lambda execution model
- 14.2 Triggers: API Gateway, S3, DynamoDB Streams, SQS
- 14.3 Dependency management and layers
- 14.4 Cold starts and strategies to reduce them
- 14.5 Limits and anti-patterns
Chapter 15 · Messaging and events
- 15.1 SQS: standard vs FIFO queues, DLQ
- 15.2 SNS: topics, subscriptions, fan-out
- 15.3 EventBridge: event buses and rules
- 15.4 Patterns: pub/sub, decoupling, saga
Chapter 16 · Content delivery and DNS
- 16.1 Route 53: record types and routing policies
- 16.2 CloudFront: distributions, caches and origins
- 16.3 ACM: free SSL/TLS certificates
- 16.4 WAF integrated with CloudFront
Chapter 17 · Containers on AWS
- 17.1 Docker: quick review of key concepts
- 17.2 ECR: private image registry
- 17.3 ECS: task definitions, services, Fargate vs EC2
- 17.4 EKS: when Kubernetes and when not
Chapter 18 · Modules: reuse and composition
- 18.1 Anatomy of a Terraform module
- 18.2 Input variables, outputs and dependencies
- 18.3 Local modules vs Terraform Registry modules
- 18.4 Module versioning with Git tags
- 18.5 Design of generic vs domain-specific modules
Chapter 19 · Workspaces and environment management
- 19.1 Terraform workspaces: use cases and limitations
- 19.2 Directory strategy per environment (dev/stg/prod)
- 19.3 Terragrunt: DRY for environment configurations
- 19.4 Environment variables and .tfvars files
Chapter 20 · Remote backends and locking
- 20.1 Configure S3 + DynamoDB as backend
- 20.2 State locking: avoiding team corruption
- 20.3 State migration between backends
- 20.4 terraform import: bring existing resources into state
Chapter 21 · Infrastructure testing
- 21.1 Terraform validate and fmt in CI
- 21.2 Checkov and tfsec: static security analysis
- 21.3 Terratest: integration tests in Go
- 21.4 Contract testing between modules
Chapter 22 · Terraform in CI/CD
- 22.1 Basic pipeline: lint → plan → apply in GitHub Actions
- 22.2 Atlantis: GitOps for Terraform
- 22.3 Terraform Cloud / HCP Terraform
- 22.4 Drift detection and automatic reconciliation
Chapter 23 · Defense in depth
- 23.1 AWS Organizations and Service Control Policies
- 23.2 AWS Config: continuous compliance
- 23.3 GuardDuty: threat detection
- 23.4 Security Hub: centralized view
- 23.5 KMS: key management and rotation
- 23.6 Secrets Manager vs Parameter Store
Chapter 24 · Observability: logs, metrics and traces
- 24.1 CloudWatch Logs, metrics and alarms
- 24.2 CloudWatch Dashboards and Contributor Insights
- 24.3 X-Ray: distributed tracing
- 24.4 OpenTelemetry on AWS
- 24.5 Managed Grafana and Managed Prometheus
Chapter 25 · Cost optimization
- 25.1 AWS Cost Explorer and budgets with alerts
- 25.2 Trusted Advisor and Compute Optimizer
- 25.3 Rightsizing: how to detect overprovisioning
- 25.4 Savings Plans vs Reserved Instances: strategic decision
- 25.5 FinOps: culture and processes to control spending
Chapter 26 · High availability and disaster recovery
- 26.1 RTO and RPO: defining objectives
- 26.2 Strategies: backup/restore, pilot light, warm standby, multi-site
- 26.3 Route 53 health checks and automatic failover
- 26.4 AWS Backup: centralized backup policy
Chapter 27 · AWS Well-Architected Framework
- 27.1 The six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, sustainability
- 27.2 Well-Architected Tool: formal reviews
- 27.3 How to apply the framework in design decisions
Chapter 28 · Serverless architectures at scale
- 28.1 Event-driven architecture with Lambda + EventBridge
- 28.2 Saga pattern for distributed transactions
- 28.3 Step Functions: orchestration of complex workflows
- 28.4 Lambda@Edge and CloudFront Functions
Chapter 29 · Data platforms on AWS
- 29.1 Data Lake with S3, Glue and Athena
- 29.2 Kinesis Data Streams and Firehose for streaming
- 29.3 Redshift: data warehousing at scale
- 29.4 Lake Formation: data governance
Chapter 30 · Multi-account and landing zones
- 30.1 Why separate workloads into different accounts
- 30.2 AWS Control Tower and Account Factory
- 30.3 Centralized log and security management
- 30.4 Terraform at multi-account scale with shared modules
Chapter 31 · Platform Engineering and Internal Developer Platform
- 31.1 Golden paths and abstractions over Terraform
- 31.2 AWS Service Catalog
- 31.3 Backstage as a developer portal
- 31.4 Terraform modules as internal product
Chapter 32 · Relevant AWS certifications
- 32.1 Cloud Practitioner: is it worth it?
- 32.2 Solutions Architect Associate → Professional
- 32.3 DevOps Engineer Professional
- 32.4 Specialty: Security, Database, Networking
- 32.5 HashiCorp Terraform Associate
Chapter 33 · Projects to consolidate what you've learned
- 33.1 Project 1: serverless blog (S3 + CloudFront + Lambda + DynamoDB)
- 33.2 Project 2: REST API with ECS Fargate + RDS + ALB
- 33.3 Project 3: data platform with Glue + Athena + Redshift
- 33.4 Project 4: multi-account landing zone with Terraform and Control Tower
