There is a conversation AlpinaShop has been deferring for seven modules, and which the previous lesson put back on the table three times: enabling the data access logs costs money, Security Command Center Premium was ruled out on price, and three thousand euros were allocated as if they were a lot of money — which for a forty-person company they are.
None of those decisions can be taken well without knowing where every euro of the bill comes from. And today nobody at AlpinaShop knows.
July's bill was €1,058. January's was €210. Nobody decided to multiply it by five: it grew lesson by lesson, service by service, without any single decision seeming expensive. When Marta opened the breakdown to prepare next year's budget, she found two line items she could not explain and which together were 40 % of the total.
This lesson opens that whole bill. By the end you will know why the cloud takes people by surprise, you will understand what a SKU is and how to query billing in BigQuery to answer "who spends what", you will build budgets that act rather than merely warn, you will know the saving levers ranked by real return, you will hunt down phantom spend with a script, and you will see AlpinaShop's bill before and after, broken down and commented on honestly — including what could not be saved and what was deliberately overspent.
Contents
- Why the cloud bill takes you by surprise
- Understanding the bill: SKUs, usage and discounts
- The billing export to BigQuery
- The queries that answer "who spends what"
- FinOps for a small business: inform, optimise, operate
- Budgets, alerts and automatic actions
- Quotas as a hard limit
- Saving levers ranked by return
- The hidden cost of traffic
- Labelling and cost allocation
- Anomaly detection and phantom spend
- The complete AlpinaShop case: before and after
- Tools: Recommender, Active Assist and the calculator
- Why the cloud bill takes you by surprise
In the traditional model, infrastructure spend was an annual, visible, negotiated decision: you bought servers, somebody signed, and for three years the cost was the same every month. It was expensive, rigid and inefficient — and predictable.
In the cloud the opposite happens, and the difference can be summed up in four points:
| Factor | Effect |
|---|---|
| Variable cost | The bill depends on usage, and usage depends on traffic, on users and on mistakes |
| Technical decisions with financial effects | A SELECT * from Lucía can cost more than the server the shop runs on |
| No natural brake | Nobody signs a purchase order to create a cluster. It is created in two clicks |
| Time lag | Today's spend shows up on the bill a month from now, once it has already happened |
The third point is the decisive one, and it deserves a real example. In 04-06, to evaluate Cloud Composer against Workflows, Marta created a Composer environment. The evaluation lasted two days, the decision was Workflows, and nobody deleted the environment. Cloud Composer does not scale to zero: it keeps permanent infrastructure running. That environment has been running for four months without anybody using it, at around €280 a month.
Nobody decided to spend €1,120. It is simply that nobody decided not to.
The first rule of cloud cost management: the problem is almost never that something is expensive. It is that nobody knew it was switched on.
- Understanding the bill: SKUs, usage and discounts
The Google Cloud bill has three levels of aggregation, and you have to learn to read them in the reverse of the order they appear.
| Level | Example | Usefulness |
|---|---|---|
| Service | Compute Engine | Too coarse: it says what but not why |
| SKU | "N1 Predefined Instance Core running in EMEA" | The useful level: a specific billable unit |
| Usage | 1,460 hours × $0.0348 | The detail |
A SKU (Stock Keeping Unit) is the smallest unit of billing: a specific combination of resource, region and mode. Each service has dozens or hundreds of them.
A real example of the Compute Engine breakdown for AlpinaShop's MIG before it was retired:
| SKU | Usage | Cost |
|---|---|---|
| N1 Predefined Instance Core running in EMEA | 1,460 vCPU-h | €29.50 |
| N1 Predefined Instance Ram running in EMEA | 5,840 GiB-h | €16.20 |
| Balanced PD Capacity in EMEA | 40 GiB-month | €3.80 |
| Network Inter Zone Egress | 12 GiB | €0.14 |
| Network Internet Egress from EMEA to EMEA | 85 GiB | €10.20 |
The immediate lesson: the cost of a VM is not "a VM". It is vCPU and memory and disk and traffic, each with its own SKU and its own price. That is why switching from e2-medium to e2-small does not halve the cost: it reduces two of the five SKUs.
The discounts that apply themselves and the ones you have to ask for
| Discount | How you get it | Typical saving |
|---|---|---|
| Sustained use (SUD) | Automatic in Compute Engine if the VM runs for a good part of the month | Up to ~30 % |
| Committed use (CUD) | You sign up: 1 or 3 years | 30-55 % |
| Free tier | Automatic, monthly | Variable |
| Spot / preemptible | Chosen when you create the resource | 60-91 % |
| Volume discounts | Negotiated with Google (large enterprises) | Variable |
The sustained use discount is interesting because it is invisible: it applies on its own, and it explains why the bill for a VM switched on all month is lower than hours × list price. And also why switching a VM off at night saves less than the arithmetic suggests: dropping below the threshold loses part of the automatic discount.
All the prices in this lesson are orders of magnitude in
europe-west1for reasoning purposes. They change, they vary by region and they depend on the contract. Always verify them on the official pricing page and in the calculator.
- The billing export to BigQuery
It was enabled in 01-04 with a promise to exploit it. The moment has come.
The billing console is good for seeing trends. To answer specific questions — why did the bill go up on Tuesday the 14th?, how much does the development environment cost? — you need SQL.
# 1. Destination dataset in alpinashop-datos
bq --location=EU mk --dataset \
--description="Billing export" \
alpinashop-datos:facturacion
# 2. The export is configured in the console:
# Billing > Billing export > BigQuery settings
# Enable "Detailed cost" (resource level), not just "Standard cost"Always enable the detailed cost export, even though it generates more rows. The difference:
| Export | Granularity | Answers |
|---|---|---|
| Standard costs | Service + SKU + project + label | "How much does BigQuery spend in production?" |
| Detailed costs | + the specific resource | "Which specific bucket costs €40?" |
| Prices | Price catalogue | Simulations |
Without the per-resource detail, you will know that Cloud Storage costs €25 but not which bucket. And that is precisely the question you need to answer.
Two practical warnings:
- The data is delayed. The export is populated several hours late and is corrected over the following days: yesterday's costs may change tomorrow. It is no good for real-time alerts; that is what budgets are for.
- The table is enormous and is partitioned by
_PARTITIONTIME. Querying it without a date filter processes months of data, and that query costs you money. It is ironic and it happens constantly.
- The queries that answer "who spends what"
These six queries cover 95 % of the real questions. They are worth saving as views.
1. Spend by project, last 30 days.
SELECT
project.id AS project_id,
ROUND(SUM(cost), 2) AS cost_eur,
ROUND(SUM(IFNULL((SELECT SUM(c.amount) FROM UNNEST(credits) c), 0)), 2) AS credits_eur,
ROUND(SUM(cost) + SUM(IFNULL((SELECT SUM(c.amount) FROM UNNEST(credits) c), 0)), 2) AS net_eur
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY project_id
ORDER BY net_eur DESCThe credits field is a nested array holding discounts, promotions and the free tier, and they are always negative values. Adding it to the cost gives the real net. Forgetting it is the number one mistake when analysing billing in BigQuery: you see inflated figures and nobody understands why they do not match the console.
2. The 20 most expensive SKUs: where the money really is.
SELECT
service.description AS service,
sku.description AS sku,
ROUND(SUM(cost), 2) AS cost_eur,
ROUND(SUM(usage.amount), 2) AS usage_amount,
ANY_VALUE(usage.unit) AS unit
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY service, sku
ORDER BY cost_eur DESC
LIMIT 203. Spend by label: the one that justifies the discipline from 01-04.
SELECT
(SELECT value FROM UNNEST(labels) WHERE key = 'entorno') AS entorno,
(SELECT value FROM UNNEST(labels) WHERE key = 'equipo') AS equipo,
(SELECT value FROM UNNEST(labels) WHERE key = 'centro-coste') AS centro_coste,
(SELECT value FROM UNNEST(labels) WHERE key = 'aplicacion') AS aplicacion,
ROUND(SUM(cost), 2) AS cost_eur
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY entorno, equipo, centro_coste, aplicacion
ORDER BY cost_eur DESC4. The most expensive individual resources (requires the detailed export).
SELECT
resource.name AS resource_name,
service.description AS service,
ROUND(SUM(cost), 2) AS cost_eur
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
AND resource.name IS NOT NULL
GROUP BY resource_name, service
ORDER BY cost_eur DESC
LIMIT 305. Daily evolution: to see the step change.
SELECT
DATE(usage_start_time) AS day,
service.description AS service,
ROUND(SUM(cost), 2) AS cost_eur
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 60 DAY)
GROUP BY day, service
HAVING cost_eur > 0.5
ORDER BY day DESC, cost_eur DESC6. UNLABELLED resources: governance debt turned into a number.
SELECT
project.id AS project_id,
service.description AS service,
ROUND(SUM(cost), 2) AS unattributed_cost
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
AND (SELECT COUNT(*) FROM UNNEST(labels) WHERE key = 'centro-coste') = 0
GROUP BY project_id, service
HAVING unattributed_cost > 1
ORDER BY unattributed_cost DESCQuery 6 is the most uncomfortable and the most useful. Everything that appears is spend that cannot be attributed to anybody. At AlpinaShop, the first time it was run, €380 out of €1,058 came out: 36 % of the bill had no owner.
- FinOps for a small business: inform, optimise, operate
FinOps is the discipline of managing cloud cost as a shared responsibility between finance, technology and the business. It has three phases you go round in a loop:
flowchart LR
I["INFORM<br/>visibility, allocation,<br/>forecasts"] --> O["OPTIMISE<br/>rightsize, discounts,<br/>remove waste"]
O --> P["OPERATE<br/>policies, automation,<br/>continuous review"]
P --> I
| Phase | What you do | At AlpinaShop |
|---|---|---|
| Inform | Export, labels, dashboards, budgets | Sections 3, 4, 6, 10 |
| Optimise | Rightsize, switch off, discounts, architecture | Section 8 |
| Operate | Policies that prevent waste, automation | Sections 6, 7, 10, 11 |
The classic mistake is to start with optimising. Without visibility, you optimise what you can see — which is usually the least important thing — and ignore the €280 Composer environment nobody knew existed. First you measure, then you cut.
Who owns the cost?
In a large company there is a FinOps team. In a forty-person one, the question is real and it has an uncomfortable answer:
| Model | How it works | Problem |
|---|---|---|
| Nobody | You look at the bill when it frightens you | AlpinaShop's, until today |
| Finance | Accounting watches the total | It does not understand what a SKU is and cannot act |
| Infrastructure only | Marta carries the whole thing | She does not control what Lucía's queries spend |
| Distributed responsibility with a coordinator | Each team sees and answers for its spend; somebody coordinates | The right one |
For AlpinaShop:
- Marta coordinates: she maintains the dashboard, reviews it monthly and makes proposals.
- Each person sees their own spend: Lucía for BigQuery and Dataflow, Dani for Cloud Run and Cloud Build.
- Management approves the multi-year commitments.
- A 30-minute meeting once a month with the dashboard open.
That monthly meeting, which looks like bureaucracy, is the FinOps practice with the best cost/benefit ratio there is. What gets looked at as a group once a month does not get out of hand.
- Budgets, alerts and automatic actions
A budget in Google Cloud does not cap spend: it warns. This surprises everybody and it has to sink in:
There is no "do not spend more than X" button. The budget sends notifications; stopping the spend is something you have to automate yourself.
A basic budget
gcloud billing budgets create \
--billing-account=BILLING_ACCOUNT_ID \
--display-name="AlpinaShop monthly total" \
--budget-amount=1200EUR \
--threshold-rule=percent=0.5 \
--threshold-rule=percent=0.8 \
--threshold-rule=percent=1.0 \
--threshold-rule=percent=1.2 \
--threshold-rule=percent=0.8,basis=forecasted-spend \
--notifications-rule-monitoring-notification-channels=CHANNEL_IDThe last rule is the most valuable and the least used: basis=forecasted-spend warns you when Google forecasts that the threshold will be exceeded by the end of the month. It warns on the 8th, not on the 26th. It is the only alert that arrives in time to do something.
Budgets worth having, rather than a single one:
| Budget | Amount | Why |
|---|---|---|
| Organization total | €1,200 | The global view |
alpinashop-prod |
€700 | The one that matters |
alpinashop-dev |
€100 | This is where everything gets out of hand |
alpinashop-datos |
€250 | BigQuery and Dataflow are variable |
| BigQuery only, by label | €150 | A single query can blow this on its own |
The automatic action
This is where the budget stops being a warning and becomes a control. The flow:
flowchart LR
B["Budget<br/>threshold exceeded"] --> PS["Pub/Sub<br/>alertas-presupuesto"]
PS --> F["Cloud Function<br/>reaccionar-presupuesto"]
F -->|"100% in dev"| A1["Stop development VMs"]
F -->|"120% in dev"| A2["Unlink billing"]
F -->|"any threshold"| A3["Notify the channel"]
# main.py — reaccionar-presupuesto function (Cloud Functions 2nd gen, 06-03)
import base64, json, os, logging
from google.cloud import compute_v1
from googleapiclient import discovery
PROYECTO_DEV = "alpinashop-dev"
ZONA = "europe-west1-b"
def reaccionar_presupuesto(evento, contexto):
datos = json.loads(base64.b64decode(evento["data"]).decode("utf-8"))
nombre = datos.get("budgetDisplayName", "")
gastado = float(datos.get("costAmount", 0))
presupuesto= float(datos.get("budgetAmount", 1))
umbral = float(datos.get("alertThresholdExceeded", 0))
ratio = gastado / presupuesto if presupuesto else 0
logging.info(json.dumps({
"severity": "WARNING", "message": "Budget threshold exceeded",
"presupuesto": nombre, "gastado": gastado, "ratio": round(ratio, 2),
}))
# Only DEVELOPMENT is acted on. Production is NEVER shut down automatically.
if "dev" not in nombre.lower():
return
if ratio >= 1.0:
apagar_vms_desarrollo()
if ratio >= 1.2:
desvincular_facturacion(PROYECTO_DEV)
def apagar_vms_desarrollo():
cliente = compute_v1.InstancesClient()
for inst in cliente.list(project=PROYECTO_DEV, zone=ZONA):
if inst.status == "RUNNING":
cliente.stop(project=PROYECTO_DEV, zone=ZONA, instance=inst.name)
logging.warning(f"VM stopped because of the budget: {inst.name}")
def desvincular_facturacion(proyecto):
# EXTREME MEASURE: it stops practically the whole project
svc = discovery.build("cloudbilling", "v1")
svc.projects().updateBillingInfo(
name=f"projects/{proyecto}",
body={"billingAccountName": ""}
).execute()
logging.critical(f"BILLING UNLINKED from {proyecto}")Three warnings to read twice:
- Never automate shutting down production. A sales spike during a campaign trips the budget; switching off the shop on the best day of the year is worse than any bill.
- Unlinking billing is destructive and partly irreversible. Resources stop and some are deleted after a grace period. It only makes sense in development, and even then it is worth thinking hard about.
- Budget messages can arrive more than once. The function has to be idempotent (06-03): stopping an already stopped VM must not fail.
- Quotas as a hard limit
Where the budget warns, the quota prevents. It is the only mechanism that really stops spend.
| Type | What it limits | Use for cost |
|---|---|---|
| Rate | Requests per minute to an API | Containing runaway loops |
| Allocation | Simultaneous resources: vCPU, IPs, disks | The most effective |
# See the CPU quotas of a region
gcloud compute regions describe europe-west1 --format="table(quotas.metric, quotas.limit, quotas.usage)"Limiting the vCPU quota in development is the most effective brake there is, because it is impossible to get round: if the quota is 8 vCPU, nobody can create a 32-vCPU VM however much they want to.
BigQuery's cost quotas
BigQuery on demand charges for bytes read, and a single badly written query can cost more than a month of servers. There are two brakes, and both should always be in place:
# 1. Limit on bytes processed per day and per user
gcloud alpha services quota update \
--service=bigquery.googleapis.com \
--consumer=projects/alpinashop-datos \
--metric=bigquery.googleapis.com/quota/query/usage \
--unit=1/d/{project}/{user} \
--value=1099511627776 # 1 TiB per user per day-- 2. Limit per individual query, in the session or in the script
SET @@maximum_bytes_billed = 107374182400; -- 100 GiBAnd in the command-line tool:
The message a blocked query returns is explicit — "Query exceeded limit for bytes billed" — and it produces the right educational effect: whoever wrote it learns to filter by partition before rewriting it.
- Saving levers ranked by return
This is the practical section. Ranked by euros saved per hour of work invested, which is the criterion that matters when the team is small.
Lever 1 — Switch off what is not being used (immediate return)
The most profitable thing is not to optimise: it is to remove. And there is always something.
# Development environments outside working hours, with Cloud Scheduler + Workflows
gcloud scheduler jobs create http apagar-desarrollo \
--location=europe-west1 \
--schedule="0 20 * * 1-5" \
--time-zone="Europe/Madrid" \
--uri="https://workflowexecutions.googleapis.com/v1/projects/alpinashop-dev/locations/europe-west1/workflows/apagar-entorno/executions" \
--http-method=POST \
--oauth-service-account-email=sa-scheduler@alpinashop-dev.iam.gserviceaccount.com
gcloud scheduler jobs create http encender-desarrollo \
--location=europe-west1 \
--schedule="0 8 * * 1-5" \
--time-zone="Europe/Madrid" \
--uri="https://workflowexecutions.googleapis.com/v1/projects/alpinashop-dev/locations/europe-west1/workflows/encender-entorno/executions" \
--http-method=POST \
--oauth-service-account-email=sa-scheduler@alpinashop-dev.iam.gserviceaccount.comThe arithmetic that convinces anybody: an environment switched on from 08:00 to 20:00 Monday to Friday is 260 hours out of the month's 730. A 64 % saving for two scheduled jobs.
With two honest caveats: the disks are still paid for even while the VM is off, and part of the sustained use discount is lost. The real saving is around 50-55 %, not 64 %.
Lever 2 — Rightsize
Google observes real usage over 8 days and makes recommendations:
gcloud recommender recommendations list \
--project=alpinashop-prod --location=europe-west1-b \
--recommender=google.compute.instance.MachineTypeRecommender \
--format="table(description, primaryImpact.costProjection.cost.units)"There are recommenders for VMs, persistent disks, Cloud SQL, IP addresses and many more. Applying them is one of the most profitable things per hour invested.
The warning: the recommendations are based on the observed window. If that window is August and the peak is in October, sizing down according to August is preparing a problem for yourself. Always cross-check against the annual peak.
Lever 3 — Commitment discounts
| Sustained use (SUD) | Committed use (CUD) | |
|---|---|---|
| How | Automatic | You sign up |
| Commitment | None | 1 or 3 years, you pay regardless |
| Saving | Up to ~30 % | 30-55 % |
| Flexibility | Total | Spend-based ones are flexible; resource-based ones are not |
| Risk | None | Paying for what you no longer use |
There are two types of commitment, and choosing the wrong one is expensive:
| Type | You commit to | Risk |
|---|---|---|
| Resource-based | X vCPU and Y GiB in a specific region and family | High: if you migrate or change family, you keep paying |
| Spend-based (flexible) | Spending at least €X/hour on a service | Low: it applies to whatever you use |
Practical rule: commit only to the stable part of your consumption, typically 50-70 % of the historical minimum. Never to the whole thing.
And the lesson AlpinaShop learnt the hard way: if Marta had signed a one-year commitment in January on the MIG's vCPUs, today — after migrating to Cloud Run in 07-02 — she would still be paying for VMs that no longer exist. Architectural optimization invalidates commitments. First you stabilise the architecture, then you commit.
Lever 4 — Spot VMs
Instances at a 60-91 % discount that Google can reclaim with 30 seconds' notice.
| Good for | Not good for |
|---|---|
| Batch processing with retries | Databases |
| Model training with checkpoints | Customer-facing web services |
| Dataflow and Dataproc workers | Anything with local state |
| CI/CD | Anything that cannot tolerate restarts |
gcloud dataproc clusters create cluster-analitica \
--region=europe-west1 \
--num-workers=2 \
--num-secondary-workers=8 \
--secondary-worker-type=spot # 8 of 10 workers at 30% of the priceThe correct pattern is the one in that command: a small stable core + a spot majority. If Google reclaims the spot instances, the job slows down but does not die.
Lever 5 — Storage classes and lifecycle
| Class | Relative cost | Minimum storage duration | Use |
|---|---|---|---|
| Standard | 1× | — | Frequent access |
| Nearline | ~0.5× | 30 days | Less than once a month |
| Coldline | ~0.3× | 90 days | Quarterly |
| Archive | ~0.12× | 365 days | Legal copies |
resource "google_storage_bucket" "datalake" {
name = "alpinashop-datalake"
location = "EUROPE-WEST1"
lifecycle_rule {
condition { age = 30 }
action { type = "SetStorageClass" storage_class = "NEARLINE" }
}
lifecycle_rule {
condition { age = 90 }
action { type = "SetStorageClass" storage_class = "COLDLINE" }
}
lifecycle_rule {
condition { age = 365 }
action { type = "SetStorageClass" storage_class = "ARCHIVE" }
}
lifecycle_rule {
condition { num_newer_versions = 3 }
action { type = "Delete" } # limit old versions
}
}The trap in the cold classes, which you need to know about before configuring them: they have early deletion charges. An object in Coldline deleted or read before 90 days is billed as if it had been there the full 90. Putting Archive on data that is queried every couple of months works out more expensive than Standard.
The last rule, about versions, solves a real problem: with versioning enabled (and it should be, because of 07-04), every modification leaves a copy. A bucket with versioning and no version limit grows indefinitely and nobody looks at it.
Lever 6 — Partitioning and clustering in BigQuery
BigQuery on demand charges for bytes read, not for time. A 500 GB unpartitioned table costs the same to query for one day as for a whole year.
CREATE OR REPLACE TABLE `alpinashop-datos.alpinashop_analitica.pedidos_opt`
PARTITION BY DATE(fecha_pedido)
CLUSTER BY categoria_producto, provincia
OPTIONS (
partition_expiration_days = 1095,
require_partition_filter = TRUE -- <<< the option that saves the bill
)
AS SELECT * FROM `alpinashop-datos.alpinashop_analitica.pedidos`;require_partition_filter = TRUE is the most profitable option in the whole of BigQuery. It rejects any query that does not filter by partition:
Cannot query over table 'pedidos_opt' without a filter over column(s) 'fecha_pedido' that can be used for partition elimination
An error message in exchange for avoiding full 500 GB scans.
The measured effect on Lucía's usual query:
| Version | Bytes read | Cost per run |
|---|---|---|
Unpartitioned, SELECT * |
480 GB | ~€2.70 |
| Partitioned, 30-day filter | 41 GB | ~€0.23 |
| + clustering by category | 6 GB | ~€0.03 |
Ninety times cheaper. And that same query run 40 times a month in a dashboard goes from €108 to €1.20.
Lever 7 — On demand versus BigQuery editions
| Model | How you pay | When it is right |
|---|---|---|
| On demand | Per TB read | Irregular or low usage. AlpinaShop |
| Standard / Enterprise (slots) | For reserved compute capacity, with autoscaling | High, sustained usage |
The break-even point sits at a constant monthly consumption equivalent to several hundred euros on demand. Below that, on demand wins almost every time. Above it, reservations also give you predictable cost, which is sometimes worth more than the saving.
Lever 8 — CDN and caching
Every response served from the Cloud CDN cache (03-03) is traffic that does not leave the origin and compute that is not executed.
| Metric | Without CDN | With CDN at an 85 % hit rate |
|---|---|---|
| Egress from the origin | 300 GB | 45 GB |
| Egress cost | ~€36 | ~€5.40 + CDN cost |
| Cloud Run instances | More | Fewer |
And the free lever almost nobody enables: compression. With gzip or brotli, a 200 KB HTML page travels as 30 KB. 85 % fewer billed bytes, and it loads faster too.
Lever 9 — Scaling to zero
Already quantified in 07-02: the catalogue went from ~€50 to ~€7 a month. The general rule:
Anything that does not receive constant traffic should scale to zero. Cloud Run, Cloud Functions and batch jobs do; VMs and clusters do not.
Summary ranked by return
| # | Lever | Typical saving | Effort | €/hour invested |
|---|---|---|---|---|
| 1 | Remove what is unused | 10-40 % | Very low | Extremely high |
| 2 | Switch off development out of hours | 50 % of dev | Low | Very high |
| 3 | require_partition_filter + clustering |
50-95 % of BigQuery | Low | Very high |
| 4 | Rightsize with Recommender | 20-40 % of compute | Low | High |
| 5 | Storage lifecycle | 40-70 % of storage | Low | High |
| 6 | CDN and compression | 60-85 % of egress | Medium | High |
| 7 | Scaling to zero | 60-90 % of the service | Medium | High |
| 8 | Spot VMs for batch | 60-90 % of those batches | Medium | Medium |
| 9 | Use commitments | 30-55 % of what is committed | Low, with risk | Medium |
| 10 | BigQuery editions | Variable | High | Low without scale |
- The hidden cost of traffic
It was introduced in 07-03 and here it is closed off with the four hiding places and their remedies.
| Path | Indicative cost |
|---|---|
| Ingress from the internet | Free |
| Within a zone, internal IP | Free |
| Between zones in the same region | ~$0.01/GB |
| Between EU regions | ~$0.02/GB |
| To another continent | $0.05-0.15/GB |
| Egress to the internet, Premium | ~$0.12/GB |
| Load balancer: rules + data processed | Fixed charge + per GB |
| Hiding place | How it shows up | Remedy |
|---|---|---|
| Cloud SQL HA replica | Continuous replication between zones | It is the price of availability. Accepted and understood |
| Application and database in different zones | Every query crosses a zone | Co-locate them deliberately |
| Logs and backups to another region | For every log line | Log buckets in the same region |
| Forgotten load balancers | A fixed charge per forwarding rule, traffic or no traffic | Periodic audit (section 11) |
- Labelling and cost allocation
The discipline from 01-04, now exploited. AlpinaShop's four labels:
| Label | Values | Answers |
|---|---|---|
entorno |
prod, dev, pruebas |
How much does development cost? |
equipo |
infra, desarrollo, datos |
Who spends? |
centro-coste |
tienda, analitica, plataforma |
Which business line is it charged to? |
aplicacion |
catalogo, informes, ml |
Which product costs what? |
Rules that make them useful:
- Values from a closed set.
equipo: variosdestroys the analysis. - On every resource that bills. A resource without a label is spend without an owner.
- Applied by Terraform, never by hand.
- Careful: not every resource accepts labels. Network egress, some load balancer SKUs and certain support charges do not carry them. There will always be an unattributed residue; what matters is that it is small and understood.
The policy that prevents creating resources without labels
It is applied in Terraform, which is where everything is created:
variable "etiquetas" {
type = map(string)
description = "AlpinaShop mandatory labels"
validation {
condition = alltrue([
for k in ["entorno", "equipo", "centro-coste", "aplicacion"] :
contains(keys(var.etiquetas), k)
])
error_message = "Mandatory labels missing: entorno, equipo, centro-coste, aplicacion."
}
validation {
condition = contains(["prod", "dev", "pruebas"], lookup(var.etiquetas, "entorno", ""))
error_message = "entorno must be prod, dev or pruebas."
}
validation {
condition = contains(["infra", "desarrollo", "datos"], lookup(var.etiquetas, "equipo", ""))
error_message = "equipo must be infra, desarrollo or datos."
}
}The plan fails if labels are missing or if their values are invalid. It is prevention at the point of creation, which is infinitely more effective than chasing orphan resources afterwards. And it is complemented by the organization's resource tags and the policies in 07-07, which act even on things created outside Terraform.
- Anomaly detection and phantom spend
Anomaly detection
Google Cloud includes automatic cost anomaly detection in the billing console: it compares spend against the historical pattern and warns about deviations. It is free, there is nothing to configure, and it is worth reviewing monthly.
Your own version, more adjustable:
-- Services whose spend yesterday deviates more than 3 sigmas from their 30-day mean
WITH diario AS (
SELECT service.description AS service, DATE(usage_start_time) AS day, SUM(cost) AS cost_eur
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 45 DAY)
GROUP BY service, day
),
estad AS (
SELECT service,
AVG(cost_eur) AS mean,
STDDEV(cost_eur) AS stddev
FROM diario
WHERE day BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 31 DAY)
AND DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)
GROUP BY service
)
SELECT d.service, d.day, ROUND(d.cost_eur,2) AS cost_yesterday,
ROUND(e.mean,2) AS mean_30d,
ROUND(SAFE_DIVIDE(d.cost_eur - e.mean, e.stddev), 1) AS sigmas
FROM diario d JOIN estad e USING (service)
WHERE d.day = DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)
AND e.stddev > 0
AND d.cost_eur > e.mean + 3 * e.stddev
AND d.cost_eur > 2
ORDER BY sigmas DESCThe d.cost_eur > 2 filter avoids the noise: a service that goes from €0.02 to €0.20 is 9 sigmas and nobody cares.
Phantom spend
Resources that cost money and give nobody any value. There is always some. In every company.
| Phantom | Why it appears | Typical cost |
|---|---|---|
| Reserved static IPs not in use | Reserved, the design changes, never released | ~€7/month each. An unused IP costs more than one in use |
| Orphaned persistent disks | The VM is deleted without marking the disk for deletion | €0.04-0.17/GB/month |
| Old snapshots | Taken and never expired | Cumulative |
| Old container images | Every build leaves one | Grows with the pipeline |
| Forgotten Vertex AI endpoints | A model is deployed to test something | High: the machine runs 24×7 |
| Development clusters left running | Created for one test | Very high |
| Composer environments | They do not scale to zero | ~€280/month |
| Load balancers with no backends | Tried out and abandoned | ~€18/month each |
| Temporary data buckets | alpinashop-dataflow accumulates staging files |
Grows indefinitely |
| Pub/Sub subscriptions with no consumer | They retain messages until they expire | Storage |
The phantom audit script
#!/usr/bin/env bash
# auditoria-fantasmas.sh — looks for resources that cost money and serve no purpose
set -uo pipefail
PROYECTOS=(alpinashop-prod alpinashop-dev alpinashop-datos alpinashop-cicd alpinashop-red)
for P in "${PROYECTOS[@]}"; do
echo "############ $P ############"
echo "--- Static IPs reserved and NOT in use (~7 EUR/month each) ---"
gcloud compute addresses list --project="$P" \
--filter="status:RESERVED" --format="table(name, region, address)"
echo "--- Unattached persistent disks ---"
gcloud compute disks list --project="$P" \
--filter="-users:*" --format="table(name, zone, sizeGb, type)"
echo "--- Snapshots older than 180 days ---"
gcloud compute snapshots list --project="$P" \
--filter="creationTimestamp<$(date -u -d '180 days ago' +%Y-%m-%d)" \
--format="table(name, diskSizeGb, creationTimestamp)"
echo "--- Vertex AI endpoints (check whether they are used) ---"
gcloud ai endpoints list --project="$P" --region=europe-west1 \
--format="table(displayName, createTime)" 2>/dev/null
echo "--- Forwarding rules: load balancers, ~18 EUR/month each ---"
gcloud compute forwarding-rules list --project="$P" \
--format="table(name, region, IPAddress, target)"
echo "--- GKE clusters ---"
gcloud container clusters list --project="$P" \
--format="table(name, location, status, currentNodeCount)" 2>/dev/null
echo "--- Cloud Composer environments (~280 EUR/month each) ---"
gcloud composer environments list --project="$P" --locations=europe-west1 \
--format="table(name, state)" 2>/dev/null
echo "--- Stopped Cloud SQL instances (still paying for disk) ---"
gcloud sql instances list --project="$P" \
--filter="state!=RUNNABLE" --format="table(name, state, settings.tier)"
echo "--- Container images older than 90 days ---"
gcloud artifacts docker images list \
"europe-west1-docker.pkg.dev/$P/alpinashop" --include-tags \
--filter="createTime<$(date -u -d '90 days ago' +%Y-%m-%d)" \
--format="table(package, version, createTime)" 2>/dev/null
echo
doneRun it on the first Monday of every month. It is half an hour that at AlpinaShop found, the first time, €430 a month.
And so the phantoms do not come back, automatic policies:
# Automatic expiry of old images in Artifact Registry
gcloud artifacts repositories update alpinashop \
--location=europe-west1 \
--update-labels=limpieza=activa
# Cleanup policy: delete untagged images older than 30 days,
# always keeping the 10 most recent versions
gcloud artifacts repositories set-cleanup-policies alpinashop \
--location=europe-west1 --policy=politica-limpieza.json[
{
"name": "borrar-antiguas-sin-etiqueta",
"action": {"type": "Delete"},
"condition": {"tagState": "UNTAGGED", "olderThan": "30d"}
},
{
"name": "conservar-recientes",
"action": {"type": "Keep"},
"mostRecentVersions": {"keepCount": 10}
}
]
- The complete AlpinaShop case: before and after
July's bill: €1,058
| Service | € | Comment |
|---|---|---|
| Cloud Composer | 280 | 🔴 The evaluation environment from 04-06, never deleted |
Cloud SQL (alpinashop-pedidos HA + replica) |
150 | Regional HA + reporting replica |
| Vertex AI (endpoint) | 140 | 🔴 The recommender endpoint deployed "just to test" |
| GKE Autopilot | 120 | The catalogue cluster |
| Dataflow | 90 | The streaming orders job |
| BigQuery | 75 | 15 storage + 60 queries |
| Compute Engine (MIG) | 50 | 2 × e2-medium 24×7 |
| Cloud Logging / Monitoring | 45 | Default retention, no exclusions |
| Network egress | 35 | Egress to the internet and between zones |
| Cloud Storage | 25 | 4 buckets, no lifecycle |
| Load balancer + IP | 20 | |
| Cloud Build | 15 | |
| Pub/Sub | 5 | |
| Secret Manager + KMS | 5 | |
| Cloud Functions | 3 | procesar-imagen-producto |
| Total | €1,058 |
The two findings Marta could not explain are the first two rows in red: €420 a month, 40 % of the bill, on two resources that gave nobody any value. The Composer one had been there four months; the Vertex AI endpoint, three. Accumulated cost of the oversight: more than €1,500.
And the detail that makes it painful: DA-002 expressly decided that recommendations would be produced in batch and not with a 24×7 endpoint. The decision was written down and it was correct. What was missing was deleting the test endpoint.
An architecture decision only saves money if somebody carries out the "and now switch off the other thing" part.
The bill afterwards: €425
| Service | Before | After | What was done |
|---|---|---|---|
| Cloud Composer | 280 | 0 | 🗑️ Deleted. Workflows is used (04-06) |
| Vertex AI | 140 | 8 | 🗑️ Endpoint removed; batch predictions (DA-002) |
| GKE Autopilot | 120 | 0 | 🗑️ Retired after 07-02 |
| Cloud SQL | 150 | 105 | Rightsized + a 1-year commitment on the stable part |
| Dataflow | 90 | 55 | Spot secondary workers + rightsizing |
| BigQuery | 75 | 38 | Partitioning + clustering + require_partition_filter |
| Compute Engine | 50 | 0 | 🗑️ MIG switched off (07-02) |
| Cloud Run | 0 | 7 | ✨ Replaces the MIG |
| Logging / Monitoring | 45 | 28 | Noise exclusions and adjusted retention (06-06) |
| Egress | 35 | 22 | CDN + compression |
| Cloud Storage | 25 | 14 | Lifecycle and a version limit |
| Load balancer + IP | 20 | 20 | Unchanged |
| Cloud Build | 15 | 12 | Clean-up of old images |
| Pub/Sub | 5 | 5 | |
| Secret Manager + KMS | 5 | 5 | |
| Cloud Functions | 3 | 3 | |
| HA Cloud VPN (2 tunnels) | 0 | 60 | ✨ New: the link to Sabadell (07-03) |
| VPC Flow Logs | 0 | 8 | ✨ New: network visibility |
| Data access logs | 0 | 35 | ✨ New: a requirement from 07-04 |
| Total | €1,058 | €425 | −60 % |
The honest reading
What was really saved, in order of impact:
- Deleting two forgotten resources: €412. 65 % of the total saving came from half an hour running an audit script. No technical optimization comes close to that return.
- Architecture decisions: €163. GKE and the MIG retired in favour of Cloud Run. But be careful about attributing everything here: Cloud Run was not chosen to save money, it was chosen for operational effort and portability (DA-001). The saving was a pleasant consequence, not the objective.
- Configuration optimizations: €76. Partitioning, lifecycle, spot, log exclusions. Real engineering work for a third of the saving that deleting two things gave.
- Use commitment: €20. Deliberately modest. Only on Cloud SQL, which is the only thing that is not going to change architecture within a year.
What was NOT saved, and why:
| Item | € | Why it stays |
|---|---|---|
| Global load balancer + IP | 20 | It is the price of having CDN, WAF, managed TLS and an anycast IP. It is now 5 % of the bill and 74 % of the catalogue's cost: it is not touched |
| Cloud SQL HA | 105 | High availability costs double. It is paid on purpose (07-06) |
| Reporting replica | Included | It stops Lucía's queries affecting orders. It is worth its price |
| Egress | 22 | Serving the shop has a cost. It is already optimised with CDN |
What it was decided to spend MORE on, on purpose — and this is the part that almost never appears in a case study:
| Item | € | Why |
|---|---|---|
| HA Cloud VPN, two tunnels | 60 | One alone would cost half and would be a single point of failure. You pay for redundancy |
| VPC Flow Logs | 8 | Without them you cannot investigate a network incident |
| Data access logs | 35 | Debt #5 from 07-04. It is a de facto legal requirement: without them a breach cannot be bounded |
€103 a month of new, deliberate spend on reliability and security. Without that nuance, the headline "we have cut the bill by 60 %" would be a half-truth. The full truth is: €736 of waste was eliminated, €260 was optimised and €103 was reinvested in things that were needed.
Optimising cost is not cutting. It is stopping paying for what adds nothing so you can pay for what does.
Unit cost: the metric that really matters
The absolute number misleads. What you have to track is the cost per unit of business:
| Metric | July | After |
|---|---|---|
| Monthly bill | €1,058 | €425 |
| Orders per month | 1,200 | 1,200 |
| Cost per order | €0.88 | €0.35 |
| Cost as % of revenue (average basket ~€65) | 1.4 % | 0.55 % |
This is the metric to present to management, for two reasons. First, because it makes different months comparable: if in October the bill goes up to €600 but 3,000 orders are processed, the cost per order drops to €0.20 and the increase is good news. And second, because it turns infrastructure from "an expense to be cut" into "a variable cost of the business", which is what it actually is.
- Tools: Recommender, Active Assist and the calculator
| Tool | What for |
|---|---|
| Recommender | Recommendations by resource type: machines, disks, IPs, IAM, Cloud SQL |
| Active Assist | The umbrella grouping Recommender, Insights and the predictions |
| Pricing Calculator | Estimating before creating |
| Anomaly detection | Included in the billing console, free |
| Billing reports | Grouping and filtering without SQL |
| Cloud Billing API | Automating queries and prices |
# All of a project's cost recommendations, at a glance
for R in google.compute.instance.MachineTypeRecommender \
google.compute.disk.IdleResourceRecommender \
google.compute.address.IdleResourceRecommender \
google.compute.image.IdleResourceRecommender \
google.cloudsql.instance.IdleRecommender \
google.cloudsql.instance.OverprovisionedRecommender; do
echo "=== $R ==="
gcloud recommender recommendations list \
--project=alpinashop-prod --location=europe-west1-b --recommender="$R" \
--format="table(description, primaryImpact.costProjection.cost.units)" 2>/dev/null
doneA tip about the calculator: use it before creating, not after the bill. Ten minutes estimating the cost of an architecture avoids next month's surprise, and above all it lets you compare two designs in euros rather than in hunches.
Common Mistakes and Tips
- Optimising before measuring. You end up fine-tuning what is visible while €280 a month goes on something nobody knew existed.
- Forgetting the
creditsarray in billing queries. The figures come out inflated and nobody understands why they do not match the console. - Querying the billing table without a partition filter. The query that analyses your cost costs you money.
- Believing that a budget caps spend. It only warns. Stopping it has to be automated, and carefully.
- Automating a production shutdown on budget. A sales spike would switch off the shop on the best day of the year.
- Committing for 1 or 3 years before stabilising the architecture. If you migrate afterwards, you pay for what you no longer use.
- Committing 100 % of your consumption. Only the stable part, 50-70 % of the historical minimum.
- Putting Archive on data that is queried every couple of months. The early deletion charges make it more expensive than Standard.
- BigQuery tables without
require_partition_filter. ASELECT *with no filter can cost more than a month of servers. - Bucket versioning with no version limit. The bucket grows indefinitely and nobody looks at it.
- Labels with free-form values.
equipo: variosdestroys the analysis. A closed set and validation in Terraform. - Reserving static IPs "just in case". A reserved IP that is not in use costs more than one in use.
- Deleting a VM without marking the disk for automatic deletion. The orphaned disk keeps billing.
- Tip: run the phantom script on the first Monday of every month. It is the highest-return activity per hour in the whole lesson.
- Tip: track cost per unit of business, not the absolute figure. It is the only thing that makes different months comparable.
- Tip: the monthly 30-minute meeting with the dashboard open is the most profitable FinOps practice there is.
- Tip: put
--max-instanceson everything that scales. It is a cost firewall (07-02). - Tip: use the calculator before creating. Comparing two designs in euros is better than arguing about them on hunches.
Exercises
Exercise 1 — Investigating a bill increase
AlpinaShop's bill has gone from €425 to €690 from one month to the next. There has been no campaign and no big deployments. Management is asking what has happened.
Write the complete sequence of SQL queries over the billing export that you would use to get to the root cause, explaining what you are looking for at each step and how you interpret the result. State at least four plausible causes and how you would tell them apart with data.
Exercise 2 — Optimising the data environment
The alpinashop-datos project costs €250 a month:
- BigQuery queries: €95 (on demand)
- BigQuery storage: €40
- Dataflow: €70
- Cloud Storage (
alpinashop-datalake): €45
Additional facts: the eventos_web table holds 1.2 TB, is not partitioned and is queried around 200 times a day from a dashboard. The Dataflow job processes orders in streaming 24 hours a day, even though almost all orders arrive between 09:00 and 23:00. The data lake holds 3 TB, of which 80 % are files more than a year old that are only read for annual audits.
Propose an optimization plan with the estimated saving of each measure, the effort, and the associated risk. Rank it by return.
Exercise 3 — Presenting the case to management
Prepare Marta's presentation to AlpinaShop's board on cost management over the last quarter. It must include: what has been achieved, how it was achieved, what is deliberately overspent, what metric will be tracked from now on, and what is being asked for next quarter.
Write it the way you would say it to a board that is not technical. One page maximum. And then explain which three communication decisions you took and why.
Solutions
Solution 1 — Investigating a bill increase
Step 1 — Locate the service and the moment. You never start with the detail: first you narrow it down.
SELECT
service.description AS service,
ROUND(SUM(CASE WHEN DATE(usage_start_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
THEN cost END), 2) AS current_month,
ROUND(SUM(CASE WHEN DATE(usage_start_time) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 60 DAY)
AND DATE_SUB(CURRENT_DATE(), INTERVAL 31 DAY)
THEN cost END), 2) AS previous_month
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 60 DAY)
GROUP BY service
HAVING current_month > 1
ORDER BY (current_month - IFNULL(previous_month, 0)) DESCThe first row gives you the service responsible for most of the €265.
Step 2 — Work out the shape over time. This is the step that gives the most information and the one almost nobody does:
SELECT
DATE(usage_start_time) AS day,
ROUND(SUM(cost), 2) AS cost_eur
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 60 DAY)
AND service.description = 'SUSPECT_SERVICE'
GROUP BY day
ORDER BY dayThe shape of the curve is the diagnosis:
| Shape | Meaning |
|---|---|
| A step that starts one day and stays there | A permanent resource was created. A phantom or a deployment |
| A spike of one or two days | A one-off job, a query, a test |
| A rising ramp | Something accumulating: storage, logs, snapshots |
| Higher sawteeth | An increase in traffic or in the frequency of a process |
Step 3 — Drill down to SKU and resource.
SELECT
sku.description AS sku,
resource.name AS resource_name,
ROUND(SUM(cost), 2) AS cost_eur,
ROUND(SUM(usage.amount), 2) AS usage_amount,
ANY_VALUE(usage.unit) AS unit
FROM `alpinashop-datos.facturacion.gcp_billing_export_resource_v1_XXXXXX`
WHERE DATE(_PARTITIONTIME) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
AND service.description = 'SUSPECT_SERVICE'
GROUP BY sku, resource_name
ORDER BY cost_eur DESC LIMIT 20Step 4 — Correlate with the changes. With the date of the step:
FECHA="2026-09-14"
gcloud logging read "
protoPayload.methodName=~'(create|insert|Create|Insert)'
AND timestamp>=\"${FECHA}T00:00:00Z\"
AND timestamp<=\"${FECHA}T23:59:59Z\"" \
--project=alpinashop-datos --limit=100 \
--format="table(timestamp, protoPayload.methodName,
protoPayload.authenticationInfo.principalEmail,
protoPayload.resourceName)"The four plausible causes and how to tell them apart:
| Cause | Signature in the data | Confirmation |
|---|---|---|
| A new, forgotten resource | A clean step, the same amount every day, a single resource.name |
Audit logs from the day of the step; the phantom script |
| A runaway BigQuery query | A spike in the "Analysis" SKU; usage.amount in TB out of line |
INFORMATION_SCHEMA.JOBS_BY_PROJECT ordered by total_bytes_billed |
| Growth in storage or logs | A gentle ramp, no step; "Storage" SKUs | The bucket size or the Logging ingestion volume |
| A real increase in traffic | Egress goes up and Cloud Run and load balancer requests all at once | Business metrics: if orders are up, it is good news |
The fourth deserves the nuance that gives you judgement: an increase proportional to business activity is not a cost problem. You verify it with the unit metric from section 12: if the cost per order holds steady, the business has grown; if it shoots up, there is inefficiency.
For the BigQuery case, the definitive query:
SELECT
user_email,
DATE(creation_time) AS day,
COUNT(*) AS queries,
ROUND(SUM(total_bytes_billed)/POW(1024,4), 2) AS tib_billed,
ROUND(SUM(total_bytes_billed)/POW(1024,4) * 5.5, 2) AS estimated_cost_eur
FROM `alpinashop-datos.region-eu.INFORMATION_SCHEMA.JOBS_BY_PROJECT`
WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
AND job_type = 'QUERY' AND state = 'DONE'
GROUP BY user_email, day
ORDER BY tib_billed DESC LIMIT 20And as soon as the cause is identified, the action is not just to fix it: it is to put in the control that stops it happening again — require_partition_filter if it was a query, the monthly script if it was a phantom, an anomaly alert if it was a ramp.
Solution 2 — Optimising the data environment
Preliminary analysis. The four items have very different causes and the order of attack is not the order of the list.
Measure 1 — Partition and cluster eventos_web. (Saving: ~€75/month · Effort: 4 h · Risk: low)
A 1.2 TB unpartitioned table, queried 200 times a day, is the definition of the problem. Every query reads 1.2 TB unless BigQuery can prune, and without a partition it cannot.
CREATE OR REPLACE TABLE `alpinashop-datos.alpinashop_analitica.eventos_web_opt`
PARTITION BY DATE(momento)
CLUSTER BY tipo_evento, pagina
OPTIONS (
partition_expiration_days = 400,
require_partition_filter = TRUE
)
AS SELECT * FROM `alpinashop-datos.alpinashop_analitica.eventos_web`;Estimate: the dashboard typically queries the last 30 days → from 1.2 TB to around 30 GB per query, and clustering trims more. A realistic reduction of 80-85 % of the query cost: €95 → ~€20.
Risk and mitigation: require_partition_filter will break existing queries with no filter. Roll it out in two phases: create the optimised table, migrate the dashboard and the saved queries, verify, and only then replace the original. Never the other way round.
Free bonus: partition_expiration_days = 400 removes events older than 400 days on its own, which also reduces storage over time.
Measure 2 — Data lake lifecycle. (Saving: ~€28/month · Effort: 1 h · Risk: very low)
2.4 TB (80 % of 3 TB) are more than a year old and are only read during annual audits. They are in Standard.
lifecycle_rule {
condition { age = 90 }
action { type = "SetStorageClass" storage_class = "NEARLINE" }
}
lifecycle_rule {
condition { age = 365 }
action { type = "SetStorageClass" storage_class = "COLDLINE" }
}
lifecycle_rule {
condition { age = 1095 }
action { type = "SetStorageClass" storage_class = "ARCHIVE" }
}The sums: 2.4 TB moving from Standard to Coldline (~0.3×) saves around 70 % of its share. €45 → ~€17.
Why Coldline and not Archive for the one-year band: with a 365-day minimum storage duration, Archive punishes any early read. An annual audit sits right on the boundary and could trigger the charge. Coldline (90 days) is the safe point; Archive is reserved for anything over three years old, which nobody is going to touch.
Measure 3 — Dataflow: rethinking the streaming. (Saving: ~€35/month · Effort: 1-2 days · Risk: medium)
A 24×7 streaming job when orders arrive between 09:00 and 23:00. Three options, and the choice depends on a business requirement, not a technical one:
| Option | Saving | Impact |
|---|---|---|
| (a) Spot secondary workers + rightsizing | ~€25 | None. Start here |
| (b) Streaming only from 08:00 to 24:00; Pub/Sub retains overnight | ~€25 | Night-time orders appear in the morning |
| (c) Replace with a batch process every 15 min | ~€50 | 15 min of latency in the analytics |
The question that decides it: does anybody look at the orders data in real time? At AlpinaShop, the honest answer is no: Lucía looks at the dashboard in the morning. Option (c) is the right one and the one that saves the most, but it requires rewriting the pipeline and that is why it is proposed after (a), which is free.
A realistic plan: apply (a) this week (~€25), and evaluate (c) next quarter.
Measure 4 — BigQuery long-term storage. (Saving: ~€8/month · Effort: 30 min · Risk: none)
BigQuery automatically applies a long-term storage price (~50 % less) to partitions not modified in 90 days. The trap: any UPDATE on a partition resets the counter.
SELECT table_name, partition_id,
ROUND(total_logical_bytes/POW(1024,3), 2) AS gib,
last_modified_time
FROM `alpinashop-datos.alpinashop_analitica.INFORMATION_SCHEMA.PARTITIONS`
WHERE total_logical_bytes > 0
ORDER BY total_logical_bytes DESCIf old partitions turn up with a recent last_modified_time, some process is rewriting them unnecessarily. Fixing it is free and the discount applies on its own.
It is also worth evaluating the physical billing model against the logical one: with highly compressible data it can work out cheaper, although the long-term discount works differently. You compare with INFORMATION_SCHEMA before changing.
Plan ranked by return:
| Order | Measure | Saving | Effort | €/hour |
|---|---|---|---|---|
| 1 | Data lake lifecycle | €28 | 1 h | 28 |
| 2 | Partition and cluster | €75 | 4 h | 19 |
| 3 | Long-term storage | €8 | 0.5 h | 16 |
| 4 | Dataflow spot (a) | €25 | 4 h | 6 |
| 5 | Dataflow to batch (c) | +€25 | 12 h | 2 |
Result: €250 → ~€115, 54 % less, with the first four measures taking around two working days.
And the measure that costs nothing and has to be applied on day one, before any of the above:
gcloud alpha services quota update \
--service=bigquery.googleapis.com \
--consumer=projects/alpinashop-datos \
--metric=bigquery.googleapis.com/quota/query/usage \
--unit=1/d/{project}/{user} --value=549755813888 # 512 GiB/user/dayBecause everything above optimises current consumption, but nothing stops somebody writing a query tomorrow that costs €300 in an afternoon. The quota is the only control that does.
Solution 3 — Presenting the case to management
Managing the cost of the technology platform — Third-quarter report Marta Ruiz, infrastructure lead
Summary. The monthly cost of our technology platform has gone from €1,058 to €425, 60 % less, without reducing any capability and having added security and continuity measures we did not have before. In business terms, the technology cost per order has come down from €0.88 to €0.35: from 1.4 % to 0.55 % of the average order value.
Where the saving comes from. Two thirds does not come from negotiating better or from cutting: it comes from having found two services switched on that nobody was using — a tool we evaluated in the spring and a recommendations model we tested in the summer — that together cost €420 a month. No person decided to spend it; nobody simply decided to switch it off. It is now fixed, and from this month we run an automatic review on the first Monday of every month so that it does not happen again. The lesson is that in the cloud spend grows by omission, not by decision, and that demands a routine.
The remaining third is technical improvements: we have moved the shop to a technology that only charges when there are visitors — overnight we pay nothing — we have reorganised the data so the dashboard queries read only what they need, and we have moved old files to cheaper storage.
What we overspend on, on purpose. I want it stated explicitly that €103 a month is new, deliberate spend: the connection to the Sabadell shop is duplicated so that an outage does not leave us without stock figures, and we have enabled the record of who accesses customer data, which is what would let us answer the Data Protection Agency if there were ever an incident. Without that record we could not bound what happened, and that is a legal exposure, not just a technical one. Optimising cost does not mean cutting back on reliability or compliance: it means stopping paying for what adds nothing so we can pay for what does.
The metric we will track. From now on we will report the cost per order, not the total. It is important to understand why: during the autumn campaign the bill is going to go up, and that will be good news. If we sell three times as much, the cost per order will fall even though the total amount grows. The absolute number says nothing on its own; the unit one does.
What we are asking for next quarter. No additional budget. Just two decisions:
- Authorisation for a one-year commitment on the database, which is the only thing we know for certain will not change. It saves around 30 % of that line item in exchange for committing.
- Half an hour a month of management's time to review the cost dashboard with us. It is the cheapest and most effective measure of all: what gets looked at as a group does not get out of hand.
The three communication decisions, and why:
1. Leading with the business number, not the technical one. The headline is not "we have migrated to Cloud Run": it is "the cost per order has come down from €0.88 to €0.35". A board cannot evaluate an architecture decision, but it knows perfectly well what it means for each order to cost half as much. Translating into the unit of the business is not simplifying: it is speaking in the currency they take decisions in.
2. Telling them about the failure before they ask. The saving could have been presented as a technical achievement while keeping quiet about €420 going on two forgotten resources. Saying it has three effects that more than make up for the discomfort: it gives credibility to everything else in the report, it justifies the monthly routine — which without the anecdote would look like unnecessary bureaucracy — and it protects you from the awkward question six months from now. A report that only tells you about successes is only half believed.
3. Anticipating the bad news and reframing it. The bill is going to go up during the campaign. If that is explained afterwards, it looks like an excuse; explained beforehand, it is foresight — and it also teaches the board to read the right metric. It is the same reason the €103 of new spend is declared prominently instead of being hidden in the net figure: if management discovers on its own a cost it was not told about, it loses confidence in the whole report, however good the result is.
And a fourth that is not visible but is there: it does not ask for money, it asks for decisions. A report that ends by asking for budget is read with suspicion; one that ends by asking for half an hour a month and an authorisation that saves money is read as management.
Conclusion
AlpinaShop's bill has stopped being a monthly surprise and become a figure that is understood, attributed and governed.
You know why the cloud takes you by surprise: variable cost, technical decisions with financial effects, the absence of a natural brake and the time lag — with the first rule engraved: the problem is almost never that something is expensive, but that nobody knew it was switched on.
You know how to read a bill: service, SKU and usage, with the realisation that a VM is not one cost but five different SKUs. You can tell the discounts that apply themselves — sustained use — from the ones you have to sign up for, and you know that the sustained one explains why switching off at night saves less than the arithmetic promises.
You have the export to BigQuery configured with detailed cost — without which you know Storage costs €25 but not which bucket — and the six queries that answer the real questions, with the two classic mistakes avoided: forgetting the credits array and querying without a partition filter. Including the most uncomfortable query, the one about unlabelled spend, which at AlpinaShop revealed that 36 % of the bill had no owner.
You understand FinOps — inform, optimise, operate — with the order that matters: first measure, then cut. And the responsibility model for a small company, with the highest-return practice in the whole discipline: a thirty-minute meeting once a month with the dashboard open.
You know how to set up budgets with the forecast rule that warns on the 8th and not on the 26th, several of them rather than one, and the automatic action via Pub/Sub and a function — with the three warnings: never shut down production, unlinking billing is destructive, and messages arrive more than once. And you know the only hard brake is quotas, including BigQuery's byte quotas, which should always be in place.
You have the nine levers ranked by return, with the first one far ahead of the rest: removing what is not used. And the others with their traps spelled out — sizing according to August when the peak is in October, committing before stabilising the architecture, putting Archive on data that is read every couple of months, versioning with no version limit. With require_partition_filter singled out as the most profitable option in the whole of BigQuery: ninety times cheaper for the same query.
You know where the cost of traffic hides and you have the labelling discipline turned into a real control, with Terraform validations that make the plan fail if a label is missing or its value is not in the closed set — prevention at the point of creation, which always beats chasing orphans afterwards.
You hunt anomalies with statistics and phantoms with a script to be run on the first Monday of every month: reserved IPs that cost more than the ones in use, orphaned disks, eternal snapshots, Vertex AI endpoints, test clusters, Composer environments and load balancers with no backends.
And you have the complete case: €1,058 broken down with two line items in red that added up to 40 % of the bill and had been sitting there for months, against €425 afterwards. With the honest reading that is the most valuable thing in the section: 65 % of the saving came from half an hour running a script, the architecture contributed less than it seems — and Cloud Run was not chosen to save money anyway — the technical optimizations gave a third of what deleting two things gave, and €103 a month is new, deliberate spend on redundancy, network visibility and data access logs. Plus the metric to present from now on: cost per order, which turns infrastructure from an expense to be cut into a variable cost of the business.
Out of all of it comes one sentence that sums up the whole lesson and links to what comes next:
Optimising cost is not cutting. It is stopping paying for what adds nothing so you can pay for what does.
And that sentence has a counterpart that has to be said just as clearly. In this lesson it was decided to pay for Cloud SQL's high availability without discussing it, the VPN tunnels were duplicated in the knowledge that one would cost half, and it was accepted that the global load balancer — now 74 % of the catalogue's cost — is not to be touched. Three reliability decisions taken on intuition, without an objective that justifies them in numbers.
Because AlpinaShop still has not answered the most basic question of all: what exactly does it mean for the shop to "work well"? There have been alerts since 06-04, but an alert tells you something happened, not whether the service is delivering what it promises. There is no declared objective, no way of knowing whether 43 minutes of downtime a month is acceptable or unacceptable, no criterion for deciding whether it is safe to deploy on a Friday, and — most serious of all, already flagged as critical debt in 07-04 — nobody has ever checked that the backups can be restored.
The next lesson answers all of that with numbers: SLIs, SLOs, error budgets, tiered high availability architectures, failure modes and their mitigation, and a disaster recovery plan with RTO and RPO — including the restore drill for alpinashop-pedidos that has been outstanding for far too long.
Google Cloud Platform (GCP) Course
Module 1: Introduction to Google Cloud Platform
- What is Google Cloud Platform?
- Setting Up Your GCP Account
- A Tour of the GCP Console
- Projects, Resource Hierarchy and Billing
- Regions, Zones and the Shared Responsibility Model
- Cloud Shell and the gcloud CLI
Module 2: Core GCP Services
- Compute Engine: Virtual Machines on Google Cloud
- Cloud Storage: Object Storage
- Cloud SQL: Managed Relational Databases
- App Engine: Platform as a Service
- Google Kubernetes Engine (GKE)
- NoSQL Databases: Firestore, Bigtable and Spanner
- How to Choose the Right Compute Service
Module 3: Networking and Security
- VPC Networks
- Cloud Load Balancing
- Cloud CDN
- Identity and Access Management (IAM)
- Cloud Armor
- Secrets and Encryption: Secret Manager and Cloud KMS
- Cloud DNS, TLS Certificates and Publishing Services Securely
Module 4: Data and Analytics
- BigQuery: The Analytical Data Warehouse
- Cloud Dataflow: Batch and Streaming Data Processing
- Cloud Dataproc: Managed Spark and Hadoop
- Cloud Pub/Sub: Asynchronous Messaging
- Cloud Data Fusion: Code-Free Data Integration
- Orchestrating Pipelines with Cloud Composer and Workflows
- Data Governance and Dashboards with Dataplex and Looker Studio
Module 5: Machine Learning and AI
- Vertex AI: The Machine Learning Platform on GCP
- AutoML: Custom Models Without Writing Code
- TensorFlow on GCP: Training and Serving Models
- Natural Language API
- Vision API
- Generative AI on Vertex AI: Gemini Models and Embeddings
- MLOps: From Model to Product with Vertex AI Pipelines
Module 6: DevOps and Monitoring
- Cloud Build: Continuous Integration on GCP
- Cloud Source Repositories and Source Code Management
- Cloud Functions: Serverless Functions
- Cloud Monitoring (formerly Stackdriver): Metrics, Dashboards and Alerts
- Cloud Deployment Manager and Native Infrastructure as Code
- Cloud Logging and Cloud Trace: Logs, Traces and Diagnostics
- Terraform on GCP: Infrastructure as Code in Practice
Module 7: Advanced GCP Topics
- Hybrid and Multicloud with Anthos
- Serverless Computing with Cloud Run
- Advanced Networking: Shared VPC, Peering and Hybrid Connectivity
- Security Best Practices
- Cost Management and Optimization
- Reliability: SLOs, High Availability and Disaster Recovery
- Governance at Scale: Organization, Policies and Auditing
