Platform Engineering Best Practices: A Practitioner's Guide
I’ve been building platforms since 2018. Back then, “platform engineering” wasn’t even a job title. We called it “the infrastructure team that also writes documentation.” Today it’s a certified discipline — the Certified Cloud Native Platform Engineering Associate from the Linux Foundation and CNCF exists because the industry realized you can’t scale developer productivity with tickets and Slack pings.
Platform engineering is the practice of building a shared self-service layer that lets developers ship code without becoming infrastructure experts. It’s not DevOps renamed. It’s not a new tool for your backlog. It’s a product — with users, SLAs, and a roadmap. If you treat it like a project, it will die.
In this guide, I’ll walk you through what we’ve learned building platforms at SIVARO and from working with teams shipping production AI systems. I’ll give you specific patterns, code samples, and the hard trade-offs nobody puts in slide decks. You’ll leave knowing exactly where to start — and where not to.
Why Most Platform Teams Fail (and How Not To)
Most people think platform engineering is about tools — Kubernetes, Backstage, Crossplane, Terraform. They’re wrong. It’s about APIs and abstractions. If your platform doesn’t reduce cognitive load for developers, it’s just another thing they have to learn.
I’ve seen three failure modes repeat:
1. Building a platform for “everyone.” You design a golden path that accommodates every team’s weird stack. End result: a monstrous config file with 47 optional fields. Nobody uses your platform because navigating it is harder than just provisioning an EC2 instance manually.
2. Selling it as a productivity tool but measuring it like infrastructure. You track uptime and resource utilization. Developers track how long they wait for a staging environment. Those metrics don’t align. Your platform looks healthy; your users are miserable.
3. Launching before you have a single user. You spend six months building the perfect internal developer platform (IDP). Then you show it to a team and they say, “We already have a script that does this.” You’ve built a product nobody asked for.
The fix is brutally simple: start with a single team. Pick the one that screams the loudest about pain — maybe it’s the team waiting two weeks for a database to be provisioned. Build them a self-service button that provisions that exact database in ten minutes. Don’t generalize until you have three teams begging for it.
I like Microsoft’s framing in their platform engineering journey guide: you start with a “paved road” — a single, opinionated path — and widen it based on demand. Don’t pave the whole city in one sprint.
The Internal Developer Platform: Build vs. Buy
Every year someone tells you to build your platform from scratch. Every year someone else sells you a magic box that claims to be a platform. Both are wrong most of the time.
Here’s my rule of thumb: buy the shell, build the guts. Use Backstage or Port or Cortex as the developer portal — that’s the UI layer. But build your own orchestration logic, because that’s where your company’s specific policies, compliance rules, and deployment patterns live.
At SIVARO, we evaluated five commercial platforms in 2023. Three of them locked us into their provisioning engine. Two of them couldn’t integrate with our on-premise GPU clusters. We ended up with Backstage as the frontend and a custom orchestrator written in Go that wraps Terraform, Helm, and a Python-based scoring engine for AI model deployments.
Why not just use Terraform Cloud? Because we needed to enforce cost budgets per environment, run pre-deployment safety checks against our internal AI safety framework, and generate compliance reports for SOC 2. That logic is too specific to buy off the shelf.
If you’re starting today, use Platform Engineering University to understand the architecture patterns before you write a single line of code. Their certified practitioner track (Platform Engineering Certified Practitioner) covers exactly this build vs. buy decision. It saved us from rebuilding a portal that already existed.
Code: A Minimal Backstage Entity with Terraform Backend
Here’s what a developer sees when they ask for a new microservice in Backstage:
yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: fraud-detection-v2
annotations:
terraform/module: "sivaro/infrastructure//modules/service"
terraform/workspace: "fraud-detection-v2-prod"
platform/owner: "team-payments"
platform/sla-p99: "100ms"
spec:
type: service
lifecycle: production
owner: team-payments
system: payments
That annotation triggers a webhook to our orchestrator, which runs this Terraform:
hcl
module "service" {
source = "git::https://github.com/sivaro/infrastructure.git//modules/service?ref=v1.2.3"
name = "fraud-detection-v2"
environment = "production"
owner = "team-payments"
cpu_limit = "4"
memory_limit = "8Gi"
gpu_count = 1
gpu_type = "a100-80gb"
scaling = {
min_replicas = 2
max_replicas = 20
target_cpu = 70
}
}
No YAML hand-crafted by a human. No ticket. No waiting. That’s the point.
Golden Paths and Paved Roads: Making It Easy to Do the Right Thing
“Golden paths” sound like marketing fluff. They’re not. They’re a concrete set of decisions you make once so developers don’t have to make them every time.
What decisions? Language runtimes, base Docker images, CI/CD pipeline templates, observability agent versions, database types, secret management patterns, ingress controllers, service mesh sidecars. Every choice you bake into the golden path is a choice a developer doesn’t have to research.
But here’s the contrarian take: golden paths should be opinionated, not prescriptive. You provide a default. Developers can opt out, but opting out comes with a cost — they have to maintain their own pipeline template or security scanning script. That’s fine. Let them.
At SIVARO, our default golden path for AI services uses:
- Python 3.12 with uv package manager
- Docker base image based on slim-bookworm with GPU CUDA 12.4
- GitHub Actions template that runs lint, type-check, unit tests, integration tests, and a model-card validation
- Vllm as the inference engine
- Ray Serve for autoscaling
- A Prometheus exporter with preconfigured dashboards for token throughput and latency percentiles
We published this as an internal template repository. A developer forks it, adds their model code, and they’re in production in under an hour.
The KodeKloud Platform Engineer Learning Path has a good section on defining golden paths with ArgoCD and Crossplane. I’d also recommend looking at how Google thinks about it – their blog on becoming a platform engineer emphasizes that the golden path is a contract, not a constraint.
Code: A Golden Path Pipeline Skeleton
yaml
name: Deploy AI Model (Golden Path)
on:
push:
branches: [main]
env:
PLATFORM_VERSION: "v2026.07"
REGISTRY: ghcr.io/sivaro
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: make lint
- run: make type-check
- run: make unit-tests
- run: make model-card-validate
deploy-staging:
needs: validate
if: github.ref == 'refs/heads/main'
steps:
- uses: sivaro/platform-actions/deploy@v2
with:
environment: staging
timeout: 600s
deploy-prod:
needs: deploy-staging
environment: production
steps:
- uses: sivaro/platform-actions/deploy@v2
with:
environment: production
approval-required: true
Notice: no custom shell scripts. Every step calls a platform action. That’s how you enforce consistency – make the platform the only way to deploy.
Observability as a Platform Feature (Not an Afterthought)
Most platform teams treat observability as a checkbox. “We set up Prometheus and shipped a dashboard.” Then they wonder why developers still page them for “why is my service slow?”
Observability must be part of the platform’s contract. Every service deployed through your golden path should automatically emit:
- Structured logs with correlation IDs
- Metrics with service owner and environment labels
- Traces that connect to the database and upstream callers
We built a sidecar – we call it sivaro-agent – that runs alongside every container. It scrapes a standard port, enriches metrics with platform metadata, and forwards to a central OpenTelemetry collector. Developers don’t configure exporters anymore. They just set an environment variable: OTEL_SERVICE_NAME=fraud-detection-v2.
The difference in debugging speed is massive. Before, a developer would say “my API is slow” and we’d spend 20 minutes figuring out which instance it was running on. Now, they open a prebuilt Grafana dashboard that shows latency by model version and input token count. They see the regression within 30 seconds.
We learned this pattern from the observability modules in the Platform Engineering University curriculum. If you want your developers to trust your platform, you have to show them you can help them when something breaks. Not just when it’s running steady.
Day 2 Operations: The Thing Nobody Talks About
Everyone loves building the platform. Nobody loves running it.
But Day 2 is where platforms fail. The first six months, you’re the hero. Month seven, you’re maintaining hundreds of Terraform state files, upgrading Kubernetes versions, rotating TLS certificates, and fixing broken Helm charts that someone “just quickly deployed.”
Here’s what I wish someone told me in 2020: your platform must be self-healing where possible, and observable in its own right. You need:
- Automated drift detection – if someone manually edits a Deployment via
kubectl edit, your platform should flag it and optionally revert it. - Upgrade automation – we use ArgoCD’s sync waves to update dependencies in a controlled order.
- A platform SLO – we target 99.9% availability for the control plane (Backstage + the orchestrator API). The services running on top have their own SLOs; they’re separate concerns.
And please, for the love of everything, do not let developers SSH into platform components. We banned shell access to our Kubernetes cluster nodes in 2022. Everything goes through the platform API. That forces us to have good APIs instead of good Sysadmins.
The CNCF’s Certified Cloud Native Platform Engineering Associate covers platform operations and lifecycle management explicitly. It’s worth the study time just to understand the mental model of treating your platform as a product that needs maintenance, not a one-time build.
Security and Compliance: Embed, Don’t Graft
Security teams love to block. Platform engineers love to enable. These two things don’t have to conflict.
The secret is embedding security controls into the golden path, not bolting them on after deployment. Run SAST, DAST, and dependency scans as part of the pipeline, not as a separate gate that someone has to open manually.
At SIVARO, we enforce:
- All images must be signed with Cosign and verified at admission control
- Secrets must be injected via External Secrets Operator, never as plaintext in manifests
- Network policies are generated automatically based on the service’s Backstage dependency graph
- Every production deployment requires a signed approval from the service owner’s security contact
We made these non-negotiable. Developers who try to bypass them hit an error message that says “Platform policy violation. Contact #platform-team in Slack.” Nine times out of ten, they’re trying to do something legitimate – we just hadn’t built a way to do it safely yet. That feedback loop is how the platform evolves.
Google’s platform engineering guide How to become a platform engineer has a good section on “security as a platform concern.” They’re right – if you leave security to a separate review board, you’ll ship slowly. If you embed it, you ship safely and fast.
Measuring Success: Are You Actually Helping Developers?
Vanity metrics will kill your platform team. “We onboarded 500 developers” means nothing if those developers are still waiting two days for a database.
We track three metrics:
-
Time to first deploy – how long from code push to production? Our current median is 8 minutes for standard services, 45 minutes for AI model deployments that need GPU validation.
-
Deployment frequency per developer – if your platform makes it harder to ship, this number goes down. We measure it per team, per month.
-
Support ticket deflection – how many fewer infra tickets are you handling? We track the ratio of platform self-service actions vs. human-requested actions. We want it above 90%.
The Microsoft platform engineering journey suggests adding a developer satisfaction survey every quarter. We do it. The results aren’t always flattering. But they tell us exactly where to invest next.
One more thing: don’t measure platform team output. Measure developer outcomes. Nobody cares that you wrote 15 Backstage plugins. They care that their staging environment provisioned in 3 minutes instead of 3 hours.
The Organizational Side: Convincing Leadership
Platform engineering doesn’t happen in a vacuum. You need executive buy-in, or you’ll starve when the next reorg hits.
The pitch is simple: platform engineering reduces time-to-market and infrastructure cost per developer. Show them data. At SIVARO, after two years of platform investment, our developer-to-environment ratio went from 1:0.2 (one environment shared by five devs) to 1:2 (each dev gets two disposable environments). Our cloud spend per developer dropped 30% because we automated cleanup of idle resources.
If your CFO asks “Why are we paying for Backstage?” you better have the numbers ready.
The Platform Engineering Certified Practitioner certification includes a module on organizational change management. I wish I’d studied it before trying to convince my CTO that platform engineering wasn’t a new name for DevOps.
FAQ
Q: Should I build a platform if my company has fewer than 50 engineers?
Probably not. The overhead of maintaining a platform team is high. Instead, use managed solutions (GitLab CI, Vercel, Render) and standardize on a single stack. Revisit when you hit 3+ teams that share infra.
Q: How long does it take to build a useful internal developer platform?
First usable version? 4–6 weeks for a very narrow path (one service type, one stack). Production-ready with golden paths and compliance? 6–9 months. Don’t promise the moon in Q1.
Q: What’s the biggest mistake platform teams make?
Overengineering for day one. You don’t need multi-region disaster recovery when you have three services. Build for the current scale, plan for the next six months, ignore the rest.
Q: Do I need to use Backstage?
No. But you need some kind of developer portal. We use Backstage because it’s open source and extensible. Some teams love Port or Cortex. The critical feature is a service catalog that ties together your provisioning, observability, and compliance data.
Q: How do I handle legacy services that don’t fit the golden path?
Don’t force them. Give legacy teams a migration window. In the meantime, let them use a “brownfield” path that only provides documentation and monitoring. They’ll eventually want the golden path when they see how fast new services ship.
Q: What about cost management?
Embed cost labels into every resource provisioned. Show developers their per-service cost in the portal. We use KubeCost + custom Prometheus metrics. Chargebacks are optional, but visibility is mandatory.
Q: Is platform engineering the same as DevOps?
No. DevOps is a cultural practice. Platform engineering is a product discipline that operationalizes that culture. Think of it like this: DevOps says “you build it, you run it.” Platform engineering gives you the tools to actually run it.
Conclusion
Platform engineering is the hardest thing I’ve ever built – and the most rewarding. When you get it right, developers stop asking you for help. They just ship. That silence is the best feedback you can get.
Start small. Pick one team. Automate their biggest pain. Prove the model. Scale from there. Ignore anyone selling you a turnkey “platform in a box.” The value is in the abstractions you design for your company, not the vendor you buy from.
The resources I linked – the Linux Foundation certification, the Platform Engineering University, and the KodeKloud learning path – will give you the foundation. The rest comes from shipping, breaking, fixing, and shipping again.
Now go build the thing your developers actually need.
Nishaant Dixit — Founder of SIVARO. Building data infrastructure and production AI systems since 2018. Built systems processing 200K events/sec.