LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Crossplane: Kubernetes-Native Cloud Infrastructure

kubernetescrossplaneinfrastructure-as-codeawsclouddevopsplatform-engineeringgitops

Crossplane is a control plane framework that turns Kubernetes into a platform for provisioning and managing cloud infrastructure. Rather than running terraform apply from a CI job, you push YAML to Kubernetes and Crossplane’s reconciliation loop ensures the real world matches what you declared. An EC2 instance, an RDS database, a GCS bucket—all of them become Kubernetes objects with standard kubectl tooling, RBAC, and GitOps workflows.

This is a compelling idea on paper and a genuinely useful pattern for platform teams building self-service infrastructure APIs. It is also an architecture with real operational costs, specific failure modes, and constraints that are not obvious until you have run it in production. This post covers the core model in depth, how to write Compositions, how authentication works, where Crossplane wins and loses against Terraform, and what production operation looks like before you commit to it.


The Core Object Model

Understanding Crossplane starts with five types of objects and how they relate. These are not metaphors—they are actual Kubernetes custom resource kinds, and the distinction between them is what makes the abstraction work.

Managed Resources

A Managed Resource (MR) is a Kubernetes object that directly represents one external infrastructure resource. RDSInstance, S3Bucket, VPC, SecurityGroup—each maps one-to-one to a real thing in your cloud account. MRs are cluster-scoped and live in the Crossplane provider’s API group. They look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
apiVersion: ec2.aws.upbound.io/v1beta1
kind: VPC
metadata:
  name: production-vpc
spec:
  forProvider:
    region: us-east-1
    cidrBlock: 10.0.0.0/16
    enableDnsHostnames: true
    enableDnsSupport: true
    tags:
      Environment: production
  providerConfigRef:
    name: aws-provider-config

The spec.forProvider block contains the configuration that the provider sends to the cloud API. The provider reconciles this object on a continuous loop—typically every 10 minutes at steady state—comparing the actual cloud resource to spec.forProvider and updating it if they differ. Drift correction is automatic; no terraform plan is required.

MRs are the lowest level of the abstraction. Platform engineers write Compositions that create MRs; application developers should rarely, if ever, create MRs directly.

Composite Resources and Claims

A Composite Resource (XR) is an abstraction over one or more MRs. You define what fields an XR exposes—“I want a database, I need to specify the storage size and engine version”—and a Composition defines how those fields map to actual cloud resources. XRs are cluster-scoped.

A Claim (also called a Composite Resource Claim or XRC) is a namespaced version of an XR. Claims allow teams to request infrastructure within their namespace without needing cluster-admin access. A platform engineer creates the XR definition; an application developer creates a Claim in their team’s namespace.

Cluster level:
  CompositeDatabase (XR)
  └── creates: SecurityGroup, DBSubnetGroup, RDSInstance

Namespace level:
  Database claim (XRC) in team-payments namespace
  └── references and creates: a CompositeDatabase

The claim-to-composite relationship is one-to-one. Creating a Claim creates a corresponding Composite Resource. Deleting the Claim deletes the Composite Resource and cascades to the MRs below.

Composite Resource Definitions

A CompositeResourceDefinition (XRD) is a cluster-scoped CRD definition that creates new Kubernetes API types. When you apply an XRD, Crossplane registers a new CRD in the cluster—e.g., CompositeDatabases.platform.example.com—with the OpenAPI schema you specify.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: compositedatabases.platform.example.com
spec:
  group: platform.example.com
  names:
    kind: CompositeDatabase
    plural: compositedatabases
  claimNames:
    kind: Database
    plural: databases
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                parameters:
                  type: object
                  required:
                    - storageGB
                    - engineVersion
                  properties:
                    storageGB:
                      type: integer
                      minimum: 20
                      maximum: 65536
                    engineVersion:
                      type: string
                      enum: ["14.9", "15.4", "16.2"]
                    instanceClass:
                      type: string
                      default: db.t3.medium
                    region:
                      type: string
                      default: us-east-1

XRDs can include CEL validation expressions (x-kubernetes-validations) for constraints the OpenAPI schema cannot express. Versions must have identical schemas in current Crossplane—conversion webhooks between XRD versions are not yet supported, so treat all versions in a single XRD as the same schema under different names.

Compositions

A Composition is the template that maps an XR’s fields to one or more MRs. Multiple Compositions can satisfy the same XRD—a CompositeDatabase XRD could have a Composition creating Aurora MySQL in AWS and another creating Cloud SQL PostgreSQL in GCP. The XR’s compositionRef or a compositionSelector label determines which Composition applies.

XRD (CompositeDatabase schema) ──┐
                                 ├── Composition A (AWS Aurora MySQL)
                                 └── Composition B (GCP Cloud SQL PostgreSQL)

Providers and Authentication

Crossplane’s providers are the bridge to cloud APIs. There are two lineages: hand-crafted official providers and Upjet-generated providers.

Upjet-generated providers are built from Terraform’s provider schemas using Upjet, a framework that generates Crossplane resources from Terraform resource definitions. This means the Crossplane AWS provider has coverage of nearly every AWS resource—it follows the Terraform AWS provider’s resource set rather than requiring Crossplane maintainers to implement each resource manually. The spec.forProvider fields largely mirror the Terraform resource arguments.

The main provider packages:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Install the AWS provider
kubectl apply -f - <<'EOF'
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-ec2
spec:
  package: xpkg.upbound.io/upbound/provider-aws-ec2:v1.14.0
  controllerConfigRef:
    name: aws-config
EOF

The monolithic provider-aws package is deprecated in favor of per-service packages (provider-aws-ec2, provider-aws-rds, provider-aws-s3, etc.). Installing only the services you use significantly reduces CRD count and provider pod memory footprint.

Authentication: No Stored Credentials

The production-grade authentication pattern for all three major clouds avoids storing static credentials in Kubernetes secrets.

AWS (IRSA on EKS): Create an OIDC provider for your EKS cluster, create an IAM role with a trust policy allowing the Crossplane service account to assume it, and annotate the service account with the role ARN.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Create the IAM role trust policy
cat > crossplane-trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "${OIDC_PROVIDER}:sub": "system:serviceaccount:crossplane-system:provider-aws-*",
        "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
      }
    }
  }]
}
EOF

aws iam create-role \
  --role-name CrossplaneProviderRole \
  --assume-role-policy-document file://crossplane-trust-policy.json

aws iam attach-role-policy \
  --role-name CrossplaneProviderRole \
  --policy-arn arn:aws:iam::aws:policy/AdministratorAccess  # scope down in production
1
2
3
4
5
6
7
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: IRSA

Azure (Workload Identity): Enable workload identity on your AKS cluster, create a managed identity, federate the Crossplane controller service account, and assign the managed identity appropriate roles.

GCP (Workload Identity): Use GKE Workload Identity to map the Crossplane service account to a GCP service account with the necessary project-level IAM bindings.

The pattern is consistent: OIDC token exchange means the cluster itself proves its identity to the cloud provider, and the provider assumes an appropriately scoped role. No credentials stored in etcd.


Writing Compositions

Modern Crossplane Compositions use pipeline mode—a sequence of composition functions where each function receives the current composite resource and desired output, transforms them, and passes the result to the next stage. The original spec.resources field is deprecated.

The Patch-and-Transform Function

function-patch-and-transform is the most common function, implementing the patch-and-transform model Crossplane users are familiar with. Install it as a Function package:

1
2
3
4
5
6
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-patch-and-transform
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.8.0

A Composition using pipeline mode looks like this:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: compositedatabases-aws
  labels:
    provider: aws
    engine: postgres
spec:
  compositeTypeRef:
    apiVersion: platform.example.com/v1alpha1
    kind: CompositeDatabase
  mode: Pipeline
  pipeline:
    - step: patch-and-transform
      functionRef:
        name: function-patch-and-transform
      input:
        apiVersion: pt.fn.crossplane.io/v1beta1
        kind: Resources
        patchSets:
          - name: metadata-passthrough
            patches:
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.region
                toFieldPath: spec.forProvider.region
        resources:
          - name: security-group
            base:
              apiVersion: ec2.aws.upbound.io/v1beta1
              kind: SecurityGroup
              spec:
                forProvider:
                  description: Database security group
                  name: crossplane-db-sg
            patches:
              - type: PatchSet
                patchSetName: metadata-passthrough
              - type: FromCompositeFieldPath
                fromFieldPath: metadata.name
                toFieldPath: metadata.name
                transforms:
                  - type: string
                    string:
                      fmt: "%s-sg"
                      type: Format

          - name: db-subnet-group
            base:
              apiVersion: rds.aws.upbound.io/v1beta1
              kind: SubnetGroup
              spec:
                forProvider:
                  description: Crossplane managed subnet group
            patches:
              - type: PatchSet
                patchSetName: metadata-passthrough

          - name: rds-instance
            base:
              apiVersion: rds.aws.upbound.io/v1beta1
              kind: Instance
              spec:
                forProvider:
                  engine: postgres
                  publiclyAccessible: false
                  skipFinalSnapshot: false
                  autoMinorVersionUpgrade: true
                  backupRetentionPeriod: 7
                  storageType: gp3
                  storageEncrypted: true
                  dbSubnetGroupNameSelector:
                    matchControllerRef: true
                  vpcSecurityGroupIdSelector:
                    matchControllerRef: true
                writeConnectionSecretToRef:
                  namespace: crossplane-system
            patches:
              - type: PatchSet
                patchSetName: metadata-passthrough
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.storageGB
                toFieldPath: spec.forProvider.allocatedStorage
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.engineVersion
                toFieldPath: spec.forProvider.engineVersion
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.instanceClass
                toFieldPath: spec.forProvider.instanceClass
              - type: FromCompositeFieldPath
                fromFieldPath: metadata.uid
                toFieldPath: spec.writeConnectionSecretToRef.name
                transforms:
                  - type: string
                    string:
                      fmt: "%s-connection"
                      type: Format
            connectionDetails:
              - type: FromConnectionSecretKey
                fromConnectionSecretKey: username
                name: username
              - type: FromConnectionSecretKey
                fromConnectionSecretKey: password
                name: password
              - type: FromConnectionSecretKey
                fromConnectionSecretKey: endpoint
                name: endpoint
              - type: FromConnectionSecretKey
                fromConnectionSecretKey: port
                name: port
            readinessChecks:
              - type: MatchString
                fieldPath: status.atProvider.dbInstanceStatus
                matchString: available

Several important details here:

matchControllerRef: true on the vpcSecurityGroupIdSelector and dbSubnetGroupNameSelector tells Crossplane to find the security group and subnet group created by the same Composition instance—this is how you link resources within a Composition without hardcoding names.

The writeConnectionSecretToRef on the RDS instance tells the provider to write the connection credentials to a Kubernetes Secret. The Composition then aggregates those credentials via connectionDetails and surfaces them to the Claim’s namespace. The developer who created the Database claim gets a secret with username, password, endpoint, and port—without knowing anything about the underlying RDS resource.

readinessChecks prevent the Composite Resource from reporting ready until the RDS instance has actually reached the available state. This matters for anything consuming the composite that polls for readiness before proceeding.

Transforms

Transforms allow field values to be modified during patching. Common transform types:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# String format
transforms:
  - type: string
    string:
      fmt: "crossplane-%s"
      type: Format

# Map a string to another value
transforms:
  - type: map
    map:
      t3.medium: db.t3.medium
      t3.large: db.t3.large

# Math (multiply storage GB * 1024 for MB)
transforms:
  - type: math
    math:
      multiply: 1024

# Convert type (string to int)
transforms:
  - type: convert
    convert:
      toType: int64

Composition Functions for Complex Logic

Patch-and-transform handles most cases. When you need conditional logic, loops, or transformations that patch-and-transform cannot express, use a programming-language function.

function-go-templating uses Helm-like Go template syntax:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
- step: go-templating
  functionRef:
    name: function-go-templating
  input:
    apiVersion: gotemplating.fn.crossplane.io/v1beta1
    kind: GoTemplate
    source: Inline
    inline:
      template: |
        {{ range $i := until (.observed.composite.resource.spec.parameters.subnetCount | int) }}
        ---
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: Subnet
        metadata:
          name: {{ $.observed.composite.resource.metadata.name }}-subnet-{{ $i }}
        spec:
          forProvider:
            region: {{ $.observed.composite.resource.spec.parameters.region }}
            cidrBlock: 10.0.{{ $i }}.0/24
            availabilityZone: {{ $.observed.composite.resource.spec.parameters.region }}{{ list "a" "b" "c" | index $i }}
        {{ end }}

function-kcl uses KCL (Kube Configuration Language), a Python-like typed language designed for Kubernetes configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
- step: kcl
  functionRef:
    name: function-kcl
  input:
    apiVersion: krm.kcl.dev/v1alpha1
    kind: KCLRun
    spec:
      source: |
        import regex
        
        oxr = option("params").oxr
        region = oxr.spec.parameters.region
        name = oxr.metadata.name
        
        items = [{
          apiVersion = "ec2.aws.upbound.io/v1beta1"
          kind = "VPC"
          metadata.name = name + "-vpc"
          spec.forProvider = {
            region = region
            cidrBlock = "10.0.0.0/16"
            enableDnsHostnames = True
          }
        }]

KCL provides type checking, schema validation, and iteration in a more structured way than Go templating. It is a good choice for complex compositions that would become unreadable in Go templates.


Developer Workflow: Creating Infrastructure

From the application developer’s perspective, Crossplane’s self-service model is straightforward. They never interact with AWS APIs, Terraform, or cloud console—only with Kubernetes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# developer creates this in their team's namespace
apiVersion: platform.example.com/v1alpha1
kind: Database
metadata:
  name: payments-db
  namespace: team-payments
spec:
  parameters:
    storageGB: 100
    engineVersion: "16.2"
    instanceClass: db.t3.large
    region: us-east-1
  writeConnectionSecretToRef:
    name: payments-db-conn
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
kubectl apply -f payments-db.yaml

# Watch the composite resource come up
kubectl get database payments-db -n team-payments -w
NAME          SYNCED   READY   CONNECTION-SECRET   AGE
payments-db   True     False   payments-db-conn    45s
payments-db   True     True    payments-db-conn    4m12s

# The connection secret is now available in the namespace
kubectl get secret payments-db-conn -n team-payments -o jsonpath='{.data.endpoint}' | base64 -d
payments-db-abc123.us-east-1.rds.amazonaws.com

The application’s Deployment references the connection secret directly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
env:
  - name: DB_HOST
    valueFrom:
      secretKeyRef:
        name: payments-db-conn
        key: endpoint
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: payments-db-conn
        key: password

Crossplane vs Terraform

This comparison matters because Terraform is entrenched in almost every organization doing cloud infrastructure, and Crossplane advocates sometimes overclaim its advantages while Terraform advocates sometimes dismiss it unfairly. The honest comparison is situational.

What Is Fundamentally Different

State model: Terraform maintains a state file that records what it believes the current state of infrastructure to be. Crossplane uses the Kubernetes API server as its state store—the observed state is written directly to the MR’s status.atProvider field. There is no separate state file to lock, corrupt, or get out of sync.

Reconciliation: Terraform reconciles on demand—plan then apply when a human or CI job runs. Crossplane reconciles continuously—approximately every 10 minutes by default, triggered immediately by any spec change. Drift is detected and corrected automatically without human intervention.

Execution environment: Terraform runs as a process, typically in CI. Crossplane runs as controllers in your cluster. Terraform’s execution is ephemeral; Crossplane’s is always-on.

Feature Comparison

Feature Terraform Crossplane
State management State files (S3 + DynamoDB locking) Kubernetes API server
Drift detection On terraform plan only Continuous, automatic correction
Resource coverage Comprehensive (30+ years of providers) Large and growing (Upjet-generated)
Abstraction model Modules + variables XRDs + Compositions + Claims
RBAC for infrastructure CI credentials / Vault integration Kubernetes native RBAC
GitOps integration External (CI applies terraform) Native (ArgoCD syncs objects directly)
Developer self-service Requires CI job or custom tooling Native (Claim = kubectl apply)
Debugging Terraform state + plan output kubectl describe + events
Non-cloud resources Excellent (DNS, GitHub, Datadog, etc.) Limited (provider availability varies)
Helm / Kubernetes-native Via kubernetes provider (awkward) Native
Learning curve Moderate (HCL, state concepts) Steep (many CRD types, Composition DSL)
Ecosystem maturity Mature Growing

When Crossplane Wins

Platform teams building self-service infrastructure APIs: Crossplane’s abstraction model—XRD defines the API, Composition implements it, Claim consumes it—is purpose-built for internal developer platforms. Teams expose a high-level “I need a database” API and hide all the cloud implementation details. Terraform can do this, but it requires wrapping Terraform in a custom API layer.

Continuous drift correction: Workloads that have noisy drift—someone clicks in the console, a cloud provider auto-modifies a tag, a security team pushes a policy—benefit from Crossplane’s reconciliation loop. Terraform only fixes drift when the next apply runs.

GitOps organizations: If ArgoCD or Flux already manages your Kubernetes workloads, extending them to manage infrastructure through Crossplane unifies your entire delivery pipeline under the same tool and workflow. The infrastructure YAML lives in the same git repo as the application YAML.

When Terraform Wins

Non-cloud resources: Terraform has providers for hundreds of services: DNS (Cloudflare, Route 53), monitoring (Datadog, PagerDuty), source control (GitHub, GitLab), identity providers (Okta), database management (PostgreSQL, MySQL). Crossplane’s provider ecosystem is more limited outside of major cloud providers.

Simple one-off operations: Terraform’s plan/apply workflow is more intuitive for humans who want to see exactly what will change before applying. Crossplane’s continuous reconciliation makes it harder to reason about “what will happen when I apply this.”

Organizations without Kubernetes: Crossplane requires a Kubernetes cluster to run. If your infrastructure team does not already run Kubernetes, adopting Crossplane means adopting Kubernetes first.

Non-EKS/AKS/GKE environments: IRSA, Workload Identity, and GKE Workload Identity require specific cloud-managed Kubernetes environments. Running Crossplane on self-managed Kubernetes in a datacenter requires a different authentication approach.

The practical answer for most organizations: use Terraform for foundational infrastructure (VPCs, Kubernetes clusters themselves, shared databases, IAM roles) and use Crossplane for application-team self-service of higher-level resources (databases per service, object storage buckets, queues). This hybrid approach uses each tool where it is strongest.


ArgoCD Integration

ArgoCD and Crossplane pair naturally: ArgoCD syncs Crossplane objects to the cluster, and Crossplane reconciles those objects against the cloud. But the bootstrap sequence requires attention—ArgoCD cannot sync Crossplane Managed Resources until the Crossplane providers that define those CRDs are installed and healthy.

Sync Wave Ordering

Use ArgoCD sync waves to ensure the installation order:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Wave 0: Crossplane itself (Helm chart)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crossplane
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  source:
    repoURL: https://charts.crossplane.io/stable
    chart: crossplane
    targetRevision: 1.17.0
  destination:
    namespace: crossplane-system
  syncPolicy:
    automated: {}

---
# Wave 1: Providers (after Crossplane is healthy)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crossplane-providers
  annotations:
    argocd.argoproj.io/sync-wave: "1"

---
# Wave 2: ProviderConfigs (after providers are healthy)
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crossplane-provider-configs
  annotations:
    argocd.argoproj.io/sync-wave: "2"

---
# Wave 3: XRDs and Compositions
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: crossplane-platform-apis
  annotations:
    argocd.argoproj.io/sync-wave: "3"

---
# Wave 4: Claims
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: team-payments-infrastructure
  annotations:
    argocd.argoproj.io/sync-wave: "4"

ArgoCD Health Checks for Crossplane Objects

ArgoCD’s default health checks do not understand Crossplane’s condition model. Add custom health checks to your ArgoCD ConfigMap:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# argocd-cm ConfigMap
resource.customizations.health.apiextensions.crossplane.io_CompositeResourceDefinition: |
  hs = {}
  if obj.status ~= nil and obj.status.conditions ~= nil then
    for i, condition in ipairs(obj.status.conditions) do
      if condition.type == "Established" and condition.status == "True" then
        hs.status = "Healthy"
        hs.message = "CRD established"
        return hs
      end
    end
  end
  hs.status = "Progressing"
  hs.message = "Waiting for CRD to be established"
  return hs

resource.customizations.health.pkg.crossplane.io_Provider: |
  hs = {}
  if obj.status ~= nil and obj.status.conditions ~= nil then
    for i, condition in ipairs(obj.status.conditions) do
      if condition.type == "Healthy" and condition.status == "True" then
        hs.status = "Healthy"
        hs.message = condition.message
        return hs
      end
    end
  end
  hs.status = "Progressing"
  hs.message = "Provider not yet healthy"
  return hs

Without these health checks, ArgoCD may consider a provider healthy before it actually is, causing wave 2 to start before wave 1 is truly complete.


Production Operations

Observing Resource State

Crossplane follows the Kubernetes conditions model. A healthy MR looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
kubectl describe rdsinstance payments-db-rds-instance

Status:
  At Provider:
    Db Instance Status:  available
    Endpoint:
      Address:  payments-db-abc123.us-east-1.rds.amazonaws.com
      Port:     5432
  Conditions:
    Last Transition Time:  2026-05-23T10:00:00Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2026-05-23T10:05:00Z
    Reason:                Available
    Status:                True
    Type:                  Ready
Events:
  Type    Reason                   Age   Message
  ----    ------                   ----  -------
  Normal  CreatedExternalResource  8m    Successfully requested creation of external resource
  Normal  ImportedExternalResource 5m    Successfully imported external resource

When something goes wrong, the Synced condition becomes False with a reason and the events contain the cloud API error:

Conditions:
  Reason:  ReconcileError
  Status:  False
  Type:    Synced
  Message: cannot create RDS instance: InvalidParameterValue: The parameter EngineVersion is not a valid version for mysql

Pausing Reconciliation

When debugging or performing manual interventions, pause a resource’s reconciliation by annotating it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Pause a specific managed resource
kubectl annotate rdsinstance payments-db-rds-instance \
  crossplane.io/paused=true

# Pause an entire composite resource (cascades to MRs)
kubectl annotate compositeDatabase payments-db \
  crossplane.io/paused=true

# Resume
kubectl annotate rdsinstance payments-db-rds-instance \
  crossplane.io/paused-

Pausing a Composite Resource pauses all its MRs. Pausing prevents Crossplane from making any changes—neither creating, updating, nor deleting. Use this when you need to manually intervene in the cloud without Crossplane immediately reverting your changes.

Deletion Protection and Orphaning

By default, deleting a Claim cascades to deleting the Composite Resource, which cascades to deleting all MRs, which deletes the actual cloud resources. This is correct behavior—but it has caused production disasters when developers kubectl delete a namespace and find their database is gone.

Two mitigations:

Deletion policy: Set spec.deletionPolicy: Orphan on MRs or Compositions to prevent Crossplane from deleting the cloud resource when the Kubernetes object is deleted. The cloud resource persists; you manage its lifecycle outside Crossplane.

1
2
3
4
spec:
  deletionPolicy: Orphan  # Kubernetes object deletes don't touch the cloud resource
  forProvider:
    region: us-east-1

Usages: Crossplane’s Usage resource can block deletion of a resource while dependencies exist:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
apiVersion: apiextensions.crossplane.io/v1alpha1
kind: Usage
metadata:
  name: protect-production-db
spec:
  of:
    apiVersion: platform.example.com/v1alpha1
    kind: Database
    resourceRef:
      name: payments-db
  by:
    apiVersion: apps/v1
    kind: Deployment
    resourceRef:
      name: payments-service

This blocks payments-db from being deleted while payments-service exists.

CRD Scale and Provider Memory

Each Upjet-generated provider installs hundreds of CRDs into your cluster. The full provider-aws monolith installs over 900 CRDs. This has real consequences:

  • etcd performance degrades with large CRD counts. Kubernetes recommends keeping CRD counts below a few hundred for best performance.
  • Each provider pod watches its CRDs and maintains controller caches. Memory usage per provider scales with the number of watched resources.
  • ArgoCD and other tools that enumerate CRDs slow down proportionally.

The solution is to use per-service provider packages and install only what you need:

1
2
3
4
5
6
7
# Instead of the monolith:
# provider-aws: v1.14.0  (900+ CRDs)

# Install only what you use:
# provider-aws-ec2: v1.14.0  (~150 CRDs)
# provider-aws-rds: v1.14.0  (~30 CRDs)
# provider-aws-s3: v1.14.0   (~15 CRDs)

A typical platform using EC2, RDS, S3, SQS, and IAM installs around 300–400 CRDs. This is manageable but not trivial—size your etcd accordingly.


Crossplane Spaces

Upbound (the company behind Crossplane) offers Spaces as a commercial product for running multiple isolated Crossplane control planes on a single Kubernetes cluster. In the open-source model, one Crossplane installation is one control plane—all teams share the same API server, CRDs, and providers.

Spaces allows one host cluster to run 50+ isolated control planes, each with its own set of providers, CRDs, and RBAC. This matters at scale: a large organization with 20 teams can give each team their own control plane without running 20 separate Kubernetes clusters for Crossplane.

The open-source version satisfies most teams starting out. Spaces becomes relevant when you hit the operational ceiling of a shared control plane—too many CRDs, too much RBAC complexity, noisy-neighbor issues between teams’ reconciliation loops.


When Crossplane Is Not the Right Tool

Honest accounting of where Crossplane underperforms:

Migration from existing Terraform: There is no automated migration path from Terraform-managed resources to Crossplane. You must import existing resources into Crossplane MRs using crossplane.io/external-name annotations, which is a manual process. For large Terraform estates, the migration cost is substantial.

Resources without Crossplane providers: Anything outside major cloud providers requires building or finding a provider. If you manage GitHub organizations, Datadog monitors, PagerDuty schedules, or custom internal APIs through Terraform today, Crossplane cannot replace those workflows without significant investment.

Debugging Composition failures: When a Composition fails mid-way through, some MRs may be created and others not. Understanding the partial state requires examining multiple objects and their events. Terraform’s plan output, which shows the entire dependency graph and what will happen, is easier to reason about for complex failures.

Teams without Kubernetes operational experience: Crossplane assumes you are already comfortable with Kubernetes objects, controllers, RBAC, and GitOps workflows. If your infrastructure team’s background is pure cloud-native tools and Terraform scripts, the Kubernetes learning curve compounds the Crossplane learning curve.

The organizational pattern that gets the most value from Crossplane: a platform team already running Kubernetes, already using GitOps, wanting to extend those workflows to cloud infrastructure, and building self-service APIs for application teams who should not need to know anything about AWS.


Getting Started

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# Install Crossplane via Helm
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

helm install crossplane \
  crossplane-stable/crossplane \
  --namespace crossplane-system \
  --create-namespace \
  --version 1.17.0 \
  --set args='{--enable-usages}'

# Verify installation
kubectl get pods -n crossplane-system
kubectl get crds | grep crossplane.io | wc -l

# Install the AWS EC2 provider
kubectl apply -f - <<'EOF'
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-ec2
spec:
  package: xpkg.upbound.io/upbound/provider-aws-ec2:v1.14.0
EOF

# Watch provider installation
kubectl get provider provider-aws-ec2 -w

# Install patch-and-transform function
kubectl apply -f - <<'EOF'
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-patch-and-transform
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.8.0
EOF

From here, define your XRD, write a Composition, and create a Claim. The official Crossplane documentation has step-by-step guides for each service provider. Start with a low-stakes resource—an S3 bucket or an SQS queue—before writing a Composition that provisions a full VPC with subnets, route tables, and NAT gateways.

The investment pays off as platform complexity grows. A small team with three services and consistent infrastructure needs is often better served by well-structured Terraform. A platform team supporting twenty application teams, each needing their own isolated database and message queue per environment, is exactly the use case Crossplane was built for.

Comments