Every CI/CD pipeline eventually develops the same set of problems. Debugging requires pushing commits and waiting for remote runners. Secrets behave differently locally. The YAML grows to 600 lines with nested conditionals and custom actions nobody understands. A new engineer spends their first week figuring out why the pipeline works in staging but not production.
Dagger attacks this at the root: your pipeline is code — real code, in Go, Python, or TypeScript — that runs inside containers on your laptop. The exact same code runs in GitHub Actions, GitLab CI, CircleCI, Jenkins, or Buildkite. No YAML. No CI vendor lock-in. No “push a commit to test a pipeline change.”
This guide covers Dagger’s architecture, writing your first pipeline in each supported language, caching strategies, secrets handling, composing modules, and integrating with real CI systems.
The Problem Dagger Solves
YAML Is Not a Programming Language
Most CI systems treat pipelines as configuration files. GitHub Actions, GitLab CI, CircleCI, and Jenkins all use YAML (or Groovy, which is arguably worse). YAML has no functions, no types, no tests, no IDE support worth mentioning, and no way to share logic between projects without copy-pasting.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
# Typical GitHub Actions — 200 lines to do what 20 lines of Go would do
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Download dependencies
run: go mod download
- name: Run tests
run: go test -race -coverprofile=coverage.out ./...
- name: Check coverage threshold
run: |
COVERAGE=$(go tool cover -func coverage.out | grep total | awk '{print $3}' | sed 's/%//')
if (( $(echo "$COVERAGE < 80" | bc -l) )); then
echo "Coverage $COVERAGE% is below threshold"
exit 1
fi
- name: Build binary
run: go build -ldflags="-s -w" -o ./build/myapp ./cmd/myapp
- name: Build Docker image
run: docker build -t myapp:${{ github.sha }} .
|
Now imagine debugging this when it fails in CI but not locally. You push a fix, wait 4 minutes for the runner to spin up, discover a typo in the shell one-liner, push again. Repeat.
Dagger’s Answer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
// The same pipeline in Go — runs on your laptop in seconds
func (m *Pipeline) Build(ctx context.Context, src *dagger.Directory) *dagger.Container {
return dag.Container().
From("golang:1.22-alpine").
WithDirectory("/src", src).
WithWorkdir("/src").
WithExec([]string{"go", "build", "-ldflags=-s -w", "-o", "./build/myapp", "./cmd/myapp"})
}
func (m *Pipeline) Test(ctx context.Context, src *dagger.Directory) (string, error) {
return dag.Container().
From("golang:1.22-alpine").
WithDirectory("/src", src).
WithWorkdir("/src").
WithExec([]string{"go", "test", "-race", "-coverprofile=coverage.out", "./..."}).
WithExec([]string{"go", "tool", "cover", "-func=coverage.out"}).
Stdout(ctx)
}
|
Run it locally:
1
2
|
dagger call test --src .
dagger call build --src .
|
Run it in GitHub Actions:
1
2
3
|
- uses: dagger/dagger-action@v1
with:
args: call test --src .
|
Same command. Same containers. Same result.
How Dagger Works
The Engine
Dagger runs a local Docker-compatible container engine (the Dagger Engine) that your SDK talks to via GraphQL. When you call dagger call, the SDK:
- Compiles your pipeline code
- Connects to the local Dagger Engine (starts it automatically if not running)
- Translates your pipeline into a DAG (Directed Acyclic Graph) of container operations
- Executes the DAG with aggressive caching and parallelism
- Returns results (strings, files, directories, container images)
Everything runs in containers — your pipeline code defines what containers to use, what files to mount, what commands to run, and what to do with the output. Dagger handles the scheduling, caching, and execution.
Caching
Dagger’s caching is automatic and deterministic. Operations are cached based on their inputs:
- The container image digest
- The mounted files (by content hash, not timestamp)
- The command being run
- Environment variables
If none of those change, the cached result is returned instantly — no re-execution. This is why dagger call test on unchanged code returns in milliseconds: the test output is cached from the last run.
Cache entries persist across pipeline runs and across CI runs (with a shared cache volume), giving you incremental builds without any manual cache configuration.
The Module System
Dagger pipelines are packaged as modules — reusable, versioned units of pipeline logic. A module is a directory with a dagger.json and your pipeline code. Modules can depend on other modules published to the Dagger registry or hosted on GitHub.
This is the key to sharing pipeline logic across an organization: write a “build Go service” module once, publish it, and every Go service references it. Updates propagate by bumping the module version.
Installation
1
2
3
4
5
6
7
8
9
10
11
12
|
# macOS
brew install dagger/tap/dagger
# Linux
curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh
# Windows (PowerShell)
winget install Dagger.Dagger
# Verify
dagger version
# dagger v0.13.0 (registry.dagger.io/engine) linux/amd64
|
Dagger requires Docker (or another OCI-compatible container runtime) to be running locally.
Writing Your First Module
Initialize a new Dagger module in your project:
1
2
3
4
5
6
|
cd my-project
# Choose your language
dagger init --sdk=go # Go
dagger init --sdk=python # Python
dagger init --sdk=typescript # TypeScript/Node.js
|
This creates:
my-project/
├── dagger.json # Module metadata and dependencies
├── dagger/
│ └── main.go # Your pipeline code (or main.py / index.ts)
└── ... your existing project files
Go SDK: Full Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
|
// dagger/main.go
package main
import (
"context"
"fmt"
)
// Pipeline defines all CI/CD operations for the service
type Pipeline struct{}
// Test runs the full test suite and returns coverage output
func (m *Pipeline) Test(
ctx context.Context,
// Source directory (pass with --src flag)
src *Directory,
// Optional: run only tests matching this pattern
// +optional
run string,
) (string, error) {
args := []string{"go", "test", "-race", "-coverprofile=coverage.out", "./..."}
if run != "" {
args = append(args, "-run", run)
}
return m.goBase(src).
WithExec(args).
WithExec([]string{"go", "tool", "cover", "-func=coverage.out"}).
Stdout(ctx)
}
// Lint runs golangci-lint and returns any violations
func (m *Pipeline) Lint(ctx context.Context, src *Directory) (string, error) {
return dag.Container().
From("golangci/golangci-lint:v1.57-alpine").
WithDirectory("/src", src).
WithWorkdir("/src").
WithExec([]string{"golangci-lint", "run", "--out-format=github-actions", "./..."}).
Stdout(ctx)
}
// Build compiles the binary for the target OS/arch
func (m *Pipeline) Build(
ctx context.Context,
src *Directory,
// +optional
// +default="linux"
os string,
// +optional
// +default="amd64"
arch string,
) *File {
return m.goBase(src).
WithEnvVariable("GOOS", os).
WithEnvVariable("GOARCH", arch).
WithEnvVariable("CGO_ENABLED", "0").
WithExec([]string{
"go", "build",
"-ldflags=-s -w",
"-o", "/out/myapp",
"./cmd/myapp",
}).
File("/out/myapp")
}
// Publish builds a Docker image and pushes it to a registry
func (m *Pipeline) Publish(
ctx context.Context,
src *Directory,
// Registry address, e.g. "ghcr.io/myorg/myapp"
registry string,
// Image tag, e.g. the git SHA
tag string,
// Registry credentials
registryUsername string,
registryPassword *Secret,
) (string, error) {
binary := m.Build(ctx, src, "linux", "amd64")
return dag.Container().
From("gcr.io/distroless/static:nonroot").
WithFile("/myapp", binary).
WithEntrypoint([]string{"/myapp"}).
WithRegistryAuth(registry, registryUsername, registryPassword).
Publish(ctx, fmt.Sprintf("%s:%s", registry, tag))
}
// CI runs the full pipeline: lint, test, build
func (m *Pipeline) CI(ctx context.Context, src *Directory) (string, error) {
// Lint and test run in parallel — Dagger handles the concurrency
lintResult, err := m.Lint(ctx, src)
if err != nil {
return "", fmt.Errorf("lint failed: %w", err)
}
testResult, err := m.Test(ctx, src, "")
if err != nil {
return "", fmt.Errorf("tests failed: %w", err)
}
// Build only if lint and test passed
m.Build(ctx, src, "linux", "amd64")
return fmt.Sprintf("Lint:\n%s\n\nTests:\n%s", lintResult, testResult), nil
}
// goBase returns a container with Go installed and source mounted
// This is shared across Test, Lint (indirectly), and Build
func (m *Pipeline) goBase(src *Directory) *Container {
return dag.Container().
From("golang:1.22-alpine").
WithMountedCache("/root/go/pkg/mod", dag.CacheVolume("go-mod-cache")).
WithMountedCache("/root/.cache/go-build", dag.CacheVolume("go-build-cache")).
WithDirectory("/src", src).
WithWorkdir("/src").
WithExec([]string{"go", "mod", "download"})
}
|
Run each function:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
# Run tests
dagger call test --src .
# Run with a specific test pattern
dagger call test --src . --run TestAuth
# Lint
dagger call lint --src .
# Build for Linux amd64 (save output to ./myapp-linux)
dagger call build --src . --os linux --arch amd64 export --path ./myapp-linux
# Run the full CI suite
dagger call ci --src .
# Publish (prompts for password interactively)
dagger call publish \
--src . \
--registry ghcr.io/myorg/myapp \
--tag $(git rev-parse --short HEAD) \
--registry-username myorg \
--registry-password env:GITHUB_TOKEN
|
Python SDK: Full Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
|
# dagger/main.py
import dagger
from dagger import dag, function, object_type
import anyio
@object_type
class Pipeline:
"""CI/CD pipeline for the Python service."""
@function
async def test(
self,
src: dagger.Directory,
coverage_threshold: int = 80,
) -> str:
"""Run pytest with coverage check."""
result = await (
self._python_base(src)
.with_exec(["pip", "install", "--quiet", "-r", "requirements-dev.txt"])
.with_exec([
"pytest",
"--cov=src",
"--cov-report=term-missing",
f"--cov-fail-under={coverage_threshold}",
"-v",
])
.stdout()
)
return result
@function
async def lint(self, src: dagger.Directory) -> str:
"""Run ruff and mypy."""
container = self._python_base(src).with_exec([
"pip", "install", "--quiet", "ruff", "mypy"
])
ruff = await container.with_exec(["ruff", "check", "."]).stdout()
mypy = await container.with_exec(["mypy", "src/"]).stdout()
return f"ruff:\n{ruff}\nmypy:\n{mypy}"
@function
async def build(self, src: dagger.Directory) -> dagger.File:
"""Build a wheel package."""
return await (
self._python_base(src)
.with_exec(["pip", "install", "--quiet", "build"])
.with_exec(["python", "-m", "build", "--wheel"])
.file("dist/*.whl")
)
@function
async def publish(
self,
src: dagger.Directory,
registry: str,
tag: str,
registry_username: str,
registry_password: dagger.Secret,
) -> str:
"""Build and push a Docker image."""
return await (
dag.container()
.from_("python:3.12-slim")
.with_directory("/app", src)
.with_workdir("/app")
.with_exec(["pip", "install", "--no-cache-dir", "-r", "requirements.txt"])
.with_entrypoint(["python", "-m", "myapp"])
.with_registry_auth(registry, registry_username, registry_password)
.publish(f"{registry}:{tag}")
)
def _python_base(self, src: dagger.Directory) -> dagger.Container:
"""Shared base container with Python and source mounted."""
return (
dag.container()
.from_("python:3.12-slim")
.with_mounted_cache(
"/root/.cache/pip",
dag.cache_volume("pip-cache"),
)
.with_directory("/src", src)
.with_workdir("/src")
)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
# Run tests with coverage threshold
dagger call test --src . --coverage-threshold 85
# Lint
dagger call lint --src .
# Build wheel
dagger call build --src . export --path ./dist/
# Publish
dagger call publish \
--src . \
--registry ghcr.io/myorg/myservice \
--tag $(git rev-parse --short HEAD) \
--registry-username myorg \
--registry-password env:GITHUB_TOKEN
|
TypeScript SDK: Full Example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
|
// dagger/index.ts
import {
dag,
Container,
Directory,
File,
Secret,
object,
func,
argument,
} from "@dagger.io/dagger";
@object()
class Pipeline {
/**
* Run tests with Vitest
*/
@func()
async test(
src: Directory,
@argument({ defaultValue: false }) ci: boolean
): Promise<string> {
const args = ["npx", "vitest", "run", "--coverage"];
if (ci) args.push("--reporter=github-actions");
return this.nodeBase(src)
.withExec(args)
.stdout();
}
/**
* Run ESLint and TypeScript type checking
*/
@func()
async lint(src: Directory): Promise<string> {
const base = this.nodeBase(src);
const eslint = await base
.withExec(["npx", "eslint", "src/", "--format=compact"])
.stdout();
const tsc = await base
.withExec(["npx", "tsc", "--noEmit"])
.stdout();
return `ESLint:\n${eslint}\nTypeScript:\n${tsc}`;
}
/**
* Build the Next.js / Vite app
*/
@func()
async build(src: Directory): Promise<Directory> {
return this.nodeBase(src)
.withExec(["npm", "run", "build"])
.directory(".next"); // or "dist" for Vite
}
/**
* Build and push a Docker image
*/
@func()
async publish(
src: Directory,
registry: string,
tag: string,
registryUsername: string,
registryPassword: Secret
): Promise<string> {
return dag
.container()
.from("node:22-alpine")
.withDirectory("/app", src)
.withWorkdir("/app")
.withExec(["npm", "ci", "--omit=dev"])
.withExec(["npm", "run", "build"])
.withEntrypoint(["node", "dist/index.js"])
.withRegistryAuth(registry, registryUsername, registryPassword)
.publish(`${registry}:${tag}`);
}
/**
* Run the complete CI pipeline
*/
@func()
async ci(src: Directory): Promise<string> {
// Lint and test run in parallel via Promise.all
const [lintResult, testResult] = await Promise.all([
this.lint(src),
this.test(src, true),
]);
await this.build(src);
return `✓ Lint passed\n${lintResult}\n\n✓ Tests passed\n${testResult}`;
}
private nodeBase(src: Directory): Container {
return dag
.container()
.from("node:22-alpine")
.withMountedCache(
"/root/.npm",
dag.cacheVolume("npm-cache")
)
.withDirectory("/src", src)
.withWorkdir("/src")
.withExec(["npm", "ci"]);
}
}
|
Secrets and Environment Variables
Secrets are a first-class concept in Dagger — they are never logged, never stored in the cache, and never appear in pipeline output.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
# Pass a secret from an environment variable
dagger call publish \
--registry-password env:GITHUB_TOKEN \
...
# Pass a secret from a file
dagger call publish \
--registry-password file:/path/to/token.txt \
...
# Pass a secret from stdin
echo "$MY_SECRET" | dagger call publish \
--registry-password stdin \
...
|
In your pipeline code, secrets are typed — the SDK enforces that you never accidentally log them:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
// Go: secret is *dagger.Secret, not a string
func (m *Pipeline) DeployToK8s(
ctx context.Context,
src *Directory,
kubeconfig *Secret, // Never a plain string
) (string, error) {
return dag.Container().
From("bitnami/kubectl:latest").
WithMountedSecret("/root/.kube/config", kubeconfig).
WithDirectory("/manifests", src.Directory("k8s")).
WithExec([]string{"kubectl", "apply", "-f", "/manifests/"}).
Stdout(ctx)
}
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
# Python: same pattern
@function
async def deploy(
self,
src: dagger.Directory,
kubeconfig: dagger.Secret, # Typed as Secret
) -> str:
return await (
dag.container()
.from_("bitnami/kubectl:latest")
.with_mounted_secret("/root/.kube/config", kubeconfig)
.with_directory("/manifests", src.directory("k8s"))
.with_exec(["kubectl", "apply", "-f", "/manifests/"])
.stdout()
)
|
Caching Strategies
Module Dependency Caches
The most impactful optimization is caching package manager downloads:
1
2
3
4
5
6
7
8
9
10
11
12
|
// Go — cache both module downloads and build cache
func (m *Pipeline) goBase(src *Directory) *Container {
return dag.Container().
From("golang:1.22-alpine").
// Module cache: keyed by go.sum content
WithMountedCache("/root/go/pkg/mod", dag.CacheVolume("go-mod")).
// Build cache: incremental compilation
WithMountedCache("/root/.cache/go-build", dag.CacheVolume("go-build")).
WithDirectory("/src", src).
WithWorkdir("/src").
WithExec([]string{"go", "mod", "download"})
}
|
1
2
3
4
5
6
7
8
9
|
# Python — pip cache
def _python_base(self, src):
return (
dag.container()
.from_("python:3.12-slim")
.with_mounted_cache("/root/.cache/pip", dag.cache_volume("pip"))
.with_directory("/src", src)
.with_workdir("/src")
)
|
1
2
3
4
5
6
7
8
9
|
// Node.js — npm cache
private nodeBase(src: Directory): Container {
return dag.container()
.from("node:22-alpine")
.withMountedCache("/root/.npm", dag.cacheVolume("npm"))
.withDirectory("/src", src)
.withWorkdir("/src")
.withExec(["npm", "ci"]);
}
|
Cache Volumes in CI
Cache volumes are automatically persisted between local runs. In CI, you need to configure the Dagger Engine to use a persistent cache volume. Most CI integrations handle this automatically:
1
2
3
4
5
6
|
# GitHub Actions — Dagger action handles cache persistence
- uses: dagger/dagger-action@v1
with:
args: call ci --src .
env:
DAGGER_CLOUD_TOKEN: ${{ secrets.DAGGER_CLOUD_TOKEN }}
|
With Dagger Cloud, cache volumes are automatically synchronized across CI runners, giving you the same warm cache on every run regardless of which runner picks up the job.
Fine-Grained File Mounting
Only mount what the pipeline step needs, so cache invalidation is minimal:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
// Don't mount the entire source directory for dependency download
// Only mount the dependency manifest — cache is only invalidated when it changes
func (m *Pipeline) installDeps(src *Directory) *Container {
return dag.Container().
From("golang:1.22-alpine").
WithMountedCache("/root/go/pkg/mod", dag.CacheVolume("go-mod")).
// Mount ONLY go.mod and go.sum — not the entire source
WithFile("/src/go.mod", src.File("go.mod")).
WithFile("/src/go.sum", src.File("go.sum")).
WithWorkdir("/src").
WithExec([]string{"go", "mod", "download"})
// Source code is added AFTER download — cache hit on go.mod/go.sum change
}
|
Composing Modules from the Registry
Dagger has a growing ecosystem of reusable modules. Instead of writing your own Docker build logic, use the official module:
1
2
3
4
5
|
# Install a module from the Dagger registry
dagger install github.com/dagger/dagger/modules/helm@v0.13.0
# Or reference it inline
dagger call -m github.com/dagger/dagger/modules/golangci --help
|
Use installed modules in your pipeline:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
// dagger.json lists dependencies
// dagger/main.go can call functions from installed modules
func (m *Pipeline) HelmDeploy(
ctx context.Context,
chart *Directory,
values *File,
kubeconfig *Secret,
) (string, error) {
return dag.Helm().
Deploy(chart, kubeconfig,
dagger.HelmDeployOpts{
Values: values,
Wait: true,
}).
Stdout(ctx)
}
|
Writing a Reusable Module for Your Org
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
// A module that every Go service in your org can use
// Publish to: github.com/myorg/dagger-modules/go-service
package main
import (
"context"
"fmt"
)
type GoService struct {
// Minimum test coverage percentage required
// +optional
// +default=80
CoverageThreshold int
}
func (m *GoService) CI(
ctx context.Context,
src *Directory,
// Container registry to push to on main branch
registry string,
tag string,
registryPassword *Secret,
) (string, error) {
// Full pipeline: lint → test → build → publish
if _, err := m.Lint(ctx, src); err != nil {
return "", fmt.Errorf("lint: %w", err)
}
if _, err := m.Test(ctx, src); err != nil {
return "", fmt.Errorf("test: %w", err)
}
return m.Publish(ctx, src, registry, tag, "ci-bot", registryPassword)
}
// ... rest of the module
|
Teams reference it:
1
2
3
4
5
6
7
8
9
|
# In each Go service's dagger.json
dagger install github.com/myorg/dagger-modules/go-service@v1.2.0
# Use it
dagger call -m go-service ci \
--src . \
--registry ghcr.io/myorg/myservice \
--tag $(git rev-parse --short HEAD) \
--registry-password env:GITHUB_TOKEN
|
Update all services to a new pipeline version by bumping the module version — no copy-pasting YAML across 50 repositories.
CI Integration
GitHub Actions
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main]
pull_request:
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run CI pipeline
uses: dagger/dagger-action@v1
with:
version: "0.13.0"
args: call ci --src .
env:
# Optional: Dagger Cloud for shared cache across runners
DAGGER_CLOUD_TOKEN: ${{ secrets.DAGGER_CLOUD_TOKEN }}
publish:
runs-on: ubuntu-latest
needs: ci
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Build and publish image
uses: dagger/dagger-action@v1
with:
args: call publish
--src .
--registry ghcr.io/${{ github.repository }}
--tag ${{ github.sha }}
--registry-username ${{ github.actor }}
--registry-password env:GITHUB_TOKEN
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GitLab CI
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
# .gitlab-ci.yml
variables:
DAGGER_VERSION: "0.13.0"
.dagger:
image: docker:24
services:
- docker:24-dind
before_script:
- apk add --no-cache curl
- curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh
test:
extends: .dagger
script:
- dagger call test --src .
build:
extends: .dagger
script:
- dagger call build --src . export --path ./dist/
artifacts:
paths: [dist/]
only:
- main
|
CircleCI
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
# .circleci/config.yml
version: 2.1
jobs:
ci:
machine:
image: ubuntu-2204:current
steps:
- checkout
- run:
name: Install Dagger
command: |
curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh
- run:
name: Run pipeline
command: dagger call ci --src .
workflows:
main:
jobs:
- ci
|
Jenkins
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
// Jenkinsfile
pipeline {
agent any
stages {
stage('Install Dagger') {
steps {
sh 'curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh'
}
}
stage('Test') {
steps {
sh 'dagger call test --src .'
}
}
stage('Build') {
steps {
sh 'dagger call build --src . export --path ./dist/'
}
}
stage('Publish') {
when { branch 'main' }
steps {
withCredentials([string(credentialsId: 'registry-token', variable: 'REGISTRY_TOKEN')]) {
sh """
dagger call publish \
--src . \
--registry registry.company.com/myapp \
--tag \$(git rev-parse --short HEAD) \
--registry-username jenkins \
--registry-password env:REGISTRY_TOKEN
"""
}
}
}
}
}
|
Interactive Debugging
One of Dagger’s biggest practical advantages: you can get a shell inside a failing pipeline step.
1
2
3
4
5
6
7
8
|
# Drop into a shell in the container at the point of failure
dagger call test --src . terminal
# Inspect what a specific step produces
dagger call build --src . directory --path /out terminal
# Run an arbitrary command in the pipeline container
dagger call --interactive test --src .
|
No more “add echo statements and push a commit to see what’s happening in CI.” Open a terminal in the exact environment your pipeline runs in, on your laptop, right now.
A Real-World Multi-Service Pipeline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
// dagger/main.go — monorepo pipeline for three services
package main
import (
"context"
"golang.org/x/sync/errgroup"
)
type Monorepo struct{}
// CI runs all service pipelines in parallel
func (m *Monorepo) CI(ctx context.Context, src *Directory) error {
g, ctx := errgroup.WithContext(ctx)
services := []string{"api", "worker", "gateway"}
for _, svc := range services {
svc := svc
g.Go(func() error {
svcDir := src.Directory("services/" + svc)
svc := &ServicePipeline{Name: svc}
_, err := svc.Test(ctx, svcDir)
return err
})
}
return g.Wait()
}
// Deploy deploys all services to a Kubernetes cluster
func (m *Monorepo) Deploy(
ctx context.Context,
src *Directory,
kubeconfig *Secret,
tag string,
) error {
services := []string{"api", "worker", "gateway"}
g, ctx := errgroup.WithContext(ctx)
for _, svc := range services {
svc := svc
g.Go(func() error {
return dag.Container().
From("bitnami/kubectl:latest").
WithMountedSecret("/root/.kube/config", kubeconfig).
WithDirectory("/manifests", src.Directory("k8s/"+svc)).
WithExec([]string{
"kubectl", "set", "image",
"deployment/" + svc,
svc + "=ghcr.io/myorg/" + svc + ":" + tag,
}).
Sync(ctx)
})
}
return g.Wait()
}
|
All three service pipelines run in parallel. Dagger handles the goroutine scheduling and container lifecycle — you write straightforward Go concurrency.
Dagger vs Alternatives
| Feature |
Dagger |
GitHub Actions YAML |
Makefile |
Earthly |
| Language |
Go/Python/TypeScript |
YAML |
Shell/Make DSL |
Earthly DSL |
| Runs locally |
✅ Identical |
❌ Different locally |
✅ |
✅ |
| Type-safe |
✅ |
❌ |
❌ |
Partial |
| IDE support |
✅ Full |
❌ Schema hints only |
❌ |
❌ |
| Testable |
✅ Unit test your pipeline |
❌ |
Partial |
❌ |
| Shareable modules |
✅ Registry + versioned |
❌ (custom actions) |
❌ |
❌ |
| Container-native |
✅ Everything runs in containers |
Partial |
❌ |
✅ |
| Caching |
✅ Automatic, content-based |
Manual cache actions |
Manual |
✅ |
| Debugging |
✅ Terminal in any step |
❌ Push and wait |
✅ |
Partial |
| Vendor lock-in |
❌ None |
✅ GitHub-specific |
❌ None |
❌ None |
When Not to Use Dagger
Dagger adds a layer of abstraction that has costs. Avoid it when:
- Simple projects: A three-step pipeline (test, build, push) that never changes is fine as YAML. Dagger’s value compounds with complexity.
- Team unfamiliar with Go/Python/TypeScript: If your ops team only knows YAML and Shell, introducing a new language for pipelines creates a maintenance burden.
- No local testing workflow: If your team never runs CI locally anyway, Dagger’s primary advantage (local == CI) doesn’t help you.
- Heavy GitHub Actions ecosystem dependency: If you rely on dozens of existing Actions (security scanners, deployment tools), wrapping them all in Dagger containers is more work than benefit.
Conclusion
Dagger delivers on a promise that CI has been pretending to keep for years: your pipeline code should be real code, testable locally, shareable across teams, and completely portable across CI vendors.
The shift from “YAML configuration” to “Go/Python/TypeScript functions” changes how pipelines are built and maintained. You get autocomplete, type checking, unit tests for your pipeline logic, and the ability to debug failures in 30 seconds rather than 8 minutes of push-and-wait cycles.
Start by converting your longest, most painful CI job — the one with 200 lines of YAML, cryptic shell one-liners, and a history of mysterious flakes. Write it as a Dagger function, run it locally, and feel the difference. The rest of the migration follows naturally.
1
2
3
|
# Get started right now
dagger init --sdk=go
dagger call --help
|
Comments