Lab: Implementing a Secure Build Pipeline with Tekton and Tekton Chains

Overview

Tekton is a powerful, Kubernetes-native open-source framework for creating continuous integration and continuous delivery (CI/CD) systems. It runs as a set of Custom Resource Definitions (CRDs) on any Kubernetes cluster, letting you define pipelines as declarative YAML that are portable across environments.

Tekton Chains is a companion project that adds automatic supply chain security to your Tekton pipelines. Once installed, Chains watches for completed TaskRuns, automatically signs their results using Cosign or other signers, and generates SLSA provenance attestations — all without requiring any changes to your existing pipeline definitions.

In this hands-on lab, you will:

• Deploy Tekton Pipelines and Tekton Chains on a local Kubernetes cluster
• Configure Chains to automatically sign artifacts and generate in-toto provenance
• Build a container image through a Tekton Pipeline
• Verify the automatically generated signature and SLSA provenance
• Add a vulnerability scanning step to the pipeline
• Explore keyless signing with Sigstore Fulcio
• Enforce signed-image policies at deployment time

By the end of this lab, you will have a fully functional secure build pipeline that produces signed, attested container images with verifiable provenance — achieving SLSA Level 2 compliance automatically.

Prerequisites

Before starting this lab, ensure you have the following tools installed on your workstation:

• Kubernetes cluster — We will use kind (Kubernetes in Docker) for a local cluster. Alternatively, minikube works as well.
• kubectl — The Kubernetes CLI, version 1.26 or later.
• Helm — The Kubernetes package manager, version 3.x.
• tkn — The Tekton CLI, used to interact with Tekton resources.
• Cosign — Part of the Sigstore project, used for signing and verifying container images.
• jq — A command-line JSON processor for inspecting provenance payloads.
• A container registry — A registry you can push to, such as GitHub Container Registry (GHCR) or Docker Hub. You will need write access and valid credentials.

This lab assumes familiarity with Kubernetes basics (pods, namespaces, configmaps) and general CI/CD concepts.

Environment Setup

Step 1: Create a kind Cluster

Start by creating a fresh Kubernetes cluster using kind:

kind create cluster --name tekton-lab
kubectl cluster-info --context kind-tekton-lab

Confirm the cluster is running:

kubectl get nodes
# NAME                       STATUS   ROLES           AGE   VERSION
# tekton-lab-control-plane   Ready    control-plane   30s   v1.31.0

Step 2: Install Tekton Pipelines

Install the latest release of Tekton Pipelines:

kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

Wait for the Tekton Pipelines pods to become ready:

kubectl get pods -n tekton-pipelines --watch

You should see the tekton-pipelines-controller and tekton-pipelines-webhook pods running:

NAME                                           READY   STATUS    RESTARTS   AGE
tekton-pipelines-controller-7f6b9b5b95-xk2rj   1/1     Running   0          45s
tekton-pipelines-webhook-6c4f8b7d4f-m9nlp      1/1     Running   0          45s

Step 3: Install Tekton Chains

Install Tekton Chains into its own namespace:

kubectl apply --filename https://storage.googleapis.com/tekton-releases/chains/latest/release.yaml

Verify Chains is running:

kubectl get pods -n tekton-chains
# NAME                                        READY   STATUS    RESTARTS   AGE
# tekton-chains-controller-5f4b7c8d6f-r7t2x   1/1     Running   0          30s

At this point, both Tekton Pipelines and Tekton Chains are running on your cluster.

Exercise 1: Configure Tekton Chains for Cosign Signing

Tekton Chains needs a signing key and configuration to know how and where to store signatures and attestations. In this exercise, you will generate a Cosign key pair and configure Chains to use OCI storage with the in-toto attestation format.

Generate a Cosign Key Pair

Cosign can generate a key pair and store it directly as a Kubernetes Secret in the tekton-chains namespace:

cosign generate-key-pair k8s://tekton-chains/signing-secrets

You will be prompted to enter a password for the private key. For this lab, you can press Enter to leave it empty. Cosign creates a Secret named signing-secrets containing the private key, public key, and password.

Verify the secret was created:

kubectl get secret signing-secrets -n tekton-chains
# NAME              TYPE     DATA   AGE
# signing-secrets   Opaque   3      10s

Configure Chains Storage and Format

Next, configure Chains to store signatures in the OCI registry alongside the image and to generate provenance in the in-toto format:

kubectl patch configmap chains-config -n tekton-chains \
  -p='{"data":{"artifacts.oci.storage":"oci","artifacts.taskrun.format":"in-toto","artifacts.taskrun.storage":"oci"}}'

This configuration tells Chains to:

• artifacts.oci.storage: oci — Store OCI artifact signatures in the OCI registry
• artifacts.taskrun.format: in-toto — Generate attestations in the in-toto format, which is the standard for SLSA provenance
• artifacts.taskrun.storage: oci — Store TaskRun attestations in the OCI registry

Restart the Chains Controller

After changing the configuration, restart the Chains controller so it picks up the new settings:

kubectl rollout restart deployment tekton-chains-controller -n tekton-chains
kubectl rollout status deployment tekton-chains-controller -n tekton-chains

How Chains Works

With Chains configured, here is what happens automatically whenever a TaskRun completes:

1. The Chains controller detects the completed TaskRun.
2. It inspects the TaskRun results for OCI image references (specifically results named IMAGE_URL and IMAGE_DIGEST).
3. It signs the image using the Cosign key stored in the signing-secrets Secret.
4. It generates an in-toto provenance attestation capturing the build details.
5. It pushes the signature and attestation to the OCI registry.
6. It annotates the TaskRun with chains.tekton.dev/signed=true.

None of this requires any modification to your Tasks or Pipelines.

Exercise 2: Create a Build Pipeline

Now you will create a Tekton Pipeline that clones a Git repository and builds a container image using Kaniko. First, set up registry credentials so Tekton can push images.

Configure Registry Credentials

Create a Kubernetes Secret with your registry credentials. Replace the placeholder values with your actual registry details:

export REGISTRY_SERVER=ghcr.io
export REGISTRY_USER=your-username
export REGISTRY_PASSWORD=your-token

kubectl create secret docker-registry registry-credentials \
  --docker-server=$REGISTRY_SERVER \
  --docker-username=$REGISTRY_USER \
  --docker-password=$REGISTRY_PASSWORD

kubectl patch serviceaccount default -p '{"secrets": [{"name": "registry-credentials"}]}'

Create the Build Task

Create a file named build-task.yaml. This Task accepts a Git repository URL, a target image name, and uses Kaniko to build and push the image:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: git-clone-and-build
spec:
  params:
    - name: repo-url
      type: string
      description: The Git repository URL to clone
    - name: image
      type: string
      description: The image reference to build and push (e.g., ghcr.io/user/app:tag)
  results:
    - name: IMAGE_URL
      description: The image URL that was built
    - name: IMAGE_DIGEST
      description: The digest of the built image
  workspaces:
    - name: source
  steps:
    - name: clone
      image: alpine/git:2.43.0
      script: |
        #!/usr/bin/env sh
        set -eu
        git clone $(params.repo-url) $(workspaces.source.path)/src
        echo "Repository cloned successfully"
    - name: build-and-push
      image: gcr.io/kaniko-project/executor:latest
      args:
        - --dockerfile=$(workspaces.source.path)/src/Dockerfile
        - --context=$(workspaces.source.path)/src
        - --destination=$(params.image)
        - --digest-file=$(results.IMAGE_DIGEST.path)
      securityContext:
        runAsUser: 0
    - name: write-url
      image: alpine:3.19
      script: |
        #!/usr/bin/env sh
        set -eu
        echo -n "$(params.image)" > "$(results.IMAGE_URL.path)"
        echo "Image URL written: $(params.image)"

Apply the Task:

kubectl apply -f build-task.yaml

Create the Vulnerability Scan Task (For Later Use)

Create vuln-scan-task.yaml — you will add this to the pipeline in a later exercise:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: vulnerability-scan
spec:
  params:
    - name: image
      type: string
      description: The image reference to scan
  steps:
    - name: scan
      image: anchore/grype:latest
      args:
        - $(params.image)
        - --fail-on
        - critical
        - --output
        - table

Create the Pipeline

Create build-pipeline.yaml that chains the clone and build steps:

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: secure-build
spec:
  params:
    - name: repo-url
      type: string
    - name: image
      type: string
  workspaces:
    - name: shared-workspace
  tasks:
    - name: build
      taskRef:
        name: git-clone-and-build
      params:
        - name: repo-url
          value: $(params.repo-url)
        - name: image
          value: $(params.image)
      workspaces:
        - name: source
          workspace: shared-workspace

Apply the Pipeline:

kubectl apply -f build-pipeline.yaml

Run the Pipeline

Create a PipelineRun to execute the pipeline. Replace the image reference with your registry:

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  generateName: secure-build-run-
spec:
  pipelineRef:
    name: secure-build
  params:
    - name: repo-url
      value: "https://github.com/GoogleContainerTools/kaniko.git"
    - name: image
      value: "ghcr.io/your-username/tekton-lab:v1"
  workspaces:
    - name: shared-workspace
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi

Save this as pipelinerun.yaml and create it:

kubectl create -f pipelinerun.yaml

Monitor the pipeline execution:

tkn pipelinerun logs -f --last
# [build : clone] Cloning into '/workspace/source/src'...
# [build : clone] Repository cloned successfully
# [build : build-and-push] INFO[0001] Resolved base image golang:1.22
# [build : build-and-push] ...
# [build : build-and-push] INFO[0045] Pushing image to ghcr.io/your-username/tekton-lab:v1
# [build : write-url] Image URL written: ghcr.io/your-username/tekton-lab:v1

The build should complete successfully. You can also check the PipelineRun status:

tkn pipelinerun list
# NAME                     STARTED        DURATION   STATUS
# secure-build-run-x7k2p   1 minute ago   1m 15s     Succeeded

Exercise 3: Verify Automatic Signing

Once the PipelineRun completes, Tekton Chains automatically detects the completed TaskRun, signs the built image, and annotates the TaskRun. This all happens in the background — no pipeline changes needed.

Wait for Chains to Sign

Chains processes completed TaskRuns asynchronously. Wait a few moments, then check the TaskRun annotations:

# Get the TaskRun name from the PipelineRun
TASKRUN=$(kubectl get taskrun -l tekton.dev/pipeline=secure-build -o name --sort-by=.metadata.creationTimestamp | tail -1)
echo $TASKRUN

# Check if Chains has signed it
kubectl get $TASKRUN -o jsonpath='{.metadata.annotations.chains\.tekton\.dev/signed}'

The output should be:

true

If it still shows empty, wait a few seconds and try again — Chains needs time to process the signing.

Verify the Signature with Cosign

Now verify the image signature using the public key from the Cosign key pair you generated earlier:

cosign verify \
  --key k8s://tekton-chains/signing-secrets \
  ghcr.io/your-username/tekton-lab:v1

You should see output confirming the verification succeeded:

Verification for ghcr.io/your-username/tekton-lab:v1 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

[{"critical":{"identity":{"docker-reference":"ghcr.io/your-username/tekton-lab"},"image":{"docker-manifest-digest":"sha256:abc123..."},"type":"cosign container image signature"},"optional":{}}]

Inspect TaskRun Annotations

Chains annotates the TaskRun with rich metadata about the signing operation:

kubectl get $TASKRUN -o jsonpath='{.metadata.annotations}' | jq .

Key annotations include:

{
  "chains.tekton.dev/signed": "true",
  "chains.tekton.dev/transparency": "https://rekor.sigstore.dev/api/v1/log/entries?logIndex=...",
  "chains.tekton.dev/signature-taskrun-...": "..."
}

The chains.tekton.dev/signed=true annotation confirms Chains successfully processed and signed this TaskRun. If a transparency log is configured, you will also see a Rekor log entry reference.

Exercise 4: Inspect SLSA Provenance

Beyond simple signatures, Tekton Chains generates full SLSA provenance attestations. These attestations describe how the artifact was built — which source was used, what build steps ran, and what tools were involved.

Fetch the Provenance Attestation

Use Cosign to verify and retrieve the in-toto attestation:

cosign verify-attestation \
  --key k8s://tekton-chains/signing-secrets \
  --type slsaprovenance \
  ghcr.io/your-username/tekton-lab:v1 | jq -r '.payload' | base64 -d | jq .

Understanding the Provenance Structure

The provenance attestation follows the in-toto Statement format with a SLSA Provenance predicate. Here is a breakdown of the key fields:

{
  "_type": "https://in-toto.io/Statement/v0.1",
  "predicateType": "https://slsa.dev/provenance/v0.2",
  "subject": [
    {
      "name": "ghcr.io/your-username/tekton-lab",
      "digest": {
        "sha256": "abc123def456..."
      }
    }
  ],
  "predicate": {
    "builder": {
      "id": "https://tekton.dev/chains/v2"
    },
    "buildType": "tekton.dev/v1beta1/TaskRun",
    "invocation": {
      "configSource": {},
      "parameters": {
        "repo-url": "https://github.com/GoogleContainerTools/kaniko.git",
        "image": "ghcr.io/your-username/tekton-lab:v1"
      }
    },
    "buildConfig": {
      "steps": [
        {
          "entryPoint": "...",
          "arguments": null,
          "environment": {
            "container": "clone",
            "image": "alpine/git:2.43.0@sha256:..."
          }
        },
        {
          "entryPoint": "...",
          "environment": {
            "container": "build-and-push",
            "image": "gcr.io/kaniko-project/executor:latest@sha256:..."
          }
        }
      ]
    },
    "materials": [
      {
        "uri": "oci://alpine/git:2.43.0",
        "digest": { "sha256": "..." }
      },
      {
        "uri": "oci://gcr.io/kaniko-project/executor:latest",
        "digest": { "sha256": "..." }
      }
    ]
  }
}

Let us walk through each field:

• subject — The artifact that was produced, identified by its registry URL and SHA-256 digest. This is what the provenance is about.
• builder.id — Identifies the build system. Tekton Chains sets this to https://tekton.dev/chains/v2.
• buildConfig.steps — Records every step that ran in the TaskRun, including the exact container images used (pinned by digest).
• materials — Lists the input artifacts consumed during the build, such as base images. Each material includes a digest for reproducibility.
• invocation.parameters — Captures the parameters passed to the TaskRun, showing exactly what inputs drove the build.

This provenance data satisfies SLSA Level 2 requirements: the build process is defined in a build service (Tekton), and provenance is generated automatically by Tekton Chains (not by the build script itself). The provenance is signed, providing tamper evidence.

Exercise 5: Add a Vulnerability Scan Task

A secure pipeline should not only sign artifacts but also verify they are free of known vulnerabilities before deployment. In this exercise, you will add a Grype vulnerability scan step to the pipeline.

Apply the Scan Task

Apply the vulnerability scan Task you created earlier:

kubectl apply -f vuln-scan-task.yaml

Update the Pipeline

Update build-pipeline.yaml to include the vulnerability scan after the build step:

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: secure-build
spec:
  params:
    - name: repo-url
      type: string
    - name: image
      type: string
  workspaces:
    - name: shared-workspace
  tasks:
    - name: build
      taskRef:
        name: git-clone-and-build
      params:
        - name: repo-url
          value: $(params.repo-url)
        - name: image
          value: $(params.image)
      workspaces:
        - name: source
          workspace: shared-workspace
    - name: vulnerability-scan
      runAfter:
        - build
      taskRef:
        name: vulnerability-scan
      params:
        - name: image
          value: $(params.image)

Apply the updated pipeline:

kubectl apply -f build-pipeline.yaml

Test with a Vulnerable Image

To demonstrate the scan catching vulnerabilities, create a Dockerfile that uses a known-vulnerable base image and push a repo or modify the parameters accordingly. If the image contains critical vulnerabilities, Grype will fail the step:

tkn pipelinerun logs -f --last
# [vulnerability-scan : scan] NAME             INSTALLED  FIXED-IN  TYPE  VULNERABILITY   SEVERITY
# [vulnerability-scan : scan] libcrypto3       3.0.12     3.0.13    apk   CVE-2024-0727   Critical
# [vulnerability-scan : scan] 1 critical vulnerability found
# [vulnerability-scan : scan] ERROR: failed to pass severity threshold
#
# TaskRun failed: step "scan" exited with code 1

The pipeline correctly fails at the scan step, preventing a vulnerable image from being promoted.

Test with a Patched Image

Now run the pipeline against a repository with an up-to-date base image. When no critical vulnerabilities are found, the scan passes:

tkn pipelinerun logs -f --last
# [vulnerability-scan : scan] No critical vulnerabilities found
# PipelineRun completed successfully

The pipeline flow is now: git-clone → build-push → vulnerability-scan. Only images that pass the vulnerability scan are signed by Tekton Chains, because Chains only processes successful TaskRuns.

Exercise 6: Keyless Signing with Fulcio (Advanced)

Managing long-lived signing keys introduces operational complexity and security risk. Sigstore’s Fulcio provides keyless signing by issuing short-lived certificates tied to an OIDC identity. In this exercise, you will configure Tekton Chains to use keyless signing.

Update Chains Configuration

Patch the Chains config to enable keyless signing:

kubectl patch configmap chains-config -n tekton-chains -p='{"data":{
  "signers.x509.fulcio.enabled": "true",
  "signers.x509.fulcio.address": "https://fulcio.sigstore.dev",
  "transparency.enabled": "true",
  "transparency.url": "https://rekor.sigstore.dev"
}}'

You also need to remove or rename the existing signing-secrets secret so Chains falls back to keyless mode:

kubectl delete secret signing-secrets -n tekton-chains

Restart the Chains controller:

kubectl rollout restart deployment tekton-chains-controller -n tekton-chains

Configure OIDC for Chains

Chains needs an OIDC token to authenticate to Fulcio. On a managed Kubernetes service (GKE, EKS, AKS), you can use workload identity. For a local kind cluster, you can configure Spiffe/SPIRE or use an ambient OIDC provider. The Tekton Chains documentation provides setup instructions for each environment.

For a production GKE setup, the service account is automatically federated:

# Example: GKE workload identity binding
gcloud iam service-accounts add-iam-policy-binding \
  tekton-chains-sa@your-project.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:your-project.svc.id.goog[tekton-chains/tekton-chains-controller]"

Run the Pipeline with Keyless Signing

Trigger a new PipelineRun:

kubectl create -f pipelinerun.yaml

After completion, verify with keyless verification by specifying the expected identity and OIDC issuer:

cosign verify \
  --certificate-identity "https://kubernetes.io/namespaces/tekton-chains/serviceaccounts/tekton-chains-controller" \
  --certificate-oidc-issuer "https://your-oidc-issuer" \
  ghcr.io/your-username/tekton-lab:v2

The verification now relies on the certificate chain from Fulcio rather than a static key pair. This approach eliminates key management entirely: each signing operation gets a fresh, short-lived certificate, and the signing event is recorded in the Rekor transparency log for auditability.

Exercise 7: Enforce Signed Images at Deployment

Signing images is only useful if you enforce signature verification at deployment time. In this exercise, you will deploy the Sigstore policy-controller to reject any container image that lacks a valid Tekton Chains signature.

Install the Sigstore Policy Controller

helm repo add sigstore https://sigstore.github.io/helm-charts
helm repo update

helm install policy-controller sigstore/policy-controller \
  --namespace cosign-system \
  --create-namespace \
  --set webhook.configMapName=policy-controller-config

Wait for the policy controller to be ready:

kubectl get pods -n cosign-system --watch

Create an Image Policy

Create a ClusterImagePolicy that requires images to be signed by your Tekton Chains key. Save this as image-policy.yaml:

apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: tekton-chains-signed
spec:
  images:
    - glob: "ghcr.io/your-username/**"
  authorities:
    - key:
        data: |
          -----BEGIN PUBLIC KEY-----
          YOUR_COSIGN_PUBLIC_KEY_HERE
          -----END PUBLIC KEY-----
      attestations:
        - name: must-have-slsa-provenance
          predicateType: "https://slsa.dev/provenance/v0.2"
          policy:
            type: cue
            data: |
              predicateType: "https://slsa.dev/provenance/v0.2"

Replace the public key with the Cosign public key you generated earlier:

# Extract the public key
kubectl get secret signing-secrets -n tekton-chains -o jsonpath='{.data.cosign\.pub}' | base64 -d

Apply the policy:

kubectl apply -f image-policy.yaml

Enforce the Policy on a Namespace

Label a namespace to enable policy enforcement:

kubectl create namespace secure-apps
kubectl label namespace secure-apps policy.sigstore.dev/include=true

Test: Deploy a Signed Image

Deploy the image that was signed by Tekton Chains:

kubectl run signed-app \
  --image=ghcr.io/your-username/tekton-lab:v1 \
  --namespace=secure-apps
# pod/signed-app created

The deployment succeeds because the image has a valid signature and provenance attestation.

Test: Deploy an Unsigned Image

Now try to deploy an image that was not signed:

kubectl run unsigned-app \
  --image=ghcr.io/your-username/unsigned-image:latest \
  --namespace=secure-apps
# Error from server (BadRequest): admission webhook "policy.sigstore.dev" denied the request:
# validation failed: failed policy: tekton-chains-signed:
# spec.containers[0].image ghcr.io/your-username/unsigned-image:latest
# signature key validation failed for authority

The admission webhook correctly rejects the unsigned image. This closes the loop: images are automatically signed during build, and only signed images can be deployed.

Cleanup

When you are finished with the lab, clean up the resources:

# Delete Tekton Chains
kubectl delete -f https://storage.googleapis.com/tekton-releases/chains/latest/release.yaml

# Delete Tekton Pipelines
kubectl delete -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

# Delete the policy controller
helm uninstall policy-controller -n cosign-system
kubectl delete namespace cosign-system

# Delete the kind cluster
kind delete cluster --name tekton-lab

Key Takeaways

• Tekton Chains provides zero-config supply chain security. Once installed and configured, it automatically signs every TaskRun result and generates SLSA provenance — no pipeline modifications required.
• SLSA provenance connects artifacts to their build process. The in-toto attestation records exactly what source, steps, and tools produced an artifact, creating an auditable chain of custody.
• Cosign verification is straightforward. A single command validates that an image was signed by your Tekton Chains instance and has not been tampered with since.
• Keyless signing eliminates key management. By integrating with Fulcio and Rekor, you can sign artifacts with short-lived certificates tied to workload identity, removing the burden of rotating and securing long-lived keys.
• Vulnerability scanning as a pipeline gate prevents insecure deployments. Adding Grype or a similar scanner as a pipeline step ensures that only images free of critical vulnerabilities proceed to signing and deployment.
• Admission control enforces the policy. Using the Sigstore policy-controller as a Kubernetes admission webhook ensures that only properly signed and attested images can run in your cluster, closing the security loop from build to deploy.

Next Steps

Continue strengthening your supply chain security knowledge with these related guides:

• Artifact Provenance and Attestations: From SLSA to in-toto — Deep dive into the SLSA framework, provenance levels, and the in-toto attestation specification.
• Signing and Verifying Container Images with Sigstore and Cosign — Comprehensive guide to Cosign, Fulcio, and Rekor for container image signing and verification.