GitHub Actions for Test Automation

Learn how to build fast, reliable CI pipelines with GitHub Actions. Master workflow syntax, dependency caching, matrix strategies, secrets management, and Playwright integration for scalable test.

#ci-cd #github-actions #testing #playwright #automation

This post is part 2 of the CI/CD Quality series. Part 1 covered quality gates — what must block a merge vs. what should inform.

Introduction — Workflows Are Code

GitHub Actions is GitHub’s native CI/CD platform. It runs workflows defined as YAML files in .github/workflows/ on every push, pull request, or scheduled trigger.

The advantage of GitHub Actions over external CI services:

  • Integrated — native GitHub UI, no third-party authentication
  • Fast — minimal latency between code push and pipeline start
  • Generous free tier — 2,000 minutes/month for private repos, unlimited for public repos
  • Marketplace — thousands of reusable actions

This post covers the core concepts for building test automation pipelines with GitHub Actions: workflow syntax, caching, parallelisation, secrets, and integrating Playwright.

Workflow Anatomy

A workflow is a YAML file that defines:

  • When it runs (triggers)
  • Where it runs (runner OS)
  • What it runs (jobs and steps)

Basic Workflow Structure

# .github/workflows/ci.yml
name: CI

# When to run
on:
  pull_request:
  push:
    branches: [main]

# What to run
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run tests
        run: npm test

Breakdown:

  • name — workflow name shown in GitHub UI
  • on — trigger events (pull request, push to main)
  • jobs — independent units of work
  • runs-on — runner OS (ubuntu-latest, windows-latest, macos-latest)
  • steps — sequential commands within a job
  • uses — reusable actions from marketplace
  • run — shell commands

:::tip[Workflow Triggers] Use pull_request for validation before merge. Use push: branches: [main] for post-merge checks. Use schedule: cron: '0 6 * * *' for daily smoke tests against production. :::

Caching Dependencies

Installing dependencies on every run is slow. npm ci takes 30–60 seconds. Do this 20 times per day, and you’ve burned 10–20 minutes.

GitHub Actions supports caching. Cache the node_modules directory and restore it on subsequent runs.

Caching with actions/cache

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'  # Automatically cache npm dependencies
      
      - run: npm ci
      - run: npm test

The cache: 'npm' parameter in actions/setup-node automatically caches ~/.npm based on package-lock.json hash.

Result: First run installs dependencies in 45 seconds. Subsequent runs restore cache in 3 seconds.

Caching Playwright Browsers

Playwright downloads browser binaries on first install. These are 300+ MB and take 20–30 seconds to download.

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      
      - run: npm ci
      
      - name: Cache Playwright browsers
        uses: actions/cache@v4
        with:
          path: ~/.cache/ms-playwright
          key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
      
      - run: npx playwright install --with-deps chromium
      - run: npm run test:e2e

Cache key is based on OS and package-lock.json. If Playwright version changes in package.json, cache invalidates and browsers re-download.

:::info[Cache Limits] GitHub Actions cache is limited to 10 GB per repository. Old caches are automatically evicted when the limit is reached. Prioritise caching dependencies over output artifacts. :::

Matrix Strategies — Testing Across Versions

Matrix strategies run the same job multiple times with different configurations. Test across multiple Node versions, operating systems, or browsers.

Testing on Multiple Node Versions

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18, 20, 22]
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: 'npm'
      
      - run: npm ci
      - run: npm test

This spawns three parallel jobs: one for Node 18, one for Node 20, one for Node 22.

Testing on Multiple Browsers

jobs:
  e2e:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        browser: [chromium, firefox, webkit]
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      
      - run: npm ci
      - run: npx playwright install --with-deps ${{ matrix.browser }}
      - run: npm run test:e2e -- --project=${{ matrix.browser }}

This runs E2E tests in parallel across Chromium, Firefox, and WebKit.

Cost: Matrix jobs consume parallel runner minutes. Free tier includes 20 parallel jobs for public repos, 5 for private repos.

:::warning[Matrix Explosion] A matrix with 3 OS × 3 Node versions × 3 browsers = 27 parallel jobs. Ensure your use case justifies the cost. For most projects, test on ubuntu-latest with Node 20 and Chromium. Add matrix coverage for LTS libraries or cross-platform desktop apps. :::

Managing Secrets

Test automation often requires credentials: API keys, database URLs, authentication tokens.

Never hardcode secrets in code or commit them to Git.

GitHub provides repository secrets and environment secrets for secure credential storage.

Adding a Secret

  1. Navigate to Settings → Secrets and variables → Actions
  2. Click New repository secret
  3. Add name (e.g. STRIPE_TEST_KEY) and value
  4. Save

Using Secrets in Workflows

jobs:
  integration-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      
      - run: npm ci
      
      - name: Run API tests
        env:
          STRIPE_TEST_KEY: ${{ secrets.STRIPE_TEST_KEY }}
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
        run: npm run test:api

Secrets are injected as environment variables at runtime. They are masked in logs — GitHub automatically redacts secret values from output.

:::tip[Environment-Specific Secrets] Use environments to separate staging and production secrets. Define staging and production environments in Settings → Environments, then reference them in workflows with environment: staging. :::

Playwright-Specific Actions

Playwright provides an official GitHub Action: microsoft/playwright-github-action.

Using the Official Playwright Action

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      
      - run: npm ci
      
      - name: Install Playwright browsers
        run: npx playwright install --with-deps
      
      - name: Run Playwright tests
        run: npm run test:e2e
      
      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 7

Uploading HTML Reports and Traces

Playwright generates HTML reports and trace files on test failure. Upload them as artifacts for debugging.

      - name: Run Playwright tests
        run: npm run test:e2e
      
      - name: Upload Playwright report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
      
      - name: Upload traces
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-traces
          path: test-results/
  • if: always() — upload report even if tests fail
  • if: failure() — upload traces only on failure (saves storage)

Accessing artifacts: Navigate to the workflow run in GitHub UI → Artifacts section → download zip.

:::info[Trace Viewer] Download trace files and open them locally with npx playwright show-trace trace.zip. Trace viewer shows full timeline, network activity, console logs, DOM snapshots, and screenshots for every action. :::

Practical Example — Full CI Workflow

Here’s a production-ready workflow combining build, lint, unit tests, E2E smoke tests, and artifact uploads.

name: CI

on:
  pull_request:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      
      - run: npm ci
      - run: npm run build
      
      - name: Upload build artifacts
        uses: actions/upload-artifact@v4
        with:
          name: build
          path: dist/

  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      - run: npm ci
      - run: npm run lint

  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      - run: npm ci
      - run: npm test

  smoke-tests:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      
      - run: npm ci
      
      - name: Download build artifacts
        uses: actions/download-artifact@v4
        with:
          name: build
          path: dist/
      
      - name: Install Playwright
        run: npx playwright install --with-deps chromium
      
      - name: Run smoke tests
        run: npm run test:e2e:smoke
      
      - name: Upload test report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: smoke-test-report
          path: playwright-report/

Key features:

  • needs: build — smoke tests wait for build job to complete
  • Artifacts passed between jobs using upload-artifact and download-artifact
  • Playwright installs only Chromium for faster smoke tests
  • Report uploaded on success or failure (if: always())

Conclusion — Automation as a First-Class Citizen

GitHub Actions makes test automation a native part of the development workflow. Workflows are versioned with your code, reviewed in pull requests, and executed on GitHub’s infrastructure.

The principles:

  • Cache dependencies to minimise install time
  • Use matrix strategies judiciously — test what matters, not every combination
  • Store secrets securely and never commit them
  • Upload artifacts for debugging failed runs
  • Integrate Playwright with official actions and trace uploads

A well-configured GitHub Actions pipeline runs in under 5 minutes for smoke tests, provides instant feedback on pull requests, and costs nothing for public repositories.

Action for this week: Add dependency caching to your CI workflow. Measure the time saved on a typical pull request run. If you don’t have CI yet, start with the basic workflow example from this post and add one test command.

Next in this series: Part 3 — E2E in Docker and Ephemeral Environments