Automating E2E Testing for Cumulocity Plugins with Cypress and GitHub Actions

Knowledge Base
, , ,
1

In a previous article, we explored the fundamentals of setting up Cypress for End-to-End (E2E) testing within the Cumulocity ecosystem. We discussed how to configure Cypress to interact with Cumulocity UI components and run basic tests manually.

This article takes the next step: Automation.

giphy

We will look at how to implement a robust Continuous Integration (CI) pipeline using GitHub Actions. The goal is to automatically test your plugins against the latest version of the Cumulocity platform on a daily schedule, ensuring compatibility and stability without requiring manual intervention.

Resources:

The Context: Continuous Deployment and Compatibility

The Cumulocity platform operates on a Continuous Deployment (CD) model. This ensures that tenants receive the latest features, security patches, and hotfixes as soon as they are available. While this is excellent for platform security and feature availability, it presents a challenge for plugin developers.

A custom plugin developed against a specific version of the Web SDK might encounter compatibility issues when the platform updates. The risks include:

  • UI Regressions: Changes in the underlying DOM or CSS classes.
  • API Changes: Deprecations or modifications in internal services.
  • Dependency Conflicts: Version mismatches between the plugin and the shell application.

To mitigate these risks, it is best practice to test your plugin regularly—ideally daily—against the latest available version of the standard applications (Cockpit, Administration, or Device Management).

The Challenge: The “Clean Room” Environment

Testing a plugin usually requires installing it onto a tenant. However, relying on a shared Development or Production tenant for automated testing has significant downsides:

  1. Environment Pollution: Installing and uninstalling plugins daily can clutter the tenant.
  2. Version Mismatches: The tenant might be running an older version of the shell application than what is currently being rolled out globally.
  3. Isolation: It is difficult to get a “clean” state where only the standard shell and your specific plugin exist.

Clean room environment
“Clean Room” Environment

The Solution: A Local Hybrid Server

To solve this, we can emulate the environment locally within a GitHub Action runner. By spinning up a local HTTP server, we can serve the latest version of the standard shell application (with our plugin injected) while proxying API requests to a real Cumulocity tenant.

This approach requires two key open-source tools provided by the Cumulocity ecosystem:

Tool 1: plugins-e2e-setup

Before we can test, we need the standard default applications (Cockpit, Administration and Device Management) of Cumulocity, to use these as shell applications, in which a plugin can be tested in. The plugins-e2e-setup tool supports this via two custom actions:

  1. collect-shell-versions: This action automates the discovery of platform versions. It can be configured to fetch the absolute latest release or specific LTS (Long Term Support) versions. This ensures your tests are always aligned with the version currently being deployed by the platform.
  2. get-shell-app: Once the version is identified, this action downloads the pre-built shell artifacts (zip files) for Cockpit, Administration, or Device Management. This provides an exact replica of the standard application used in production, saving you the overhead of setting up a complex build environment for the shell itself.

Tool 2: c8yctrl

Once the shell is downloaded, we use cumulocity-cypress, specifically the c8yctrl utility.

  • Local Server: It utilizes the Node.js http library to spin up a local web server that serves the downloaded shell and your plugin.
  • Proxy Middleware: Crucially, it uses http-proxy-middleware to forward any API requests (such as /inventory or /alarm) to a configured remote Cumulocity tenant.

This allows the Cypress test runner to interact with a local frontend that feels real, backed by live data, without ever deploying the plugin to the cloud.

Optimizing with Caching

When running automated tests daily, efficiency is key. Downloading dependencies and large shell applications from scratch on every run consumes significant time, network bandwidth and other resources which might also have an impact on costs. The workflow included below implements a strategic caching mechanism to handle this.

1. Caching Node Modules

The workflow uses actions/cache to store your node_modules.

  • The Key: node-modules-${{ github.sha }} or a hash of your package-lock.json.
  • The Benefit: If you are testing multiple plugins in a single repository (a monorepo setup), or if you re-run a job, you don’t need to execute npm ci repeatedly. The dependencies are pulled from the GitHub cache, significantly speeding up the “Install dependencies” phase.

2. Caching Shell Applications

Shell application builds (e.g., the Cockpit zip file) are large artifacts.

  • The Strategy: The workflow checks if the specific version of the shell (e.g., 1023.14.2) has already been downloaded in a previous run.
  • The Mechanism: It uses a cache key based on the version string: shell-apps-${{ steps.extract-version.outputs.version }}.
  • The Benefit: If the Cumulocity platform hasn’t been updated since the last test run, the workflow skips the download entirely and restores the shell apps from the cache. This is especially useful if you run a build matrix to test your plugin against Cockpit, Devicemanagement, and Administration simultaneously—they can all share the same cached artifacts.

Preparation: Configuring GitHub Environment Variables

This automation relies on connecting to a Cumulocity tenant. To do this securely, you must configure Environment Variables and Secrets in your GitHub repository.

  1. Navigate to your repository on GitHub.
  2. Go to Settings > Secrets and variables > Actions.
  3. You will see two tabs: Secrets (for sensitive data like passwords) and Variables (for non-sensitive configuration).

You must define the following:

Repository Secrets (Encrypted):

  • C8Y_PASSWORD: The password for the user running the tests.

Repository Variables (Visible):

  • C8Y_TENANT: The ID of your tenant (e.g., t12345).
  • C8Y_BASEURL: The full URL of your tenant (e.g., https://my-tenant.cumulocity.com).
  • C8Y_USERNAME: The username for the test account.
  • C8Y_SHELL_TARGET: The target application to test (e.g., cockpit).
  • C8Y_CYPRESS_URL: The local URL where c8yctrl will run (usually http://localhost:4200).
  • C8YCTRL_PORT: The port for the local server (e.g., 4200).
  • C8Y_FAVORITES_ASSET_ID: (Optional) Specific to the example plugin, used to target a known asset ID in your tenant for testing.

Note: Properly separating Secrets from Variables ensures you don’t accidentally expose credentials in your build logs.

image
image1174Ă—754 47.2 KB

Implementation: The Daily E2E Workflow

Below is a breakdown of the GitHub Actions workflow. You can view a reference implementation in the Cumulocity Favorites Manager Plugin repository.

1. Workflow Configuration

We define a workflow that runs on a schedule (cron) and maps our GitHub variables to the workflow environment.

name: E2E Daily Test Runner

on:
  workflow_dispatch:
  schedule:
    - cron: "0 0 * * *" # Runs daily at midnight UTC

env:
  C8Y_TENANT: ${{ vars.C8Y_TENANT }}
  C8Y_USERNAME: ${{ vars.C8Y_USERNAME }}
  C8Y_PASSWORD: ${{ secrets.C8Y_PASSWORD }}
  C8Y_BASEURL: ${{ vars.C8Y_BASEURL }}
  C8Y_SHELL_TARGET: ${{ vars.C8Y_SHELL_TARGET }}
  C8Y_CYPRESS_URL: ${{ vars.C8Y_CYPRESS_URL }}
  C8Y_FAVORITES_ASSET_ID: ${{ vars.C8Y_FAVORITES_ASSET_ID }}
  C8YCTRL_PORT: ${{ vars.C8YCTRL_PORT }}

2. Fetching the Shells

The first job uses collect-shell-versions to query the latest available version tag. It then passes this version to get-shell-app, which handles the heavy lifting of downloading and extracting the Cockpit zip file.

    runs-on: ubuntu-latest
    outputs:
      shell-version: ${{ steps.collect-shell-versions.outputs.version }}
    steps:
      - name: Collect Latest Shell Version
        id: collect-shell-versions
        uses: Cumulocity-IoT/plugins-e2e-setup/collect-shell-versions@main
        with:
          include-latest: true
          versions-length: 1

      # (Version extraction logic omitted - see full file below)

      - name: Get Cockpit shell app
        uses: Cumulocity-IoT/plugins-e2e-setup/get-shell-app@main
        with:
          shell-name: cockpit
          shell-version: ${{ steps.extract-version.outputs.version }}
          shell-path: shell-apps

3. Build, Deploy, and Test

In the second job, we combine the downloaded shell with our custom plugin.

  1. Build: Compile the plugin using npm run build.
  2. Combine: Move both the built plugin and the downloaded shell content into a dist/apps directory.
  3. Serve: Run npx c8yctrl. This starts the local server using the environment variables we defined earlier to authenticate with the cloud.
  4. Test: Run npm run e2e:run to execute the Cypress suite.
    needs: [install-dependencies, download-shell-apps]
    runs-on: ubuntu-latest
    steps:
      # ... (Node setup and cache restoration) ...

      - name: Build Plugin
        run: |
          npm run build
          mkdir -p dist/apps
          PACKAGE_NAME=$(node -p "require('./package.json').name")
          cp -r "dist/$PACKAGE_NAME" dist/apps/

      - name: Copy shell apps to dist/apps
        run: |
          cp -r shell-apps/* dist/apps/

      - name: Start server with apps deployed
        run: |
          npx c8yctrl &
          npx wait-on ${{ env.C8Y_CYPRESS_URL }}/apps/${{ env.C8Y_SHELL_TARGET }} --timeout 120000
          echo "Server is up!"

      - name: Run E2E Tests
        run: |
          npm run e2e:run`

The Complete Workflow File

For developers looking to integrate this immediately, here is the complete, standard GitHub Action configuration. Ensure you have set up the Secrets and Variables in your repository settings first.

name: E2E Daily Test Runner

on:
  workflow_dispatch:
  schedule:
    - cron: "0 0 * * *" # Runs daily at midnight UTC

env:
  C8Y_TENANT: ${{ vars.C8Y_TENANT }}
  C8Y_USERNAME: ${{ vars.C8Y_USERNAME }}
  C8Y_PASSWORD: ${{ secrets.C8Y_PASSWORD }}
  C8Y_BASEURL: ${{ vars.C8Y_BASEURL }}
  C8Y_SHELL_TARGET: ${{ vars.C8Y_SHELL_TARGET }}
  C8Y_CYPRESS_URL: ${{ vars.C8Y_CYPRESS_URL }}
  C8Y_FAVORITES_ASSET_ID: ${{ vars.C8Y_FAVORITES_ASSET_ID }}
  C8YCTRL_PORT: ${{ vars.C8YCTRL_PORT }}
  C8YCTRL_ROOT: ${{ vars.C8YCTRL_ROOT }}

permissions:
  contents: read
  pull-requests: write

jobs:
  install-dependencies:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 1

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Save node_modules to cache
        uses: actions/cache/save@v4
        with:
          path: |
            node_modules
            /home/runner/.cache/Cypress
          key: node-modules-${{ github.sha }}

  download-shell-apps:
    runs-on: ubuntu-latest
    outputs:
      shell-version: ${{ steps.collect-shell-versions.outputs.version }}
    steps:
      - name: Collect Latest Shell Version
        id: collect-shell-versions
        uses: Cumulocity-IoT/plugins-e2e-setup/collect-shell-versions@main
        with:
          include-latest: true
          versions-length: 1

      - name: Extract version
        id: extract-version
        run: |
          VERSION=$(echo '${{ steps.collect-shell-versions.outputs.shell_versions }}' | jq -r '.[0].version')
          echo "version=$VERSION" >> $GITHUB_OUTPUT

      - name: Cache shell apps
        id: cache-shell-apps
        uses: actions/cache@v4
        with:
          path: shell-apps
          key: shell-apps-${{ steps.extract-version.outputs.version }}

      - name: Download shell apps
        if: steps.cache-shell-apps.outputs.cache-hit != 'true'
        run: mkdir -p shell-apps

      - name: Get Cockpit shell app
        if: steps.cache-shell-apps.outputs.cache-hit != 'true'
        uses: Cumulocity-IoT/plugins-e2e-setup/get-shell-app@main
        with:
          shell-name: cockpit
          shell-version: ${{ steps.extract-version.outputs.version }}
          shell-path: shell-apps

      - name: Get Administration shell app
        if: steps.cache-shell-apps.outputs.cache-hit != 'true'
        uses: Cumulocity-IoT/plugins-e2e-setup/get-shell-app@main
        with:
          shell-name: administration
          shell-version: ${{ steps.extract-version.outputs.version }}
          shell-path: shell-apps

      - name: Get Devicemanagement shell app
        if: steps.cache-shell-apps.outputs.cache-hit != 'true'
        uses: Cumulocity-IoT/plugins-e2e-setup/get-shell-app@main
        with:
          shell-name: devicemanagement
          shell-version: ${{ steps.extract-version.outputs.version }}
          shell-path: shell-apps

      - name: Save shell apps to cache
        uses: actions/cache/save@v4
        if: steps.cache-shell-apps.outputs.cache-hit != 'true'
        with:
          path: shell-apps
          key: shell-apps-${{ steps.extract-version.outputs.version }}

  test-components:
    needs: [install-dependencies, download-shell-apps]
    runs-on: ubuntu-latest
    environment: e2e
    name: Build and test Favorites Manager Plugins
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 1

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Restore node_modules from cache
        uses: actions/cache/restore@v4
        with:
          path: |
            node_modules
            /home/runner/.cache/Cypress
          key: node-modules-${{ github.sha }}
          fail-on-cache-miss: true

      - name: Restore shell apps from cache
        uses: actions/cache/restore@v4
        with:
          path: shell-apps
          key: shell-apps-${{ needs.download-shell-apps.outputs.shell-version }}
          fail-on-cache-miss: true

      - name: Build Favorites Manager Plugins
        run: |
          npm run build
          mkdir -p dist/apps
          # Extract package name from package.json and copy built plugin to dist/apps
          PACKAGE_NAME=$(node -p "require('./package.json').name")
          cp -r "dist/$PACKAGE_NAME" dist/apps/

      - name: Copy shell apps to dist/apps
        run: |
          cp -r shell-apps/* dist/apps/

      - name: Start server with apps deployed
        run: |
          npx c8yctrl &
          npx wait-on ${{ env.C8Y_CYPRESS_URL }}/apps/${{ env.C8Y_SHELL_TARGET }} --timeout 120000
          sleep 10
          echo "Server is up!"

      - name: Run E2E Tests for Favorites Manager UI Plugin
        env:
          CYPRESS_C8Y_TENANT: ${{ env.C8Y_TENANT }}
          CYPRESS_C8Y_USERNAME: ${{ env.C8Y_USERNAME }}
          CYPRESS_C8Y_PASSWORD: ${{ secrets.C8Y_PASSWORD }}
          CYPRESS_C8Y_CYPRESS_URL: ${{ env.C8Y_CYPRESS_URL }}
          CYPRESS_C8Y_SHELL_TARGET: ${{ env.C8Y_SHELL_TARGET }}
        run: |
          echo "Running tests for Favorites Manager UI plugin"
          npm run e2e:run
        timeout-minutes: 10

      - name: Stop server
        if: always()
        run: |
          pkill -f "c8yctrl" || echo "Server was not running"

      - name: Upload Cypress screenshots (on failure)
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: cypress-screenshots
          path: cypress/screenshots
          if-no-files-found: ignore

      - name: Upload Cypress videos (on failure)
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: cypress-videos
          path: cypress/videos
          if-no-files-found: ignore

A successful run of the e2e tests
A successful run of the e2e tests1440Ă—611 42.7 KB

Conclusion

Automated E2E testing is a critical component of professional plugin development. By leveraging the specialized tools plugins-e2e-setup and c8yctrl alongside GitHub Actions, you can ensure that your extensions remain compatible with the rapid release cycle of the Cumulocity platform. This setup provides early warnings for potential regressions, allowing you to react quickly and maintain high service quality for your end-users.

3 Likes