Embedded DevOps: CI/CD for Firmware & Embedded Linux

Content

The course consists of the following topics and is tailored to the team’s toolchain, build system and target hardware for in-house deliveries. We work with GitLab CI/CD, CMake and containers on a real C/C++ project with an embedded Linux target; the concepts transfer to other CI systems and bare-metal targets.

– Why CI/CD is different in embedded projects:

• Delivery target device instead of cloud: what transfers from DevOps and what doesn’t
• Typical blockers: proprietary compilers, licenses, target hardware, long builds
• The pipeline as quality and evidence tool, not just a build automaton – Toolchain into the container:
• Packaging cross-compile toolchains (GCC/Clang, vendor SDKs, Yocto/Buildroot SDK) reproducibly into Docker images
• Handling proprietary and license-bound compilers in the pipeline
• CMake toolchain files, presets and build variants per target
• Reproducible builds: pinning versions, versioning the build environment
• Hands-on: build the toolchain image and run the project’s first pipeline build – GitLab CI/CD for embedded teams:
• Jobs, stages, rules, caching and artifacts in practice
• Pipeline structure for multiple targets and build variants
• Getting build times under control: caching, parallelization, incremental builds
• Hands-on: set up a multi-stage pipeline with a build matrix – Testable firmware architecture:
• Hardware abstraction and layering so logic becomes testable without the target
• Bringing legacy code under test step by step
• Unit tests on the host with GoogleTest/Catch2, mocks for peripherals and drivers
• Hands-on: extract hardware-independent logic and test it in the pipeline – Static analysis and quality gates:
• clang-tidy, cppcheck, warnings as errors, coding standards (MISRA/CERT as orientation)
• Capturing code coverage and surfacing it in merge requests
• Defining quality gates that block the merge
• Hands-on: analysis and coverage jobs with merge request reports – Hardware-in-the-loop (HiL) and target tests:
• Self-hosted runners at the test bench: attaching target hardware to the pipeline safely
• Flash, boot, test and collect results – automated
• Emulation (QEMU) as an intermediate step between host test and real hardware
• Dealing with scarce hardware: scheduling, reservation, stability of HiL jobs
• Hands-on: target test job on a runner with attached hardware or emulation – Embedded Linux in the pipeline:
• Automating Yocto/Buildroot builds, caching for long image builds
• Image tests and system tests in containers or QEMU
• Separation: application pipeline vs. system image pipeline – Artifacts, versioning and release:
• Versioning, naming and storing firmware artifacts traceably (package registry)
• Generating a Software Bill of Materials (SBOM) automatically (SPDX/CycloneDX)
• Signing firmware, release pipeline with manual gate
• Update and OTA packages as pipeline output
• Hands-on: release job with versioned, signed artifact and SBOM – Evidence and compliance from the pipeline:
• Pipeline results as evidence for CRA and other industry standards
• Traceability: requirement → test → pipeline run → release
• Documentation-as-code: versioning documentation with the code and building it automatically
• Hands-on: traceability report and documentation as pipeline artifacts – Adoption in your own project:
• Bringing an existing project into the pipeline step by step: order and quick wins
• Dealing with resistance: long builds, little hardware, legacy code
• Working out a rollout plan for your own team

The course is deliberately tool-agnostic: we work hands-on with GitLab CI/CD, and the patterns – toolchain containers, host tests, runners at the test bench, artifacts and evidence – work just the same with GitHub Actions, Jenkins or Azure DevOps.

The actual course content may differ from the above depending on the trainer, delivery, duration and the composition of participants.