Testing and CI¶
How the driver is checked: host tests against a model of the chip, a fuzzer that corrupts what the chip sends back, compiled documentation, an install test, ESP32 and Cortex-M builds, static analysis and measured quality gates, on every pull request.
Coverage matrix¶
docs/coverage-matrix.md has one row for each parameter of the datasheet. tools/check_matrix.py
runs in tools/preflight.sh and fails when a row cites a symbol that is not in the code, a test id that is not in
tests/, or when the row counts in the header are wrong.
Host tests¶
The tests build the real driver against a model of the ADS9324 written from the datasheet: the three register
banks with the AIN9 to AIN16 bank in reverse order, the 24-bit frames and the two-frame read, the data frame in 8,
4, 2 and 1 lanes with 16- and 24-bit words, the DIAG byte and the randomizer, all 15 channel modes, the test
patterns, the window comparator with its latched flags and the alarm pin, the ADC calibration with its timing,
standby and the reset flag. Faults can be injected into every transfer. The tests build with the firmware's
flags, -fno-exceptions -fno-rtti, and warnings as errors, on GCC and Clang.
cmake -S . -B build -DADS9324_BUILD_TESTS=ON
cmake --build build
ctest --test-dir build --output-on-failure
| File | Covers |
|---|---|
tests/test_startup.cpp |
Lifecycle, the start-up sequence and its order, identity checks, reset, a failure injected at every transfer, standby and wake |
tests/test_readout.cpp |
Data frames: every range, clipping, offset binary, the randomizer, DIAG and PRBS, 24-bit words, all 15 channel modes, every lane count, stuck and floating lines, test patterns, the self-test |
tests/test_config.cpp |
Setters: the mirrored bank mapping of every channel, the shadow equals the chip after every setter, minimal traffic, write ordering, Verify::ReadBack, validation |
tests/test_features.cpp |
The filters and the SYSREF pulse, the window comparator and the alarm pin, the ADC calibration, the reference, the model's knobs |
tests/test_types.cpp |
Enumerations, Validate(), the pure helpers (ranges, rates, decoding) |
tests/test_registers.cpp |
The driver's register map against the model's: 116 registers, reset values, banks |
tests/test_errors.cpp |
The base Error values are 0 to 9; every chip-specific value (16 and up) is returned by some call, and the three the chip cannot produce never are |
tests/test_version.cpp |
The version getters follow the generated header (also against a 9.8.7 probe) |
tests/test_kconfig.cpp |
KconfigDefaults() with and without CONFIG_ values |
tests/fuzz/ |
The fuzz target; CTest replays its corpus (ads9324_fuzz_ads9324_corpus), see below |
tests/compile_fail/ |
Code that must not compile, each for its EXPECT: diagnostic: a port without DelayUs(), a void GpioSet() or GpioGet(), a DelayUs(uint16_t), a narrowed CaptureLanes(), a Log() returning int, a hook without noexcept, an invalid CheckedConfig, a daisy chain in a CheckedConfig, a class that is not a port |
tests/mock_ads9324.hpp |
The chip model, with its own register map and a Knobs struct for every assumption in Limitations |
tests/check_version_copy.cmake |
_config/Doxyfile carries the CMake version |
examples/host/ |
The host example, built with the test flags and run (ads9324_host_example) |
tests/check_install.cmake |
Installs into a scratch prefix and builds tests/install_consumer/ with find_package(ads9324) |
tests/check_doxygen.cmake |
Doxygen with _config/Doxyfile and warnings as errors, when doxygen is installed |
The model stands alone¶
tests/mock_ads9324.hpp is written from the datasheet, not from the driver: it does
not include ads9324_registers.hpp or ads9324_types.hpp. Its register map, field masks, reset values, timing and
data-frame layout are its own, so a wrong constant in the driver cannot hide behind the same wrong constant in
the model (handbook KB-TST-030). test_registers.cpp compares the two for every documented register, so a wrong
address, reset value or bank fails a test that names the register.
When that test stops, one of the two maps is wrong: check it against the datasheet section the model names, and fix that one. A new register goes into both. Every behaviour found on hardware becomes a knob or a fault mode of the model and a test that fails on the code before the fix.
Fuzz tests¶
Everything the driver reads comes from the chip over the bus, so the data read is what a noisy cable, a wrong part
or a broken chip controls. tests/fuzz/fuzz_ads9324.cpp lets libFuzzer choose a
configuration, then up to 64 driver calls, and on every transfer what the bus does: pass it to the model, fail it,
flip bits of the data read or replace it. Between calls it can change the chip: a stuck or floating data line, a
brown-out, a reset flagged in the DIAG byte, a slow calibration, the other readings of the datasheet's open
questions. After every call it checks the driver's promises and aborts, naming the line, when one breaks:
- a failed call returns an
Errorthe enum defines and a zero value; - before
EnsureInitialized()no call makes a transfer; - only documented registers are written, only with documented codes (a bank, a range, a lane count, an alarm function), and no read-only register and no daisy-chain register;
- every readout frame sends zeros and has the length the configuration implies;
- the ADC calibration is never started with a digital filter on, and a configuration change never turns it off;
- a reset flag in a data frame leaves the driver uninitialised;
- every call ends: the time the port is asked to wait is bounded;
- while no fault is injected the shadow equals the chip after every setter;
- every log line is a non-empty plain-text message.
# libFuzzer needs Clang (on macOS, Homebrew's LLVM)
cmake -S . -B build-fuzz -DCMAKE_CXX_COMPILER=clang++ -DADS9324_BUILD_TESTS=ON -DADS9324_BUILD_FUZZERS=ON
cmake --build build-fuzz --target fuzz_ads9324
./build-fuzz/tests/fuzz/fuzz_ads9324 -max_total_time=120 tests/fuzz/corpus/fuzz_ads9324
The same file also builds without libFuzzer as fuzz_replay_fuzz_ads9324, and CTest replays every input in
tests/fuzz/corpus/fuzz_ads9324/ on GCC and Clang
(ads9324_fuzz_ads9324_corpus): the seeds, and every input that ever broke a promise (name it regress-*), so a
fixed bug stays fixed. CI fuzzes for 120 s on every pull request, from the seeds plus a corpus it keeps between runs
(job Host quality / fuzz).
Quality gates¶
The Host quality job measures the library code, inc/ and src/ only (tests and examples are left out), and
fails when a value is over the limits of this repository's assurance level, high (set in
.kokeb/handbook.yml, defined by the handbook's ADR-0020):
| Gate | Tool and how it is measured | Limit at high |
|---|---|---|
| Complexity | lizard 1.24.1, modified cyclomatic complexity (-m: a switch counts once) per function |
CCN at most 15 |
| Function length | lizard, lines of code per function | at most 60 |
| Parameters | lizard, per function | at most 6 |
| Line coverage | GCC 14 --coverage -O0, every CTest, gcovr 8.7 --merge-lines (a template's lines count once across its instances), --exclude-unreachable-branches --exclude-throw-branches |
at least 85 % |
| Branch coverage | same run | at least 75 % |
| MC/DC | GCC -fcondition-coverage, same run |
measured, not gated at high (100 % at safety) |
| Sanitizers | Clang, -fsanitize=address,undefined -fno-sanitize-recover=all, every CTest |
no report |
| Fuzzing | libFuzzer with ASan and UBSan, 120 s per target | no broken promise |
The reports are in the run's host-quality-reports artifact and its step summary. To run the gates locally:
# Coverage (GCC 14 or later), then the complexity limits
cmake -S . -B build-cov -DCMAKE_CXX_COMPILER=g++-14 -DADS9324_BUILD_TESTS=ON \
-DCMAKE_CXX_FLAGS="--coverage -O0 -fcondition-coverage"
cmake --build build-cov && ctest --test-dir build-cov
gcovr --root . --filter inc/ --filter src/ --merge-lines --exclude-unreachable-branches --txt build-cov
lizard -m -C 15 -L 60 -a 6 inc src
# Sanitizers (Clang)
cmake -S . -B build-san -DCMAKE_CXX_COMPILER=clang++ -DADS9324_BUILD_TESTS=ON \
-DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-sanitize-recover=all -O1 -g"
cmake --build build-san && ctest --test-dir build-san --output-on-failure
The fuzz gate runs locally as shown in Fuzz tests.
clang-tidy 20 (_config/.clang-tidy: bugprone, CERT, clang-analyzer,
cppcoreguidelines, modernize, performance, portability and readability, each disabled check with its reason) and
cppcheck (strict) run in the C++ lint and Repository checks jobs and fail on any finding.
The MISRA job (ci-misra.yml) runs the shared ru-misra workflow of
ci-tools at level high: it traces a host build and the Cortex-M build (ads9324_optimised_build) of every
translation unit, checks them against MISRA C++:2023 and the handbook's C++20 profile (the CodeQL coding-standards
and profile packs, plus the validated clang-tidy and cppcheck checks), and reports each finding against the
handbook's compliance matrix and deviation records. A finding is fixed in the code; a deliberate exception names
its deviation on the line before it (// codeql::misra_deviation_next_line(MD-CPP-009) above the .ipp include
of inc/ads9324.hpp). Until the repository is public the job runs with open_source_prerelease: true, the
declaration that the code is released under the licence in LICENSE and is to be published.
Documentation snippets¶
Every C++ block on this site is included from tests/docs/, which the host build
compiles (target ads9324_doc_snippets) with the driver instantiated over the documented port. A change to
the API that breaks a page breaks the build first. The README's example is a generated copy of the same file;
kokeb-docs check fails when it drifts.
CI workflows¶
Everything runs from ci.yml, which calls one workflow file per check and ends in
the CI gate job, the single required check (CI gate / gate).
| Job | What it checks |
|---|---|
| Host tests | GCC, then Clang (one job), build and run every test above, then build and run examples/host on its own |
| Host quality | The gates above: complexity, coverage, sanitizers, fuzzing |
| Cross-compile | arm-none-eabi-g++ for Cortex-M0+, M4F and M7F with the strict flags, warnings as errors (nothing linked) |
| C++ lint | clang-format and clang-tidy over the sources |
| MISRA | MISRA C++:2023 and the C++20 profile on traced host and Cortex-M builds, at level high (see Quality gates) |
| Docs | Strict site build (_config/mkdocs.yml), figure and generated-region drift, Doxygen (_config/Doxyfile), offline link check, Lighthouse |
| Repository checks | One job: DCO on pull requests, the org standard, Markdown and YAML lint, the Kokeb Labs handbook rules for .kokeb/handbook.yml (by rule ID), template drift (warns only) and cppcheck over the driver sources; a second job runs the handbook's source checks (include guards, bus requirements, raw access) |
| CI gate | Passes only if every job above passed: the one required check |
The ESP32 examples job builds every app in examples/esp32/app_config.yml, for each listed ESP-IDF version and
build type, with the repository root as the driver's component, and lints the example sources. The gate needs it
too.
The workflows call the shared, versioned workflows of
kokeb-labs/ci-tools at @v1.
On hardware¶
The ESP32 example (Examples) is the bench test: it logs four channels twice a second. Follow the bench steps on the Limitations page, apply a known voltage to AIN9 and AIN16, and record the board and the result in the pull request's bench-status section.