Verification, Validation and TDD in Critical Firmware
Verification, Validation and TDD for Critical Converter Firmware
Why the firmware of a power converter is a critical reliability component — and how Premium SA treats it as such
| 1981 Year Boehm formulated the V&V distinction | V-Model EN 50128 methodological framework for railway firmware | MC/DC Minimum coverage required at SSIL 4 | 143 MISRA C:2012 rules (16 mandatory) |
Executive Summary
A modern power electronics converter is, today, as much software as hardware. The controller firmware executes regulation loops, protections, digital calibration, active thermal management, telemetry and, in an increasing number of cases, functional safety functions certified to railway SSIL or automotive ASIL. Any defect in that firmware is indistinguishable — from the end operator’s perspective — from a hardware defect: the equipment does not work, does not fulfil its mission, triggers contractual penalties. The difference is that firmware does not physically age, but it does degrade in terms of confidence if it is not subjected to a disciplined process of verification and validation.
This article reviews the fundamentals of software quality assurance applied to embedded power electronics firmware: Boehm’s critical distinction between verification and validation, the Test-Driven Development discipline, the V-Model of EN 50128, the MiL → SiL → PiL → HiL → SYSiL testing pyramid, black-box and white-box techniques, and the “terminological swamp” that persists in the industry. The thesis is direct: critical firmware is designed with the same discipline with which we design a power module — mission profile, derating, FMEA, V&V — because it shares the same customer and the same contractual commitments.
1. Firmware as a Critical Component — Why Discipline Matters
A typical Premium SA converter from fifteen years ago carried minimal firmware: an elementary state machine, a few PI loops implemented in fixed-point arithmetic, hardware protections reinforced with a couple of software checks. The complexity of the embedded code fitted in a few kilobytes and could be reviewed line by line by an expert in an afternoon. That era is over.
The modern converter — integrating digital calibration, active thermal management, predictive DC bus ripple control, CAN/Ethernet/Modbus communication with the operator’s supervisor, PHM telemetry with RUL estimation, OTA firmware update with cryptographic signature, and possibly active safety functions certified to railway SSIL — carries tens of thousands of lines of C code, organised in modular architectures with HAL, RTOS, priority tasks and communications libraries. The probability that such code is perfectly correct on its first writing is, simply, zero.
2. Verification and Validation — Boehm’s Distinction
The most clarifying distinction in software quality assurance was formulated by Barry Boehm in 1981 through two apparently similar but methodologically distinct questions:
2.1 The practical differences
| Dimension | Verification | Validation |
|---|---|---|
| Question | Are we building the product right? | Are we building the right product? |
| Orientation | To the internal technical process | To the customer and real use |
| Reference | Technical specification | User needs |
| Timing | Continuous, phase by phase | At the end of development or integration |
| Dominant techniques | Static analysis, reviews, unit testing, MISRA, MC/DC | System tests, acceptance tests, full HIL |
| Granularity | “Small-scale” tests | “Large-scale” tests |
| Who decides | The technical team | The customer / end user |
| Automatable | Mostly yes | Partially — the final word is human |
| Type of error detected | Implementation errors vs. the spec | Spec errors vs. the real world |
2.2 Why both are necessary
Impeccable verification without validation produces technically correct code that satisfies a specification that was itself defective — the team meets its obligations and the customer receives a product that does not solve their problem. Vigorous validation without verification produces code that “works in demo” but hides structural defects that emerge under operating conditions not contemplated in the acceptance test. The two disciplines are complementary and attack different classes of error, and no serious software engineering strategy can do without either of them.
3. The Terminological Swamp — Error, Defect, Failure, Incident
Any serious discussion about software quality carries the historical vocabulary problem. Multiple key terms — verification, validation, test, error, defect, fault, failure — are overloaded with conflicting meanings in the literature. A team that uses these terms interchangeably commits, unwittingly, reasoning errors that translate into process errors. It is worth anchoring the vocabulary to the IEEE standard.
| Term | Precise definition | Example in a power converter |
|---|---|---|
| Error | Incorrect human action. Mistake in specification, design or coding | An engineer assumes the current sensor is bipolar when it is unipolar |
| Defect / Fault / Bug | Internal manifestation of the error. Incorrect state embedded in the code | A branch of the OCP protection calculation that never executes because the condition is impossible |
| Failure | Observable inability of the system to fulfil its function — the visible symptom | The converter does not trigger OCP protection when current exceeds the threshold |
| Incident | Anomalous event during execution that merits recording and analysis | The operator reports an unexpected stop that does not match any known cause |
4. The V-Model — Verification at Every Level
The methodological framework that articulates verification throughout the software lifecycle is the V-Model. Its proposal is structural: to each level of specification corresponds a symmetric level of verification. The left branch descends from requirements to code; the right branch ascends from unit verification to final acceptance. The V-Model is the backbone of EN 50128 (railway), IEC 61508 (generic), ISO 26262 (automotive) and DO-178C (avionics).
4.1 The V correspondences
| Specification level (left branch) | Verification level (right branch) | Question answered |
|---|---|---|
| System requirements (SyRS) | Acceptance / system validation tests | Does the system meet user needs? |
| Software requirements (SRS) | System tests on firmware | Does the firmware satisfy its functional specification? |
| Architecture design | Integration tests between modules | Do the modules collaborate as designed? |
| Module design | Unit tests per module | Does each module do what its contract says? |
| Source code | Static analysis + inspection | Is the code free of dangerous patterns? |
4.2 SSIL — Software Safety Integrity Levels
EN 50128 defines five software integrity levels: Basic Integrity and SSIL 1 to SSIL 4. Each level imposes mandatory V&V techniques, scaled in rigour according to function criticality. Premium SA maps its applications to these levels according to the responsibility of each firmware block:
| SSIL level | Typical application in Premium SA converters | Mandatory EN 50128 techniques |
|---|---|---|
| Basic Integrity | Non-safety-related functions — telemetry, log, UI | Code review, functional tests |
| SSIL 1-2 | Cabin auxiliaries, standard OVP/OCP/OTP protections | MISRA C static analysis, unit tests with I/O coverage |
| SSIL 3 | Non-brake-critical traction converters, EV BMS | MISRA + decision coverage + response time analysis |
| SSIL 4 | Active safety functions (regenerative brake inhibition) | MC/DC + formal models + diverse redundancy |
5. Test-Driven Development — Tests as a Design Decision
Test-Driven Development (TDD) emerged in the early 2000s as one of the pillars of Extreme Programming and represents a radical inversion of the traditional workflow. In the classic order, the programmer writes functionality first and then — if there is time — adds tests to verify it. In TDD, the order is inverted: the test is written first, before the functionality exists.
5.1 The Red-Green-Refactor cycle
| Phase | Developer action | Code state |
|---|---|---|
| Red | Write a unit test for functionality that does not yet exist | Test fails — confirms the test measures something real |
| Green | Write the minimum code to make the test pass | Test passes — the minimum behaviour is implemented |
| Refactor | Restructure the code to eliminate duplication and improve design, without altering behaviour | Tests still pass — behaviour is preserved |
The cycle repeats in small increments — minutes, not hours. After a TDD session, the developer has produced both the production code and its test battery, and each line of the former is backed by at least one of the latter. The deepest methodological consequence is not having tests, but the inverted order: thinking the test before the implementation forces the developer to understand exactly what success or failure of the functionality means. That is, ultimately, an act of design.
5.2 TDD as a partial substitute for traditional design
5.3 TDD, BDD and ATDD — the evolution towards behaviour
Pure TDD is effective but has a limit: it is purely technical. A team using only TDD can produce code that passes all unit tests and yet turns out to be inconsistent with the real needs of stakeholders. Two variants have emerged to close this gap. BDD (Behavior-Driven Development) extends TDD by focusing on the expected behaviour of the system, expressed in a ubiquitous language (Given-When-Then) that both developers and customers can understand — the specification is not translated into tests, the specification is the test. ATDD (Acceptance Test-Driven Development) combines requirement specification with automated acceptance tests written before the code. The three practices are complementary: TDD ensures each function is well designed technically; ATDD ensures the complete system fulfils its business purpose; BDD provides the linguistic bridge between the two.
6. Testing Fundamentals — What We Test and How
As Edsger Dijkstra famously noted, testing can be very effective at demonstrating the presence of errors, but is hopelessly inadequate for demonstrating their absence. This limitation is structural: the input domain of any non-trivial program is so large that exhaustively testing all combinations is, in practice, impossible. The discipline of testing therefore consists in selecting a finite, representative subset of the domain that effectively exercises the critical characteristics of the system.
6.1 The test case and the oracle
The basic unit of testing is the test case: a defined set of conditions, input data and an expected result under which the software is executed to determine in a measurable way whether it meets a requirement. Each test case presupposes, implicitly or explicitly, an oracle: the arbiter that decides whether the result obtained is correct. In simple systems, the oracle is the specification itself. In complex systems where no closed specification exists, the oracle is a human expert — the tester who decides whether the observed behaviour matches the expected behaviour.
6.2 Black-box, white-box, and the critical distinction
| Aspect | Black-box (functional testing) | White-box (structural testing) |
|---|---|---|
| Knowledge of code | None — only the specification is known | Total — includes internal implementation |
| Case design | From expected inputs/outputs | From control flow paths |
| Coverage metrics | Requirements / equivalence class coverage | Statement, branch, MC/DC, path coverage |
| Detects | Discrepancies vs. functional specification | Structural defects, dead code, untested branches |
| Example in Premium SA firmware | Converter regulates output voltage with < 2 % error under standard load profiles | Each branch of the ADC interrupt handler executes at least once |
| Limitation | Does not detect defects in unspecified code | Does not detect missing requirements |
7. The MiL → SiL → PiL → HiL → SYSiL Pyramid
For embedded firmware — where code runs on a microcontroller coupled to a real physical system — testing discipline adds a layer of complexity that does not exist in classical application software. Tests cannot be limited to digital inputs and outputs; they must close the loop with the physical behaviour of the converter. The X-in-the-Loop methodology structures this challenge in five progressive levels:
| Level | What is tested | Test platform |
|---|---|---|
| MiL — Model-in-the-Loop | System model (Simulink, Stateflow) | Pure simulation on PC; no C code yet |
| SiL — Software-in-the-Loop | C code compiled for PC | Execution on host; I/O are simulated |
| PiL — Processor-in-the-Loop | Binary code in an emulator of the target processor | Verifies object code correctness without physical hardware |
| HiL — Hardware-in-the-Loop | Binary code on the real microcontroller | Firmware runs on target HW; plant simulated in real time |
| SYSiL — System-in-the-Loop | Complete physical embedded system | Final validation on bench with real hardware and real plant |
Each level resolves a class of defects before adding an additional layer of complexity. MiL captures modelling and control algorithm errors. SiL captures model-to-code translation errors. PiL captures target-compiler-specific errors, aggressive optimisations and finite arithmetic problems. HiL captures firmware-hardware integration defects: timing, interrupts, races, peripherals. SYSiL is the final validation with the real plant.
8. Premium SA’s D4Co-SW Process
Premium SA applies to firmware the same methodological framework as to hardware: the sequential gates D2W → D2F → D4R → D4Co, adapted to the specific techniques and artefacts of software.
| Gate | Meaning for firmware | Exit criteria |
|---|---|---|
| D2W — Design to Work | Firmware executes nominal functions | SRS signed, modular architecture defined, MiL/SiL prototype running |
| D2F — Design to Function | Firmware meets full functional specification | Unit tests ≥ 90 % branch coverage, HIL integration passed |
| D4R — Design for Reliability | Firmware maintains behaviour under adverse conditions | Fault injection passed, timing analysis, RTOS stress testing |
| D4Co — Design for Compliance | Firmware meets regulations and is maintainable for 30 years | EN 50128 SSIL dossier, MISRA C clean, FRACAS active |
8.1 Non-negotiable techniques — MISRA, MC/DC, HIL
- MISRA C:2012 — static analysis with Polyspace, Coverity or PC-lint on all C code. The 16 mandatory rules and 111 required rules are exit conditions of the D2F gate.
- MC/DC for SSIL ≥ 3 — every Boolean decision is tested with n+1 cases where each individual condition changes the result while the others remain fixed; for SSIL 4 it is complemented with formal analysis.
- HIL as exit criterion — no firmware with responsibility over active safety closes the D4Co gate without a complete HIL programme covering the mission profile space.
- Fault injection for SSIL ≥ 2 — sensor failures, analogue signal noise, communication loss and memory corruption are simulated. The system must always enter a safe state: PWM off, output high impedance, alarm active.
8.2 FRACAS applied to software
The FRACAS methodology — Failure Reporting, Analysis and Corrective Action System — that Premium SA applies to hardware extends to firmware. Any defect detected in the fleet is analysed at the appropriate level (function, module, shared library), the root cause is documented, and the correction is propagated: to all platforms sharing the code, via cryptographically signed OTA update, and to the development process, via a new review rule or new test case that prevents reintroduction of the defect. Fleet firmware improves over time, just as hardware improves through field returns.
9. Synthesis — Firmware as an Engineering Product
Critical firmware in a Premium SA converter is designed with the same discipline as a power module. The pillars are six and are chained:
| Pillar | Question answered | Premium SA practice |
|---|---|---|
| Precise vocabulary | Are we talking about the same problem? | IEEE glossary — error / defect / failure / incident differentiated |
| V&V per Boehm | Are we building the product right and are we building the right product? | Continuous phase-by-phase verification + final validation with customer |
| V-Model + EN 50128 | Is each level verified symmetrically? | Mapping SRS ↔ SyRS, module ↔ unit test, architecture ↔ integration |
| TDD / BDD / ATDD | Does the test precede the code? | TDD on critical modules; ATDD for system requirements |
| Black-box + white-box | Is the spec met and is all code exercised? | Branch coverage for SSIL 1-2, MC/DC for SSIL 3-4 |
| MiL → SiL → PiL → HiL → SYSiL | Does the firmware work on real hardware with a real plant? | Exhaustive HIL before SYSiL; nothing reaches the fleet without passing HIL |
About Premium PSU — With over 40 years of experience, Premium SA designs and manufactures custom power converters in Barcelona for railway, energy, industrial, data center and defence applications. Firmware development is integrated with hardware design from the conceptual phase of each project, following EN 50128, IEC 61508 and ISO 26262 practices.
Developing a converter with safety-related firmware? Talk to our engineering team.






