Confidence in an embedded product comes from demonstrating its required behaviour under both normal and failure conditions. Successful operation is part of that evidence. Disconnected devices, timeouts, unexpected inputs, power interruptions and extreme operating conditions also need deliberate attention.
Start with measurable requirements, test them at suitable levels and establish what the system should do when an operation cannot complete. Recovery, reporting a fault and stopping are all behaviours that can be specified and verified.
Make requirements observable and measurable
A useful behavioural requirement describes a condition, an action and an observable result. Timing requirements should state the deadline and the conditions under which it applies. Failure behaviour should be defined with the same precision.
For example, a device might have these two requirements:
- Under the specified operating conditions, complete an operation and report the expected result within the defined time limit.
- If the operation cannot complete within that limit, report a timeout and follow the defined failure response.
These describe different behaviours and provide identifiable test objectives. A test may check several outputs, such as returned values, device state and timing, where they belong to the same scenario. Each check should have an explicit expected result.
Keep the relationship between requirements and test evidence visible. One well-defined test may demonstrate a requirement, while boundaries, configurations or failure conditions can require additional cases. Traceability establishes what has been demonstrated; the number of tests follows the behaviour being verified.
Give each test level a clear purpose
Unit tests exercise individual pieces of code and their expected behaviour. They provide repeatable checks for calculations, decisions, state changes and error-handling paths without requiring the complete device.
Simulation tests exercise the implementation against controlled scenarios and expected outputs. Where the simulator provides the necessary interfaces and controls, these tests should be automated too. Simulation can make it practical to reproduce particular failures and run them repeatedly during development.
Device tests exercise the implementation together with the actual hardware. They establish confidence in integration, real interfaces and the operating conditions that a simulator may not reproduce accurately. The aim is identifiable device-test evidence for requirements that must be demonstrated on the target system.
A simulator needs its own checks against the actual device. Agreement with a model is useful evidence only to the extent that the model represents the behaviour being tested. Document its limitations and decide which requirements still need direct hardware evidence.
Plan failure, boundary and configuration cases
List the relevant failures and define the expected response before implementing the test. Include device disconnection, delayed or absent responses, unexpected input and interruption during an operation where they apply to the product.
Exercise boundaries and extremes as well as ordinary values. For a timeout, test responses before, at and after the specified deadline, with the expected boundary behaviour defined. For constrained data or resources, test the supported limits and the required response when they are exceeded.
Include representative successful operation for each supported configuration and distinct path, then add the applicable failure and boundary scenarios. One successful case provides a baseline; the rest of the evidence demonstrates behaviour under the other required conditions.
Cover identified failure modes and credible combinations according to the system's risks. No finite test suite demonstrates every conceivable failure. Keep assumptions and remaining limitations explicit so that coverage can be reviewed meaningfully.
Use a deliberate fallback for unexpected paths
A generic fallback can be useful early in development. A message such as “Error: this code path is unimplemented” makes unfinished behaviour visible while specific handling is being added.
For a release, the fallback must provide the defined response. Recover and log the event where recovery is appropriate and the system can continue in a known valid state. Where that cannot be established, follow the required stop or intervention policy. An unimplemented required path remains incomplete even if its fallback produces a clear log message.
Test both the specific error handlers and the generic fallback. Verify the reported outcome, retained state and subsequent operation, including whether further work is permitted. A failure that produces no exception can still leave the system in the wrong condition.
Verify recovery and diagnostics together
Recovery should leave evidence of the original failure, the attempts made and the final outcome. Check retry limits, timeout behaviour and escalation alongside the diagnostic records. Where human intervention is required, verify that the need is made clear.
Power interruption needs checks appropriate to the product, including behaviour after restart and the validity of any retained state. Fault injection on real equipment should use a controlled test arrangement appropriate to its hazards. Test scenarios and expected outcomes should be established before execution.
For more on the information needed during investigation, see Diagnostics are part of product design.
Set a release gate against the agreed scope
The release decision should be made against an agreed set of requirements and recorded test results:
- All requirements committed for that release have passed their verification criteria.
- No known critical faults remain unresolved.
- Known remaining issues have acceptable handling and are documented with the release.
Documenting an issue does not make it acceptable automatically. Assess its consequences and confirm that it does not invalidate a required behaviour or leave a critical risk. Record the build, configuration and hardware associated with the evidence, and repeat relevant checks when those change.
Passing a release gate establishes confidence in the agreed scope and tested conditions. It does not establish that the software contains no undiscovered defects. Clear requirements, repeatable tests and explicit limitations make that confidence easier to assess and maintain.
A practical starting point
Build a requirements-to-verification checklist with the expected behaviour, test level, configurations and result for each entry. Use it to plan normal, failure and boundary scenarios, automate repeatable checks and identify the hardware evidence still needed.
For further reading, NASA's requirements-validation guidance discusses testable requirements and coverage of normal and off-nominal conditions.