Designing a Qualified Onboard Software Production Pipeline using ECSS Standards
ECSS software assurance is essential for qualifying onboard software used in space missions. This article explains how ECSS standards can be applied in software engineering and software product assurance to support a qualified software production pipeline compatible with Model-Based Systems Engineering (MBSE) approaches and capable of assisting in the qualification of onboard software up to criticality A.
Introduction to ECSS Software Assurance
This essay describes the managerial and technical challenges of providing ECSS software assurance for software products developed for space projects following ECSS standards. A background on the description of critical software products and the requirements imposed by ECSS is presented, followed by the challenges associated with meeting these requirements. Finally, a design for a software product assurance pipeline is described. Hence, the aim of this essay is to present a convincing argument for investing in early software assurance efforts with progressive scaling as the phases of the space project advance.
Definition of Software Criticality in ECSS Software Assurance
Criticality in the space environment is a term used to identify systems or components whose functions could potentially have mission-threatening consequences if they were to cease operating due to any internal or external cause. This applies to all domains of space engineering, including software.
Software criticality for a system or component is assessed based on the criticality of the functions it executes, specifically its highest criticality function (ECSS-Q-ST-40C, 15 February 2017). Function criticality is assessed based on the severity of the potential consequences that could arise from that given function not operating as intended (ECSS-Q-ST-40C, 15 February 2017). Hence, software criticality follows the following logic:
Software Criticality Categories
Software criticality is broken down into four possible categories, as shown in Figure 2. These categories dictate the level of documentation needed to qualify a software product according to the space engineering software standard (ECSS-E-ST-40C, 30 April 2025). This entails that, according to the assigned level of criticality of a software product, one can then tailor the standard and its requirements. In the case of low criticality categories such as C and D, this can severely reduce the workload required for product assurance. This tailoring is presented in detail in (ECSS-E-ST-40C, 30 April 2025) Annex R Table R-1.
Following this logic of tailoring the standard, it is easy to see that the amount of effort required for ECSS software assurance greatly increases as software criticality increases. This is why software criticality A is rarely applied and typically reserved for software products that could lead to potential loss of life.
Impact of High-Criticality Software on the Project Life Cycle
Taking into consideration the exponential complexity of software product assurance efforts according to criticality categorisation, it is also to be expected that there could be potential impacts on projects that contain these critical software products regarding schedule, availability of domain experts, and overall budget.
A common representation of the impact of quality assurance in a project life cycle can be seen in Figure 3, where the relationship between cost and time when failure is found is represented. Figure 3 clearly illustrates that the sooner a failure is caught, the cheaper it is to mitigate that given failure. However, Figure 3 does not take into account the costs for product assurance efforts and focuses instead on the costs for handling the failure once found.
In the case of software product assurance, and specifically Category A software, Figure 3 can be recontextualised to show that not only the costs of a failure increase with time, but also those of implementing quality assurance processes. Furthermore, after a certain threshold the cost no longer is that of a repair or small implementation, but rather that of a full restart of the software project. This is graphically represented in Figure 4, where we can see three plots representing software categories A, B and C, being plotted with regard to cost of software assurance versus ECSS project phases. This plot is based on the author’s experience in the field of onboard software, as well as experiences shared with other industry experts during the ESA software product assurance conference at ESTEC in Q4 2025.
Figure 4 also shows that at a certain point a cost threshold is reached, after which the sunk cost fallacy applies due to the fact that it would be less costly to restart the software project with proper quality assurance strategies than continue investing in trying to rigorously qualify a code base that has no built-in qualification tools around it. This is mostly due to the requirements imposed by (ECSS-E-ST-40C, 30 April 2025) on critical software (Categories A, B and C), such as for example 5.8.3.5 part b, which lists 100% code coverage for Category A software in the following areas:
- Source code statement coverage.
- Source code decision coverage.
- Source code modified condition and decision coverage.
Furthermore, requirement 5.8.3.5 part e requires that the developer achieves 100% object code coverage for code that cannot be directly traced to source code.
Achieving this kind of code coverage in all these areas for any given onboard space application code base can prove a daunting task, and in most cases gaps in the test coverage will occur. This leaves the development team with no option but to provide justification for these gaps in coverage, for which deep knowledge and understanding of the code base is required. Taking this into consideration, it is clear that trying to apply 100% test coverage on a code base that has not been developed alongside all the needed tests is a gargantuan effort. It should also be noted that when trying to justify any gaps in test coverage, it will most likely prove quite challenging for developers to remember the very fine details of a design executed during Phase B if they are doing 100% code coverage in Phase D.
As such, the premise of this report is to convince the reader that the right time to apply ECSS software assurance and software qualification processes is during Phase 0, and that the best way to extend it to the following phases is by means of a qualified Continuous Integration (CI) pipeline for regression testing, such as the one proposed in the following section.
Design of a Continuous Integration Pipeline for Space Applications
The design of the proposed continuous integration pipeline for space applications is based around section 5.8 of (ECSS-E-ST-40C, 30 April 2025), which details the software code verification process and as such defines the requirements for achieving software product assurance. Its rough structure can be seen in Figure 5, which shows the step-by-step flow and the requirements addressed in each stage of the pipeline. The pipeline design is based on the understanding that software will be developed following best practices in version control, where software development is done in separate branches that can then be merged into the main software branch. Merge requests will trigger pipeline execution, ensuring that new code added to the application does not affect previously verified software. This is typically referred to as regression testing. The following sections provide details on each of the individual stages.
Static Code Analysis in ECSS Software Assurance
Static code analysis is a testing methodology that can be run on source code prior to execution. It is widely used in critical software development to check fulfilment of coding standards such as MISRA-C, and to identify bugs and security vulnerabilities. Some of the issues static code analysis can find prior to code execution are:
- Cyclomatic complexity levels above the recommended maximum of 10, leading to increased chances of programming defects.
- Type safety discrepancies such as type mismatches.
- Dead code or unreachable code.
- Memory leaks, stack overflow, null pointer dereferences, dangling pointers, use-after-free errors, and similar issues.
As such, implementing static code analysis as one stage of the software qualification pipeline helps fulfil the following (ECSS-E-ST-40C, 30 April 2025) 5.8.3.5 requirements:
- Part a.3: By checking compliance with a coding standard such as MISRA-C, static code analysis helps fulfil the requirement to be in conformance with coding standards.
- Part a.5: By checking for bugs and vulnerabilities in the source code, static code analysis provides part of the proof needed for verification of this requirement.
- Part a.7: Static code analysis helps identify corner cases that might result in runtime errors and otherwise remain undiscovered until operation.
- Part f: Measures of code robustness are greatly complemented by documentation produced by static code analysis tools.
Unit Tests for ECSS Software Assurance
Unit tests are software-based tests that can be written alongside the source code and are meant to target the smallest functional elements of the code, often referred to as units. Implementation of unit testing as one of the stages of the software qualification pipeline helps fulfil the following (ECSS-E-ST-40C, 30 April 2025) 5.8.3.5 requirements:
- Part a.7: Unit testing provides assurance at software function level that all possible behaviours, inputs and outputs of the function have been tested, and that as such the tested functions will not produce runtime errors.
- Part b: Code coverage, as shown in Figure 6, for the different categories listed by these requirements can only be achieved via unit testing:
- Source code statement coverage: Achieved by white-box testing with unit tests for each statement present in the code base.
- Source code decision coverage: Achieved by white-box testing with unit tests for each boolean expression of the code base.
- Source code modified condition and decision coverage (MC/DC): Achieved by testing the following elements of a function with unit tests:
- Each entry and exit point is invoked.
- Each decision takes every possible outcome.
- Each condition in a decision takes every possible outcome.
- Each condition in a decision is shown to independently affect the outcome of the decision.
Part c: Results from unit testing campaigns can provide all the necessary metrics to support code coverage evidence.
Functional Tests in ECSS Software Assurance
Unlike the earlier two stages, functional tests are project specific and are meant to verify that the software under test is compliant with project specifications. As such, these tests are highly related to requirements and the architectural elements of the software design, providing traceability between them that can be used in the software product assurance process. Implementation of functional tests into the software qualification pipeline helps fulfil the following (ECSS-E-ST-40C, 30 April 2025) 5.8.3.5 requirements:
- Part a.1: By providing direct traceability between requirements, design elements, source code and functional tests, this stage can demonstrate consistency between all these elements.
- Part a.3: Similarly to Part a.1, the achieved traceability from automatic functional testing also supports this requirement.
- Part a.5: By doing functional testing, consistency between previously tested software units is achieved. This is integration testing.
- Part a.6: By testing safety-related functionalities, this requirement can be fulfilled.
- Part a.7: Functional tests further prove that there are no runtime errors.
- Part a.8: Functional testing can also prove that error-handling methods work as expected.
Conclusion and Next Steps
As shown in the analysis and design presented in this essay, the implementation of an early ECSS software assurance pipeline can be greatly beneficial to a project’s schedule and budget, and in some cases it is absolutely essential.
As next steps, it is expected that a conference proceeding for SPIE 2026 will be prepared, presenting further details on this topic, as well as the implementation of the design presented here.
References
ECSS official standards portal: https://ecss.nl/
ECSS-E-ST-40C. (30 April 2025). ECSS-E-ST-40C Rev.1 Space engineering software. Noordwijk, The Netherlands: ESA-ESTEC Requirements & Standards Section.
ECSS-Q-ST-40C. (15 February 2017). ECSS-Q-ST-40C Rev.1 Space product assurance. Noordwijk, The Netherlands: ESA-ESTEC Requirements & Standards Division.
ECSS-Q-ST-80C. (30 April 2025). ECSS-Q-ST-80C Rev.2 Software product assurance. Noordwijk, The Netherlands: ESA-ESTEC Requirements & Standards Section.
Hamilton, T. (2024, November 21). Code Coverage Tutorial: Branch, Statement & Decision Testing. Retrieved from GURU99: https://www.guru99.com/code-coverage.html
Hayhurst, K. J. (2001). A Practical Tutorial on Modified Condition/. NASA.
IBM. (n.d.). Regression testing. Retrieved from IBM: https://www.ibm.com/think/topics/regression-testing
Kraeling, M. (2019). 11 – Safety-Critical Development. In R. Oshana, & M. Kraeling, Software Engineering for Embedded Systems (pp. 394-426).
McCabe, T. (1976). A Complexity Measure. IEEE, 308 – 320.
Miramar, H. (2025, September 19). What Is Static Code Analysis and How Does It Work. Retrieved from Mergify: https://articles.mergify.com/what-is-static-code-analysis/
SEAC. (2025). ECSS Standards Course. Retrieved from https://seac-business-school.mykajabi.com/products/ecss-standards-2
Frequently Asked Questions
