Functional Safety: the Additional Effort

Product development is already expensive in itself, now in addition you are to develop for functional safety. Many additional steps and documents stack up, see e.g. our list or the comprehensive blog post.

How can one estimate the effort for the development of such safety-critical systems? There is little information which is publicly available, so what seems reasonable to me, I have compiled here: references.

My goal was to develop a few simple factors for coarse estimates of embedded systems (software & electronics) that are easy to remember. The factors should express the effort as a multiple of the effort for a standard development project.

You can find the factors in this table:

     

Practical Factor Quality & Safety Level (Examples)

Goals

  • Activities & Results
 
1 (Base)

"Normal" Product Development

  • none
 

Function under normal conditions

  • Product, e.g.
    • Electronics (manufacturing data)
    • Embedded software (code)
 
3

Structured Development

  • CMMI Level 3
  • aSPICE Level 3
  • ISO 26262 QM
  • DO-178C DAL-E
 

Maintainability, Extendability, Quality

additional:

  • Documents:
    • Requirements
    • Design description
    • Basic requirements traceability
  • Partial verification:
    • Reviews
    • Test
  • Configuration & change management
  • Process descriptions
  • Own standards
    • Requirements, Coding...
 
5

Critical Development

  • ISO 26262 ASIL-B/ A
  • DO-178C DAL-D/ C
 

Basic Functional Safety

additional:

  • Detailed planing: safety plans
  • Detailed requirements
    • Incl. traceability
  • Full verification:
    • Tests (incl. robustness, range...)
    • Code coverage analysis
    • Rigorous code reviews
    • Rigorous reviews of requirements & tests
    • Quality assurance audits
  • Formal change management
  • Documentation acc. standards
    • All design decisions
    • Traceability against requirements
  • Safety analyses
 
7

Highly Critical Development

  • ISO 26262 ASIL-D
  • DO-178C DAL-B/ A
 

Full Functional Safety

additional:

  • (Semi-)formal requirements
  • In-depth verification:
    • More und deeper tests
    • Independent reviews
    • Complete code coverage analysis
    • Comprehensive quality assurance audits
  • Full requirements traceability
  • Detailed safety analyses
 

As mentioned, these factors are only suitable for an estimation "by rule of thumb". For a more accurate estimation of your concrete project we offer our estimation tools or contact us for a workshop.

What is the most important conclusion to be drawn from these figures, apart from their use in coarse estimates? KISS! ...Keep It Simple. Every feature multiplies in effort! Which can only be reduced by omitting as many unnecessary features as possible...

Do you have other sources, numbers or experiences? Please write to me or comment below.

Andreas Stucki

Keywords/ Tags

No comments

What is Your Opinion?

Share On

References

The references displayed as a table:

     

Quality Level

Practical  Factor (see above) Factors acc. [1] Factors acc. [2] Factors acc. [2] Factors acc. [3] Factors acc. [3]
"Normal" Product Development Base: 1 Base: 1.0        
Structured Development 3 3.2 Base: 1.0 Base: 3.0 Base: 1.0 Base: 2.5*
Critical Development 5 4.4 1.2 3.6 := 3.0 * 1.2 2.0..2.9 5.0..7.3 := 2.5 * (2.0..2.9)
Highly Critical Development 7 5.7 1.7 5.1:= 3.0 * 1.7 4.4..6.4 11..16 := 2.5 * (4.4..4.6)

* Note that the base for [3] is CMMI level 2/3, a lower level than [1] , where level 3 is assumed; in [2] also level 2/ 3 is presumed, but the numbers are nearer at [1]. As a result, the steps to safety levels for [3] are probably too high. This has been corrected in the choice of this base.

And the corresponding links:

[1]  V. Hilderman: Calculate Critical Safety Cost Easy : (only effort, no tools cost included)

  • from "Basic Development" to "Requirements, Design, Test" to "Non-Certified Safety": plus 50% plus 50% plus 40% = 3.15
  • from "Non-Certified Safety" to "DAL-D/ ASIL-B": plus 40% = 1.4
  • from  "DAL-D/ ASIL-B" to "DAL-B/ ASIL-D": plus 30% = 1.3

[2] V. Hilderman: DO-178C Cost versus Benefits :

  • from "DAL-E" to "DAL-D": plus 5% = 1.15
  • from  "DAL-D" to "DAL-B": plus 35% plus 10% = 1.5

[3] Rockwell-Collins: Certification Cost Estimates for Future Communication Radio Platforms :

Refers to "industry established metrics" (p 26) and "industry averages" (p 27) of unknown source and to "Mentor Graphics" for hardware (p 27).

  • p 27: stating 75..150% more effort than Hilderman ("25..40%") for DO-178B, "presuming [..] CMMI Level 2 or 3 software engineering principles are used": from "Level 2/ 3" to "DAL-D": plus 100..190% = 2.0..2.9
  • p 29: from  "DAL-D" to "DAL-B": plus 54% plus 43% = 2.2