Stone pile on the beach

Functional Safety

What is the Development Effort?

Time to Read 4 min

Product development is already expensive in itself, now in addition you are to develop for functional safety. Many additional steps and documents stack up, see e.g. our list or the comprehensive blog post.

How to estimate the development effort of safety-critical systems? There is little publicly available information, what seems reasonable to me, I have collected and arranged:

Additional Effort Compared to "Normal" Development: Factors

My goal was to develop a few simple factors for coarse estimates of embedded systems (software & electronics) that are easy to remember. The factors should express the effort as a multiple of the effort for a standard development project.

You can find the factors in this table:

Practical Factor Quality & Safety Level (Examples)

Goals

  • Activities & Results
1 (Base)

"Normal" Product Development

Function under normal conditions

  • Product, e.g.
    • Electronics (manufacturing data)
    • Embedded software (code)
3

Structured Development

  • CMMI Level 3 (defined)
  • aSPICE Level 3 (established)
  • ISO 26262 QM
  • DO-178C DAL-E
  • ISO 13849 PL a

Maintainability, Extendability, Quality

additional:

  • Documents:
    • Requirements
    • Design description
    • Basic requirements traceability
  • Partial verification:
    • Reviews
    • Test
  • Configuration & change management
  • Process descriptions
  • Own standards
    • Requirements, Coding...
5

Critical Development

  • ISO 26262 ASIL-A/ B
  • DO-178C DAL-D/ C
  • IEC 61508 SIL-1/ 2
  • ISO 13849 PL b/ c/ d
  • IEC 62304 Class A/ B

Basic Functional Safety

additional:

  • Detailed planing: safety plans
  • Detailed requirements
    • Incl. traceability
  • Software Tool Qualification
  • Full verification:
    • Tests (incl. robustness, range...)
    • Code coverage analysis
    • Rigorous code reviews
    • Rigorous reviews of requirements & tests
    • Quality assurance audits
  • Formal change management
  • Documentation acc. standards
    • All design decisions
    • Traceability against requirements
  • Safety analyses
7

Highly Critical Development

  • ISO 26262 ASIL-C (?)/ D
  • DO-178C DAL-B/ A
  • IEC 61508 SIL-3/ 4
  • ISO 13849 PL e
  • IEC 62304 Class C

Full Functional Safety

additional:

  • (Semi-)formal requirements
  • In-depth verification:
    • More und deeper tests
    • Independent reviews
    • Complete code coverage analysis
    • Comprehensive quality assurance audits
  • Full requirements traceability
  • Detailed safety analyses

Effort per Line of Code for Functionally Safe Software

Lines of code (LOC) are a common, albeit not very accurate, measure of the scale of a software project. LOC for different programming languages are an even poorer measure for software projects. But since most functionally secure projects are implemented in C, the numbers are at least comparable in this respect.

This table shows which effort per line of code can be estimated by rule of thumb to implement functionally safe software projects.

Note that almost the same effort can be expected to "make safe" an existing software, since usually a profound refactoring of the software results from the safety requirements.

The order of magnitude for the efforts can be found in this table (where in the range your project falls depends mainly on its complexity, signal processing code e.g. easily reaches or exceeds the respective upper effort!)

Effort
Ph/ kLOC
Quality & Safety Level (Examples)

Goals

  • Activities & Results as above
125..500

Critical Development

  • ISO 26262 ASIL-A/ B
  • DO-178C DAL-D/ C
  • IEC 61508 SIL-1/ 2
  • ISO 13849 PL b/ c/ d
Basic Functional Safety
400..2500

Highly Critical Development

  • ISO 26262 ASIL-C (?)/ D
  • DO-178C DAL-B/ A
  • IEC 61508 SIL-3/ 4
  • ISO 13849 PL e (?)
Full Functional Safety

Ph/ LOC: Person Hours/ Thousand Lines of Code

For very coarse initial estimates, the simple rate of 1 Ph/ LOC (one hour per line of code) can be used.

Use of the Factors: Please Note

  • For very small and very complex projects, the effort will be higher.
    For the first category, because the overhead of all the documentation is relatively higher, for the second, because the complexity determines the effort.
  • For projects with only little safe functionality ("simple safety functions") the factors are rather too optimistic.
    This is because the effort for the implementation of e.g. safety mechanisms for hardware errors and also the entire effort for  processes and documentation is proportionally larger compared to the actual basic function. In a "normal project" only this basic function would be implemented.
  • The factors of course only apply to the functionally safe part of a project. Whenever possible the safety functions should be safely separated from the rest ("Isolation", "Freedom from Interference"), so that the effort does not accumulate over the whole product. Unless of course the whole product performs a safety function, e.g. a FADEC in aviation or a sensor for an automated driving function.

As mentioned, these factors are only suitable for an estimation "by rule of thumb". For a more accurate estimation of your concrete project we offer our estimation tools or contact us for a workshop.

What is the most important conclusion to be drawn from these figures, apart from their use in coarse estimates? KISS! ...Keep It Simple. Every feature multiplies in effort! Which can only be reduced by omitting as many unnecessary features as possible...

Andreas Stucki

Do you have other sources, numbers or experiences? Do you have additional questions? Do you have a different opinion? If so, email me  or comment your thoughts below!

Author

Comments

No Comments

What is Your Opinion?

* These fields are required

References

The references displayed as a table:

Quality Level

Practical Factor (see above) Factors acc. [1] Factors acc. [2] Factors acc. [2] Factors acc. [3] Factors acc. [3] Factors acc. [4] Factors acc. [4] Factors acc. [5]*** Factors acc. [5]
Factors acc. [5]                    
"Normal" Product Development Base: 1 Base: 1.0         Base: 1.0 Base: 2.0**    
Structured Development 3 3.2 Base: 1.0 Base: 3.0 Base: 1.0 Base: 2.5*        
Critical Development 5 4.4 1.2 3.6 := 3.0 * 1.2 2.0..2.9 5.0..7.3 := 2.5 * (2.0..2.9) 1.5..4.0 3.0..8.0 := 2.0 * 1.5..4.0 0.125..0.5 Base: 5.0 ^= 0.25****
Highly Critical Development 7 5.7 1.7 5.1:= 3.0 * 1.7 4.4..6.4 11..16 := 2.5 * (4.4..4.6) 5.0..10.0+ 5.0..20.0 := 2.0 * 5.0..10.0 0.4..2.5 8..50
:= 20 * 0.4..2.5

* Note that the base for [3] is CMMI level 2/3, a lower level than [1] , where level 3 is assumed; in [2] also level 2/ 3 is presumed, but the numbers are nearer at [1]. As a result, the steps to safety levels for [3] are probably too high. This has been corrected in the choice of this base.

** The basis was also adapted here, since "Functional System" in the context of the source (automotive) probably already has an aSPICE level.

*** Ph/ LOC

**** geometric mean

And the corresponding links:

[1] V. Hilderman: Calculate Critical Safety Cost Easy : (only effort, no tools cost included)

  • from "Basic Development" to "Requirements, Design, Test" to "Non-Certified Safety": plus 50% plus 50% plus 40% = 3.15
  • from "Non-Certified Safety" to "DAL-D/ ASIL-B": plus 40% = 1.4
  • from "DAL-D/ ASIL-B" to "DAL-B/ ASIL-D": plus 30% = 1.3

[2] V. Hilderman: DO-178C Cost versus Benefits :

  • from "DAL-E" to "DAL-D": plus 5% = 1.15
  • from "DAL-D" to "DAL-B": plus 35% plus 10% = 1.5

[3] Rockwell-Collins: Certification Cost Estimates for Future Communication Radio Platforms :

Refers to "industry established metrics" (p 26) and "industry averages" (p 27) of unknown source and to "Mentor Graphics" for hardware (p 27).

  • p 27: stating 75..150% more effort than Hilderman ("25..40%") for DO-178B, "presuming [..] CMMI Level 2 or 3 software engineering principles are used": from "Level 2/ 3" to "DAL-D": plus 100..190% = 2.0..2.9
  • p 29: from "DAL-D" to "DAL-B": plus 54% plus 43% = 2.2

[4] How the ISO 26262: 2018 Update Affects You: The Cost of ASIL Compliance :

"For example, to plan, execute, verify, and document compliance, the following effort multipliers could be considered:

Functional System : 1
ASIL A : 1.5x – 3x
ASIL B : 2x – 4x
ASIL C : 5x – 8x
ASIL D : 10x+"

[5] Cost of highly safety critical software

"DAL A: 3..12 SLOC/ day
DAL B: 8..20 SLOC/ day
DAL C: 15..40 SLOC/ day
DAL D: 25..64 SLOC/ day"

SLOC: Source Line Of Code

This yields for DAL A/ B: 0.4..2.5 LOC/ h and for DAL C/ D: 2..8 LOC/ h.

[6] Coverity: Risk Mitigation for DO-178C

"In typical cases, the cost of DO-178 certification can range from $25 to $100 per line of code—
that’s $2.5 million to $10 million for 100,000 lines of code!" (p 1) results at an hourly rate of 50 USD in: 0.5..2 LOC/ h.

Projects? Ideas? Questions? Let's do a free initial workshop!