All material on this web site is copyrighted by Honourcode, Inc. 1999-2013.


Systems Engineering Return on Investment (SE-ROI)

Results from 15 Years of Research, with data from 94 programs

There is a strong quantified relationship between systems engineering and program success. The relationship can be seen in the following charts that plot three program success measures against the level of SE effort used on the program. (Click on each chart to see a full-size version.)

  • Optimum SE activity for median programs is 14.4% of the total program cost.
    Chart-Cost v ESEE

 Median programs used much less SE than the optimum
  • ROI is 3.5:1 to re-allocate additional effort into SE (for median programs)

No significant correlation exists between SE activities and technical quality of the system.

  • Technical quality measured by performance against stakeholder-desired parameters
  • It appears that an over-emphasis on requirements defeats creating better systems, even within the same cost and schedule.

 

 Similar relationships also available for eight subordinate SE activities
  • Mission/Purpose Definition
  • Requirements Engineering
  • System Architecting
  • System Integration
  • Verification & Validation
  • Technical Analysis
  • Scope Management
  • Technical Leadership/Management

Optimizing level of Technical Leadership/Management simultaneously optimizes cost, schedule, and stakeholder acceptance.

A new estimation method is now available for optimum program SE effort based on program characteristics

  • Estimates the optimum level rather than simply matching what other programs have done.

Downloadable information:

Contact Eric Honour for information at ehonour@hcode.com or +1 (615) 614-1109.

Background Information

What Did We Know Before?

Results from the preliminary (2004) Value of Systems Engineering work:

  • Better technical leadership correlates to program success. [Ancona 1990, Miller 2000]
  • Better/more systems engineering correlates to shorter schedules by 40% or more, even in the face of greater complexity. [Franz 1995, Honour 2004]
  • Better/more systems engineering correlates to lower development costs, by 30% or more. [Gruhl 1992, Barker 2003, Kludze 2004, Honour 2004]
  • Optimum level of systems engineering is about 15% of a total development program. [Gruhl 1992, Honour 2004]
  • Programs typically operate at about 6% systems engineering. [Kludze 2004, Honour 2004]
  • Parametric cost estimation of systems engineering is possible. [Valerdi 2004]
  • SE practices correlate to program success. [Gamgee 2006]

Ontology of Systems Engineering

The research used a review of the current systems engineering standards to define categorization of eight activities that appears to represent the industry ontology ("shared conceptualization"). The table below shows the eight categories compared with standards.

Download a usable version of this table

Approach

The project data combines the prior 2004 survey data with interview data performed between 2008 and 2010. The interview data included:

  • Project characterization data such as project size, project type, development phases, bounding parameters, risk levels.
  • Project success data such as cost/schedule compliance and technical quality measures.
  • Systems engineering data such as hours expended on systems engineering tasks, quality of those tasks, specific nature of the methods and tools used

Demographic information on the interviews is available. Detailed data is protected by confidentiality agreements and by separation of the data from the project identifications. Only aggregated data is shared with others.

Support

SE-ROI has been primarily funded by Honourcode, Inc.and the Defence and Systems Institute (DASI) of the University of South Australia.

Page last modified: 26 Mar 13