Skip to main content
×
×
Home

Relative impact of early versus late design decisions in systems development

  • James J. Y. Tan (a1), Kevin N. Otto (a2) and Kristin L. Wood (a1)
Abstract

To better understand the impact of early versus late design decisions, a study was undertaken on the root causes of missed requirements in new product development and their impact on development cost through rework activities. The context is the industrial development of unmanned aerial vehicles. The aim is to understand the occurrence rate of missed requirements, their root causes, and their relative impact. A quantitative approach of counting requirements changes and using engineering documentation enabled traceability from observation back to root cause origin. The development process was partitioned into sequential program segments, to categorize activities to before and after concept and design freeze. We found that there was a significant difference in the rate of design defects arising before and after concept freeze; and found there was a significantly higher number of corrective activities required for design defects arising earlier before concept freeze. The revision rate of concept phase decisions was over 50%, and the rework multiplier if detected late was over 10X. In combination, design decisions made before design freeze accounted for 86% of the total expected program cost, and 34% was determined before concept freeze. These results quantify and support the anecdotal 80–20 impact rule for design decisions.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Relative impact of early versus late design decisions in systems development
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Relative impact of early versus late design decisions in systems development
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Relative impact of early versus late design decisions in systems development
      Available formats
      ×
Copyright
Distributed as Open Access under a CC-BY 4.0 license (http://creativecommons.org/licenses/by/4.0/)
Corresponding author
Email address for correspondence: esengcorp@gmail.com
References
Hide All
Aerospace Industries Association2009 S3000L International procedure specification for Logistics Support Analysis (LSA).
Agarwal, H., Renaud, J. E., Preston, E. L. & Padmanabhan, D. 2004 Uncertainty quantification using evidence theory in multidisciplinary design optimization. Reliability Engineering and System Safety 85 (1–3), 281294.
Agte, J., Shougarian, N. & De Weck, O. 2012 Multistate analysis and optimization of a geared turbofan engine lubrication system. In 12th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference and 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, p. 5637. American Institute of Aeronautics and Astronautics.
Allard, T.2005 Navy’s $100m chopper can’t fly in bad light. Retrieved September 30, 2016, from http://www.smh.com.au/news/National/Navys-100m-chopper-cant-fly-in-bad light/2005/03/18/1111086017635.html.
Almefelt, L. & Andersson, F. 2004 Requirements as a means to drive innovation: a reason based perspective. In ASME 2004 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pp. 8998. American Society of Mechanical Engineers.
Almefelt, L., Berglund, F. & Nilsson, P. 2006 Requirements management in practice: findings from an empirical study in the automotive industry. Research in Engineering Design 17 (3), 113134.
Anderson, J. D. J. 2004 Fundamentals of Aerodynamics. McGraw–Hill.
Ball, L. J., Evans, St. B. T. J., Dennis, I. & Ormerod, T. C. 1997 Problem-solving strategies and expertise in engineering design. Thinking and Reasoning 3 (4), 247270.
Ball, L. J., Onarheim, B. & Christensen, B. T. 2010 Design requirements, epistemic uncertainty and solution development strategies in software design. Design Studies 31 (6), 567589.
Batchelor, T.2016 Taxpayers foot 1billion after MOD ‘failed to properly test’ Type 45 destroyer engine. Retrieved September 30, 2016 from http://www.express.co.uk/news/uk/692341/Type-45-destroyer-engine-problems-analyst-claims-MoD-failed-properly-test-engines.
BBC News2002 ‘Wobbly’ millennium bridge fixed. Retrieved September 30, 2016 from http://news.bbc.co.uk/2/hi/uk_news/1026224.stm.
Bergsjö, D., Almefelt, L. & Malmqvist, J. 2010 Supporting requirements management in embedded systems development in a lean-influenced organization. In 11th International Design Conference, DESIGN 2010, Dubrovnik, Croatia 17-20, pp. 10251034.
Bernard, B., Eric, C. & Archag, T.2015 Stop multibillion dollar development delays. Oilver Wyman, http://www.oliverwyman.com/insights/publications/2015/jun/stop-multibillion-dollar-development-delays.html#.V9akTCh96Uk. Retrieved on 12th September 2016.
Bertoni, M., Bertoni, A. & Isaksson, O. 2016 Evoke: A value-driven concept selection method for early system design. Journal of Systems Science and Systems Engineering. doi:10.1007/s11518-016-5324-2.
Boehm, B. & Basili, V. R. 2001 Software defect reduction top 10 list. Computer 34 (1), 135137.
Boehm, B. W. & Papaccio, P. N. 1988 Understanding and controlling software costs. IEEE Transactions on Software Engineering 14 (10), 14621477.
Brothers, C.2009 Germany and france delay decision on airbus military transport. Retrieved September 30, 2016 from http://www.nytimes.com/2009/06/12/business/global/12airbus.html?_r=2&ref=europe&mtrref=en.wikipedia.org&gwh=7F9EBE538FF6A8649C4E390CE9169DBE&gwt=pay.
Burr, T.2008 Chinook MK 3 Helicopters (Rep. No. HC-512). National Audit Office, London.
Camburn, B. A., Jensen, D., Crawford, R., Otto, K. & Wood, K. 2015a Evaluation of a strategic method to improve prototype performance with reduced cost and fabrication time. In DS 80-4 Proceedings of the 20th International Conference on Engineering Design (ICED 15) vol. 4: Design for X, Design to X, Milan, Italy, 27–30.07.15.
Camburn, B., Dunlap, B., Gurjar, T., Hamon, C., Green, M., Jensen, D., Crawford, R., Otto, K. & Wood, K. L. 2015b A systematic method for design prototyping. ASME Journal of Mechanical Design (JMD) 137 (8), Paper No: MD-14-1487, doi:10.1115/1.4030331.
Camburn, B., Dunlap, B., Kuhr, R., Viswanathan, V., Linsey, J., Jensen, D., Crawford, R., Otto, K. & Wood, K. L. 2013 Methods for prototyping strategies in conceptual phases of design: framework and experimental assessment. In Proceedings of the ASME 2013 International Design Engineering Conferences & Computers and Information in Engineering Conference, IDETC/CIE 2013, Portland, OR, August 4–7, 2013, DETC2013-13072.
Camburn, B., Sng, K., Perez, K. B., Otto, K., Wood, K. L., Jensen, D. & Crawford, R. 2015c The way makers prototype: principles of DIY design. In Proceedings of the ASME 2015 International Design Engineering Conferences & Computers and Information in Engineering Conference, IDETC/CIE 2015, Boston, MA, August 2–5, 2015, DETC2015-46295.
Capers, R. S. & Lipton, E. 1993 Hubble error: Time, money and millionths of an inch. The Academy of Management Executive (1993–2005), pp. 4157. https://griffith.rl.talis.com/items/DA245EAD-ECCD-E776-5446-29AC358C3A35.html.
Charette, R. N. 2008 What’s Wrong with Weapons Acquisitions. IEEE Spectrum.
Chong, D. S., Van Eerde, W., Chai, K. H. & Rutte, C. G. 2011 A double-edged sword: The effects of challenge and hindrance time pressure on new product development teams. IEEE Transactions on Engineering Management 58 (1), 7186.
Clarkson, J. & Eckert, C.(Eds) 2010 Design Process Improvement: A Review of Current Practice. Springer Science & Business Media.
Clausing, D. P. 1994 Total Quality Development. (A Step by Step to World Class Concurrent Engineering) . The American Society of Mechanical Engineers.
Cooper, R. G.1990 Stage-gate systems: a new tool for managing new products. Business Horizons May–June 1990.
Cooper, R. G. 2011 Winning at New Products, Creating Value Through Innovation. Basic Books.
Cooper, R. G. & Kleinschmidt, E. J. 1988 Resource allocation in the new product process. Industrial Marketing Management 17 (3), 249262.
Crawley, E., De Weck, O., Magee, C., Moses, J., Seering, W., Schindall, J. & Whitney, D. 2004 The Influence of Architecture in Engineering Systems (Monograph). MIT Press.
Department of Defense2005 MIL-HDBK-516B: Airworthiness Certification Criteria.
Department of Defense2015 Department of Defense Instruction, Number 5000.02. Department of Defense.
Department of Defense, System Management College2001 Systems Engineering Fundamentals. Department of Defense.
De Weck, Olivier, L. & Eckert, C. 2007 A Classification of Uncertainty for Early Product and System Design. MIT Press.
Dunlap, B., Hamon, C., Camburn, B., Crawford, R., Jensen, D., Green, M., Otto, K. & Wood, K. L. 2014 Heuristics-Based Prototyping Strategy Formation: Development and Testing of a New Prototyping Planning Tool. In Proceedings of the ASME 2014 International Mechanical Engineering, Congress & Exposition, IMECE2014-39959, November 14–20, 2014, Montreal, Quebec, Canada.
Eckert, C., Isaksson, O. & Earl, C.2012 Product property margins: an underlying critical problem of engineering design. In TMCE 2012.
Geoffrey, B. 1994 Product Design for Manufacturing and Assembly Volume 26. Computer Aided Design.
Govan, F.2013 £2 billion Spanish navy submarine will sink to bottom of sea. Retrieved September 30, 2016, from http://www.telegraph.co.uk/news/worldnews/europe/spain/10073951/2-billion-Spanish-navy-submarine-will-sink-to-bottom-of-sea.html.
Hales, C. & Gooch, S. 2011 Managing Engineering Design. Springer Science & Business Media.
Hamada, H. 1996 The Importance of Getting it Right the First Time. European Community Quarterly Review 4 (3), 1996. As presented at the October 1991 EuroPACE Quality Forum.
Hazelrigg, G. A. 1998 A framework for decision-based engineering design. Transactions American Society of Mechanical Engineers Journal of Mechanical Design 120, 653658.
Huang, E., Xu, J., Zhang, S. & Chen, C. H. 2015 Multi-fidelity model integration for engineering design. Procedia Computer Science 44, 336344.
Keller, R., Eger, T., Eckert, C. M. & Clarkson, P. J. 2005 Visualising change propagation. In DS 35: Proceedings ICED 05, the 15th International Conference on Engineering Design, Melbourne, Australia, 15–18.08.2005.
Kenway, G. K. & Martins, J. R. 2014 Multipoint high-fidelity aerostructural optimization of a transport aircraft configuration. Journal of Aircraft 51 (1), 144160.
Ladyman, J., Lambert, J. & Wiesner, K. 2013 What is a complex system? European Journal for Philosophy of Science 3 (1), 3367.
Mark, V. A., Robert, S. L., Shelia, E. M. & Obaid, Y.2006 Historical cost growth of completed weapon system programs. RAND Corporation, RAND.
NASAError cost escalation through the project life cycle. Retrieved September 2016 from https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20100036670.pdf.
North Atlantic Treaty Organization2009 STAGNAG 4671: UAV system airworthiness requirements.
Obaid, Y., David, E. S., Mark, A. L. & Frances, M. L.2005 Lessons learnt from the F/A 22 and F-18 E/F developmental programs. RAND Corporation.
Ohno, T. 1988 Toyota Production System: Beyond Large-Scale Production. CRC Press.
Otto, K. N. & Kristin, L. W. 2001 Product Design, Techniques in Reverse Engineering and New Product Development. Prentice Hall.
Pandey, V., Mourelatos, Z. & Castanier, M. 2014 Decision topology assessment in engineering design under uncertainty. In ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, V02BT03A04.
Perez, R. E., Liu, H. H. & Behdinan, K. 2004 Evaluation of multidisciplinary optimization approaches for aircraft conceptual design. In AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY.
Ries, E.2011 The lean startup: How today’s entrepreneurs use continuous innovation to create radically successful business. Crown Business.
Saad, N. M., Al-Ashaab, A., Maksimovic, M., Zhu, L., Shehab, E., Ewers, P. & Kassam, A. 2013 A3 thinking approach to support knowledge-driven design. The International Journal of Advanced Manufacturing Technology 68 (5–8), 13711386.
Saldana, J. 2015 The Coding Manual for Qualitative Researchers. SAGE Publications Ltd.
Simon, H. A. 1973 The structure of ill structured problems. Artificial Intelligence 4 (3/4), 181201.
Tassey, G.2002 The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, RTI Project, 7007(011).
Thomke, S. H. 1998 Managing experimentation in the design of new products. Management Science 44 (6), 743762.
Ullman, D. 2015 The Mechanical Design Process. McGraw–Hill.
Ulrich, K. T. & Scott, A. P. 1993 Does Product Design Really Determine 80% of Manufacturing Cost? Cambridge Massachusetts, Sloan School of Management, MIT Press.
Ulrich, K. T. & Scott, A. P. 1998 Assessing the importance of design through product archaeology. Management Science 44 (3), 352369.
Vadaro, M. J.2013 LeMessurier Stands Tall: A Case Study in Professional Ethics. The AIA Trust.
VDI-Standard1993 VDI 2221 Systematic approach to the development and design of technical systems and products. The Association of German Engineers.
Williard, N., He, W., Hendricks, C. & Pecht, M. 2013 Lessons learned from the 787 Dreamliner issue on lithium–ion battery reliability. Energies 6 (9), 46824695.
Wong, K.2006 What grounded the airbus A380? Retrieved September 30, 2016, from http://www.cadalyst.com/management/what-grounded-airbus-a380-5955.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Design Science
  • ISSN: -
  • EISSN: 2053-4701
  • URL: /core/journals/design-science
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed