Application of Lean Methods in Product Development Testing

(1)

Politécnica de Madrid

Politecnico di Milano

Royal Institute of Technology, Stockholm

Title:

Application of Lean

Methods in Product

Development Testing

A Case Study from the

Manufacturing Industry

Master Thesis by:

Daniel Walew

Tutor name:

Case Company: Thesis ID:

Prof. Alberto Portioli Staudacher, Politecnico di Milano

Prof. Sergio Terzi, University of Bergamo MWFL, USA

2013:143

(2)

(3)

I. Abstract

A broad research foundation exists on Lean management in the manufacturing context. Furthermore, the implementation of Lean in product development is discussed by an increasing number of publications. Yet, little documentation of the specific application in product development testing has been published. This master thesis provides insights into the specific environment of product development testing and the application of Lean methods.

Utilizing a systematic literature research, the beginning of this work elaborates principles of case study research, the context of testing as part of product development and the Lean management framework. The findings are synthesized into a priori construct for the case research. Main pillar for this construct is the value stream mapping method. It combines the analysis of the current state, the development of the future state and a strategy for the implementation of improvements.

Central part of this thesis is the in-depth case study of a global operating manufacturing company in the off-highway machinery market. Three product development testing sites were visited by the author in order to apply the previously defined case study framework. Through cross case analysis common process characteristics of the current state are derived. From a micro level perspective the relations within and across the testing process are shown.

Needs, values, wastes, interruptions and other process parameters are systematically analyzed; improvements are elaborated and prioritized to develop a common future state for the testing process. A partial takt time driven test process could be developed. To visualize the process task boards were introduced and the value stream map was digitalized and connected to a management information system.

(4)

II. Acknowledgements

At this point I would like to express my gratitude to all individuals and organizations who have supported my endeavor that led to this thesis.

First and foremost, I would like to thank all people and the three universities: Universidad Politécnica de Madrid (UPM), Madrid, Spain

Instituto Politecnico di Milano (POLIMI), Milan, Italy Royal Institute of Technology (KTH), Stockholm, Sweden

who enable the International Master of Industrial Management (IMIM) program. This master thesis is the concluding part of a two year intense learning experience in a truly international context, in class and among class mates, which was an

invaluable opportunity for me.

I would like to express my gratitude to my academic tutors, Prof. Alberto Portioli Staudacher and Prof. Sergio Terzi, for the support and patience they provided during the elaboration of this thesis. Their professional advices gave me the possibility to balance the academic insights gained from this thesis with the solution orientated company internal work.

I gratefully acknowledge the generous support of the case company. Foremost, I would like to thank the global director of product testing for the trust and excellent mentorship provided. I am deeply indebted to the global planning manager who did not only offer key insights to the case but also arranged an ideal work environment in the United States headquarters of the case company. Additionally I would like to thank all individuals who provided their time for interviews and meetings.

(5)

III.

IV. List of Tables

Table 2-1, Process of Building Theory from Case Study Research and

application in this thesis ... 16

Table 2-2, Data collection methods used in the thesis ... 19

Table 2-3, Interview, Basic Overview ... 20

Table 2-4, Interviewing, possible pitfalls and problems ... 21

Table 3-1, “Fundamental elements of new product development” ... 26

Table 3-2, Stage activities and gate deliverables of a typical Stage-Gate® System ... 28

Table 4-1, Value attributes of product development task ... 47

Table 4-2, Value assessment for product development testing ... 48

Table 4-3, Product Development Waste ... 49

Table 4-4, Waste assessment framework as FMEA approach ... 50

Table 4-5, Major Lean tools used in production environment ... 54

Table 4-6, Lean PD approaches ... 56

Table 7-1, Steps of case study research ... 67

Table 7-2, Possible data matrices by Locher (2008, p.26) and application in the case study ... 69

Table 7-3, Tools for value stream walk through ... 71

Table 7-4, Overview on the future state analysis steps used in this case study work ... 73

Table 8-1, Average demand for new test tasks per day ... 76

Table 8-2, Most time consuming test activities combined from P/T and L/T for FP and FH Platform ... 81

Table 8-3, Summary of information sources across the three visited test sites ... 102

Table 8-4, Identified internal / external customers and the required outputs of the visited product testing sites A and C. ... 102

Table 8-5, Value Stream Process Step Cross Case Overview ... 104

Table 8-6, Matrices of each site in comparison ... 105

Table 9-1, Value aspect of the process steps “Approve test requirement” and “Write test task” ... 107

Table 9-2, Value aspect of the process step “Review Draft Test Task” ... 108

(9)

Table 9-4, Kaizen Ideas collected for the process steps: “Writing TT” and

“Review Draft TT” ... 111 Table 9-5, Required Kaizen to enable constant performance to customer needs. . 112 Table 9-6, High priority waste aspects leading to workflow interruptions ... 116 Table 9-7, List of required Kaizen activities for the future state of MWFL testing

process, part 1 ... 122 Table 9-8, List of required Kaizen activities for the future state of MWFL testing

process, part 2, 48 waste aspect were sorted out due to a priority

index less than 12 as assessed in 9.1.3. ... 124 Table 9-9, Direct time reduction from process improvement ... 125 Table 12-1, Structured observation sheet, page 1, basic information, time data,

input flow ... 148 Table 12-2, Structured observation sheet, page 2, output, critical drivers, rough

(10)

V. List of Figures

Figure 3-1, An overview of the Stage-Gate® System ... 27

Figure 3-2, Product development process ... 28

Figure 3-3, Spiral development - a series of "build-test-feedback-revise" iterations in parallel to a Stage-Gate® process ... 31

Figure 3-4; Spiral model of software development ... 32

Figure 3-5, The basic V-model of the Systems Engineering Process ... 33

Figure 3-6, “Total time variation due to variation in process milestones” ... 37

Figure 3-7, “Theoretical product development cost versus reliability curve” ... 38

Figure 3-8; Life cycle cost methodology flow for testing planning ... 38

Figure 3-9, Relation between design verification ranges ... 41

Figure 4-1, “System Boundaries of the product development system“ ... 45

Figure 4-2, “The Value Creation Process” ... 46

Figure 4-3, 13 Lean PD principles of Morgan and Liker (2006) and number of literature sources with comparable findings ... 55

Figure 6-1, Typical forest harvester machine ... 64

Figure 6-2, Product development process at MWFL ... 65

Figure 6-3, Simplified product development organigram of MWFL and case study focus shaded ... 65

Figure 7-1, Initial process order assessment in VSM Workshop ... 69

Figure 7-2, Process matrices for an individual process step used in the VSM of this thesis ... 70

Figure 8-1, FP Platform - Demand rate for new test tasks per day ... 77

Figure 8-2, FH Platform - Demand rate for new test tasks per day ... 77

Figure 8-3, Boxplot of process time (P/T) of test for the Forest Harvester (FH) Platform in days for coded test types ... 79

Figure 8-4, Calculated contribution to the overall accumulated test time for each coded test type group of the Forest Harvester (FH) Platform ... 79

Figure 8-5, Histogram of process time for the Forest Planter (FP) platform tests ... 80

Figure 8-6; SIPOC framework created during the VSM workshop at Site A ... 82

Figure 8-7; Current State map of product testing site A of MWFL ... 85

Figure 8-8, Current State map of product testing site B of MWFL ... 93

Figure 8-9; SIPOC framework created during the VSM workshop at site C... 95

(11)

Figure 8-11, Task board to follow the test setup ... 100

Figure 9-1, Part of current state VSM of site A with waste highlighted ... 109

Figure 9-2, Waste priority, severity and value of the process steps in the product testing value stream ... 113

Figure 9-3, Value aspects identified and their summarized assessment value... 114

Figure 9-4, Waste aspects identified, summarized in priority index PI ... 114

Figure 9-5, Decoupling point Lab Task List in Product Testing ... 117

Figure 9-6, Product testing operational value stream using takt time and task board ... 119

Figure 9-7, Future state value stream map for MWFL product testing operations .. 123

Figure 9-8, Improvement and Kaizen schedule for test task review ... 127

Figure 9-9, Information areas required for test task definition and review checklist derived from case research, ... 127

Figure 9-10, Results of rough comparison of IT tools available at MWFL, Target: find a system that allows an implementation of shared checklists. ... 128

Figure 9-11, A3 report of the implementation of visual management for test and machine setup... 132

Figure 12-1, Semi structured interview sheet, example ... 147

Figure 12-2, Online Checklist implemented in MS Share Point ... 152

Figure 12-3, Data gathering overview, Number of sources per Checklist Item identified ... 152

(12)

VI. List of Acronyms and Specific Vocabulary

Acronyme Explanation

5S Workplace organization method BOM Bill of material

CHE Chief Engineer

D. Eng. Design Engineers EOQ Economic Order Quantity ETF Economic Test Frequency

EU European Union

FH Forest Havester (Product Platform of MWFL) FMEA Failure Mode and Effect Analysis

FP Forest Planter (Product Platform of MWFL) GES Global Engineering System

GTS Global Test System

KBS Knowledge-based system

L/T Lead Time

LCC Life cycle cost

LTE Leading Test Engineer MS Microsoft Corporation

MWFL Mid Western Forest and Lawn Ltd (Diguised Case Company Name)

NPD New Product Development

NVA Non Value Added5

P/T Process Time

PD Product Development

PDCA Improvement cycle: Plan, Do, Check, Act

PDMA Product Development & Management Association PDVSM Product Development Value Stream Map

PE Program Engineer

PT Product Testing

PT Product Testing (Functional Unit in Case Organization) PT Site Mngt Product Testing Site Manger

SAP ERP System

SIPOC Supplier-Input-Process-Output-Customer framework Takt Time period used to syncronize activities

TDB Test Date Base

TE Test Engineer

Tech Assist Blue Collar Test Technician TP Global Test Planning Manager TPDS Toyota Product Development System TPM Test Project Manager

TPS Toyota Production System

TSM Test Site Manager

TT Test Task

VISIO Microsoft Visio 2013, visualisation software

(13)

1 Introduction

The global competition in manufacturing industries is leading companies to closely examine all areas of business operations. New product development is a major driver of the value a company can provide to the customer. However, the development time and required resources to achieve products fulfilling customer needs are major investments.

To meet the challenges of the increased market competition, the implementation of Lean in product development is discussed by an increasing number of publications. Yet, little documentation of the specific application in product development testing has been published.

Testing is often the first realization of design in a real world product or system and therefore a critical element for a successful product release. Although computer aided engineering analysis has become a powerful tool to meet the product development challenges, physical testing remains an important part of new product development processes with changing requirements.

This master thesis provides insights into the specific environment of physical product development testing and the application of Lean methods. Literature research provides a set of analytical tools to analyze the process and to elaborate improvement strategies. The framework is applied on a case study, involving three product testing sites of a global operating manufacturing company in the off-highway machinery market.

1.1 Background

In various cases early phase product development projects contain design problems leading to mismatch with customer needs, technical errors, problems regarding manufacturing and maintainability of the product (Thomke & Bell 2001). Testing, Verification and Validation, as methods to detect and resolve these problems, have a central role in new product development process (Cooper & Edgett 2008; Ulrich & Eppinger 2004; Wheelwright & Clark 1992b)

(14)

product development process in order to ensure a product that meets customer requirements. The case introduction chapter will clarify the different deliverables of testing and the interrelation with the new product development process.

The thesis work was performed in a context of an industrial manufacturing company operating on a business to business market. Hence, the findings of this thesis have to be distinguished from testing in software development, pharmaceutical development or other non-manufacturing companies.

1.2 Research Objectives and Research Questions

The intention of this thesis is to provide case based insights about the application of Lean management principles in the context of testing as part of new product development.

Various approaches towards process assessment and improvement are discussed in the literature. New product development processes are a heavily researched field of management science due to their strategic importance for a company. For the specific area of product testing, however, the literature base is less dense. The applicability of Lean management principles in testing is underrepresented. In this thesis the author has the objective to provide further evidence on the applicability of Lean management.

1. How can the internal process of testing as part of new product development be assessed in a manufacturing company?

2. To which extend are Lean principles able to improve the performance of a testing process?

3. How can process improvements be part of a continuous improvement effort? The more specific objectives can be stated as following:

- Define the role of testing as part of a new product development process and elaborate measures that define the performance of the testing process.

- Find methods that allow a process analysis and improvement in the context of testing.

- Apply the most suitable method in a company case study, generate results and discuss the findings.

(15)

2 Research Methodology

This thesis is mainly qualitative in its nature. Quantitative data is partly used for the exploratory case study.

The research view of this thesis regards the business processes in product development mainly as information and knowledge exchange between individuals and groups in an international business environment; hence an exploratory, interview-based case study design was found to be a disciplinary convention (Barratt et al. 2011; Piekkari et al. 2009). Some of the most widely-cited accounts for the inductive case study research method (Yin 2003) and (Eisenhard 1989) were followed but needed some adaptation in order to server the predefined single case setting. The main target of the case study is to provide insights into the application of Lean thinking in the testing part of a new product development process.

2.1 Case Study Research Process

(16)

Research Process Eisenhard

Research process

in this Thesis Methods Used Chapter 1 Getting Started Definition of research question Possibly a priori construct Background and initial framework Research Question Literature Review VSM Guideline 3 4 5

2 Selecting Case Predefined Single Case, - - 3 Crafting Instruments and protocols Multiple Data Collection Methods Qualitative/Quantitative - VSM as Framework - Interview structure - Observation structure Literature Review Interview Legacy Data Screening 7

4 Entering the Field Overlap data collection and analysis Flexible and Opportunistic data collection 2 VSM Workshops Semi Structured Interview/ Observation VSM Interview Observation 8.2 8.3 8.4 5 Analysing of Data Within-case analysis Cross Case pattern search

Within Case Analysis Interview

Quantitative data review 8.5 6 Shaping Hypotheses Iterative tabulation of evidence Replication, not

sampling, logic across cases

Search evidence for “why” behind

relationships

Future state definition for the case company Implementation plan for future state

Application of Lean principles on future state of testing. Value, Waste Assessment and Prioritization 9 7 Enfolding Literature Comparison with conflicting literature Comparison with similar literature Answering and discussion of research question Case Review, Literature review 10.1

8 Reaching Closure Contribution, Limitation and Further Research

(17)

Following a more detailed description of the individual steps:

1. Getting Started and 2. Selecting Case

Before the beginning of this thesis work, the research topic was specified by a project definition in the case study company. For that reason the case selection of step 2 was pushed forward. This phenomenon has been reported as being a common issue

for young researchers and could also be called Step 1: access negotiation (Tan 2011). It has to be admitted that the limitation to a single case, clearly reduces

the possibility of building theory from the case study. Nevertheless, the structured process of case study research was chosen, to allow comparison with similar research in other cases.

To achieve a foundation for the formulation of a research question a basic investigation of the topic background, the fundamental aspects of process implication in the case company and relevant boundaries were worked out. The Lean product development framework was found to be an applicable priori construct in order to focus attention and structure the gained knowledge. Also Value Stream Mapping (VSM) provided an initial starting point. However, with the given constraints in the case company adaptations to the framework were needed, which were partly contradicting the Lean product development framework. As a result of this initial learning process, the research questions were updated after the literature review in chapter 5.

3 Crafting Instruments and Protocols

As stated by (Eisenhard 1989) typically multiple combined case data collection methods are used. A disciplinary convention is that interviews are the very dominant source of data collection in case studies published in international business- and information systems journals (Dube´ & Pare´ 2003; Piekkari et al. 2009). Hence, interviews were selected to be a major pillar of the conducted research combined with observations. Eisenhard (1989), however, pointed out the “highly synergetic” combination of qualitative and quantitative data sources. For that reason and to serve the specific research question “To which extend are Lean principles able to improve

the performance of a testing process?” legacy data was also analyzed.

(18)

4 Entering the Field

“A striking feature of research to build theory from case studies is the frequent overlap of data analysis with data collection. “(Eisenhard 1989) The selected VSM method gave the possibility to assess a basic framework of the process that was under investigation from a management perspective, followed by specific semi- structured interviews and observations of relevant process activities and participants in order to validate and improve framework. It became clear that certain process steps had different characteristics depending on the point of view of the interviewee. In these situations the VSM method allowed quick adaptation and emphasis on critical aspects without losing the connection to a broader view. The possibility to make adjustment during the data collection process turned out to be a key feature of the field work as mentioned by (Eisenhard 1989). A daily log of the field work was maintained to support the subsequent data analysis.

5 Analyzing of Data

The first step of the within case data analysis was a write-up of the interview and observations performed. In parallel the different draft versions of value stream maps, which were created during the field work in paper and pencil technique, were merged into an overall value stream map. Especially qualitative information on process flow from the interviews was integrated into the value stream map. When observations of complete process steps were available, specific matrices could be measured. However, the low number of samples did not allow validating all relevant matrices of the value stream map in general. A second and third field visit was made to broaden the case understanding.

6 Shaping Hypotheses

(19)

Main focus of this thesis is answering the research questions through case study research. The possibility for generalized hypothesis shaping and verification is limited by the mainly qualitative data, the limitation on one case company and the five month duration of the complete thesis project. Cross case comparison or long term longitudinal study should be addressed in following work. The in-depth insight the presented thesis offers on the product testing environment builds a foundation for further research.

7 Enfolding Literature

“An essential feature of theory building is comparison of the emergent concepts, theory, or hypotheses with the extant literature. This involves asking what is this similar to, what does it contradict, and why.”(Eisenhard 1989, p.544)

After the actual case study research the research questions are answered. Each question is discussed considering supporting and contradicting literature where applicable.

8 Reaching Closure

A major limitation for the thesis work was the limited time range of 5 months. So the step of iteration between data analysis and theory building was given by external boundary. However, the target of the thesis was to reach a level of acceptable theoretical saturation. Since the definition of “acceptable“ is not exact a detailed review of limitation and suggestions for further research is included in this thesis.

2.2 Data Collection

During the different phases of the work the data collection methods and tools were selected depending on the applicability, availability of data and the highest significance of results. The following Table 2-2 gives an overview on the data collection methods applied in the different parts of the presented master thesis.

Chapter 2 3 4 5 6 7 8 9 10

Literature Review x x x x x x x x

Interview x x x

Observation x

Data Analysis x x

(20)

2.2.1 Literature review

Major parts of this thesis are derived from intensive literature research. Literature was used to provide a basic foundation for the research and to perform derived results with the extant literature. In addition, the academic methods used were derived from literature review in order to align with the rigor of case study research. The literature was initially searched by key words in Google Scholar. Further related sources where either taken from the resource section of sighted articles or by following related reading suggestions of the journal publisher.

Sources of literature search were:

- POLIMI, KTH and UPM libraries by using their online catalogs of journals - Google Scholar

- Elsevier, Sage, JSTOR and other journal publishers as required 2.2.2 Interviews

The central importance of interviews for successful case study research was shown by a review of leading operations management and international business journals (Barratt et al. 2011; Piekkari et al. 2009). The basic information about the interviews conducted for this work can be found in Table 2-3.

Number of Interviews: 30 (during site visits: 21, during Preparation: 7) Who conducted the Interview: The author

Who was interviewed: Test management local and global, Test planning manager, leading test engineers, design

engineers, test project manager, test engineers, test technicians

Type of Interview Semi structured Table 2-3, Interview, Basic Overview

(21)

Artiﬁciality of the interview

Interrogating with a stranger under time pressure is an artificial situation that the researcher should be aware of.

Lack of trust

Information that is considered “sensitive” by the interviewee might not be disclosed. In this case the data gathering remains incomplete.

Lack of time

If the time is not sufficient to gather all relevant information, the interview remains incomplete. A further effect can be that the interviewee gets into a stressful situation under which opinions are made up. In this case the data gathered is not reliable.

Level of entry

This pitfall warns the researcher about the difficulty to conduct interviews with senior managers if the company is entered in a lower level.

Elite bias

By interviewing solely high level employees the researcher might fail to gather data of the broader situation (Miles & Huberman 1994). In addition, the risk is given to overweight data from high status, well-informed interviewees and to undervalue data from less articulated, lower status ones (Heiskanen & Newman 1997). Hawthorne effects

A change of the object of investigation can happen due to the presence of the researcher. The intrusive character of qualitative interviews can lead to interference with peoples’ behavior (Fontana 2000).

Constructing knowledge

If an interviewer triggers reflection on the side of the interviewee, by asking specific but open formulated questions about issues that were never actively considered by the interviewee, knowledge is actively constructed. Although reflection is a

declared target for the interview, a risk is given that interviewees construct logical and consistent, but unrealistic stories, to appear knowledgeable and rational. Ambiguity of language

‘‘Asking questions and getting answers is a much harder task that it may seem at ﬁrst. The spoken or written word has always a residue of ambiguity, no matter how carefully we word the questions or how carefully we report or code the answers’’(Fontana 2000, p.645).

Interviews can go wrong

Situations that lead to one of the named problems and pitfalls in very strong way or even end up with emotional conflict situations might be abandoned.

Table 2-4, Interviewing, possible pitfalls and problems, (Myers & Newman 2007; Webb et al. 1966)

(22)

For that reason, the interviews conducted for this thesis were designed according a seven ‘Guidelines’ recommendation for qualitative interviewing by (Myers & Newman 2007). In following Myers and Newman’s interview guidelines (in italic) are compared to the actual application in this thesis.

1. Situating the researcher as actor

In this part of the framework the interviewer “situates” him selves in a social context to the interviewee.

The author of the presented thesis conducted the interviews as a sole researcher, however, with specific management assignment to investigate process reality in operations. As such, he was an unknown colleague from the headquarters of the case company, not a consultant. Being a graduate student with specific work experience in the industrial field of the case company, was another characteristic of the researcher. This is how the researcher was presented to the group during a morning briefing at the first day. An unstated, but noticeable aspect of the role was being male, around late 20’s, foreigner of German origin.

2. Minimize social dissonance

The interviewer has to avoid all aspects of social dissonance by managing first impression, following a dress code and using the appropriate language and terminology.

(23)

3. Represent various “voices”

This point is also known as “triangulation of subjects” and an important consideration to avoid elite bias. Informants can be differentiated as guides, stars or gatekeepers.

During the field work an initial value stream mapping event was held by the author. Together with a manager the different process steps were summarized, arranged according to the workflow and matrices defined. Having this VSM as guidance allowed organization of further interviews with the responsible process owners. The interviews were conducted with the site management, technical experts, lab engineers and technicians, with particular attention to process owners from connected process steps. That way, statements about information flow could be triangulated from a sender and receiver perspective.

4. Everyone is an interpreter

Mayer and Newman used this guideline to create awareness about the interpretive nature of interviews from the side of the researcher, the interviewee and the audience reading the results.

In this aspect the lack of recording possibilities has to be mentioned, which was not allowed due to the behavioral code of conduct. However, although the error of data gathering due to interpretation increases, a recording would have increased the social dissonance error. By using semi structured interviews and notes the data quality could be improved. In any case, an interview team with a role for asking questions and observing behavior and a role to take notes would have been first choice. (See: 7.1.1, page: 71 for further detail)

5. Use Mirroring in questions and answers

To avoid the impact of the interviewer by asking leading questions or other pitfalls related to the question formulating, (Myers & Newman 2007) suggests using certain questioning frameworks.

(24)

6. Flexibility

The method of semi-structured and unstructured interviewing requires flexibility and improvisation in order to follow the most interesting lines of information. Surprises should be expected.

For the interviews of this research a semi-structured approach was followed. A question sheet was prepared (12.1) with questions that were formulated according to the information required to complete the current state VSM. The sheet, however, was equipped with space for additional notes. The flexible adaptation of the sheet made it possible to follow only questions of areas where the interviewee was knowledgeable. It turned out to be an invaluable advantage for the field work to have a prepared question sheet with the flexibility of adaptation, because it enabled a large number of interviews often triggered by the advice of a previous informant without losing the focus of the research. Marschan and Piekkari (2004) referred to this effect as “snow-ball” sampling.

7. Confidentiality of disclosure

Researcher transcripts, record and technology have to be kept confidential and secure.

(25)

3 New Product Development and the Role of Testing

3.1 Product Development Process

Before selecting and applying methods of process analysis and improvement in this thesis, a definition of a product development system is derived in this chapter. The chapter does not intend to provide a holistic overview on existing product development process concepts, but defines the context in which the main area of this research is set.

Several definitions of new product development (NPD) are present in the academic literature and the practitioner context of the industry.

For the Product Development & Management Association (PDMA) a NPD process is a set of tasks and steps that describe the repetitive conversion of ideas into salable products or services (PDMA 2006).

An only slightly more specific definition for new product development is given by Ulrich and Eppinger who underline the starting point of NPD as ‘perception of market opportunity’ and the end as ‘production, sale and delivery of a product’

(Ulrich & Eppinger 2004).

Oppenheimer addressed the wide definition space by writing: “Product Development (PD) is a broad term that includes all conceivable tasks involved in the design of technology-based objects or missions which provide value to the product stakeholders”(Oppenheim 2004, p.353).

In fact, the stream of articles on NPD published in leading journals has grown in absolute number and percentage from 1989 to 2004 (Page & Schirr 2008). Although a review of 815 scientific articles could identify “encouraging signs of a maturing discipline”(Page & Schirr 2008, p.232), growing sophistication of NPD models and changing market environment make NPD research still highly valid.

Loch and Kavadias underline the evolutionary character of new product development by stating that “no ‘theory of NPD’ exists, and there is no consensus on whether one can and should exist.”(Loch & Kavadias 2008, p.1)

(26)

processes evaluates the concept in terms of market opportunity and customer needs. A selection process compares different concepts and choses the most promising. After the selection a detailed design phase begins which has to ensure that product or services can be offered to customers. Table 3-1 lists the fundamental NPD process phases as identified by Loch and Kavadias (2008).

Fundamental elements of new product development A variant

generation process

Identifies new combinations of technologies,

processes, and market opportunities with the potential to create economic value.

Variants are generated by directed search and ‘blind’ combination of unrelated elements (creativity)

A selection process Chooses the most promising among the new combinations for further investment (of financial,

managerial, physical, and/or human resources) according to consistent criteria.

A transformation process

Converts (‘develops’) opportunities into

economic goods and codified knowledge (embodied in a design) – products or services to be offered to

customers.

A coordination process Ensures the information flow, collaboration,

and cooperation among multiple parties, involved in the NPD activities

Table 3-1, “Fundamental elements of new product development”(Loch & Kavadias 2008, p.4)

The possibility to distinguish different phases of a product development made stage models of NPD early a highly researched topic. However, by reviewing 815 journal articles between 1989 and 2004 Page and Schirr found that stage process research has decreased and research streams like “Strategy”, “Integration”, “Teams” and “Radical” became top four research streams (Page & Schirr 2008, p.243).

Although the research interest in ‘stage-gate’ product development processes has decreased they still have a primary role in the practical application, often referred to as ‘Phased Project Planning’, ‘Gate Systems’, ‘Stage-Gate Systems’ or ‘Phase-Gate Systems’. The process might also use company specific terminology.

(27)

Figure 3-1, An overview of the Stage-Gate® System (Cooper 1990)

Since multiple projects of new product development can pass the staged process in parallel as part of an overlapping or concurrent project design (PMI 2011), the individual project leader has to ensure that activities are performed during the stages and has to present specified deliverables during the gate screening. In the following Table 3-2 the major activities of each stage and gate as suggested by Cooper 1990 are summarized.

Stage or Gate Activities and Deliverables Idea New product idea.

Gate 1 Initial screen: first decision to commit resources to the project Also considered are ‘must meet’ and ‘should meet’

criteria like strategic alignment, project feasibility, magnitude of the opportunity, differential advantage, synergy with the firm’s core

business and resources, and market attractiveness. Financial criteria are not measured.

Stage 1 Preliminary assessment to determine: - Project’s technical facts

o development and manufacturing feasibility o cost and time of executing

- Project’s market facts

o market size, market potential, likely market acceptance

(28)

Stage or Gate Activities and Deliverables

Stage 2 Definition: still a stage prior to product development in which: - Detailed project definition

- Market and customer ‘s needs are identified and transferred to technical and economical values

- Manufacturability, costs- and investment required are investigated

- A detailed financial analysis is conducted Are elaborated.

Gate 3 Decision on Business Case:

Results of financial analysis are assessed with ‘must meet’ criteria due to the heavy spending in the following stage. Further decisions concern the product concept, the product features, attributes and specifications, the preliminary operations and marketing plans.

Stage 3 Development: development of the product and detailed testing, marketing and operations plans.

Gate 4 Post-Development Review: check of the product characteristics, quality of development work and financial review, based on new and more detailed data.

Stage 4 Validation: test variability of the project considering product, production process, customer acceptance and economic aspects. Gate 5 Pre-Commercialization Decision:

Final point where a project could be killed, Results of previous stage and financial projections are reviewed.

Stage 5 Commercialization: execution of operations- and marketing launch plans

Post-Implementation Review

A review of the project ends the Stage-gate model. Strengths and weaknesses of the project are highlighted, if possible learning is implemented.

Table 3-2, Stage activities and gate deliverables of a typical Stage-Gate® System (Cooper 1990)

Not all stages are required to be performed. Maylor reduces Coopers model to a three stages and four gate process (Maylor 2010). Ulrich and Eppinger presented a new NPD process consisting of six phases which include feedback processes within each phase rather than a gate screening (Ulrich & Eppinger 2004).

Figure 3-2, Product development process (Ulrich & Eppinger 2004)

(29)

Providing a holistic picture on these would exceed the research intend of this thesis. However, each of these processes tries to provide a strict model to manage risk and increase efficiency (Veryzer 1998).

The importance of a clearly defined and implemented NPD process gets apparent when group- or organizational boundaries are crossed. The typical NPD process differences like timing and number of stages or gates, combined with an anticipation that other groups would understand and accommodate the own structure, was found to be a major sources of delay during a design projects (Yassine et al. 2003).

The problem Yassine et al. investigated fits into the more general NPD research stream of “Teams-Integration” which was identified as top research stream across 815 articles selected from ten leading marketing management, NPD and R&D journals. Other major streams of NPD research are “External Alliances”, “New Product Development Strategy”, “Development Speed”, “Radical Products”, “Ideation and Creativity”, “Success-Failure Factors” and “Stage Process” (Page & Schirr 2008, pp.232, 241). To give an overview on this range of research is far beyond the focus of this thesis.

However, according to a meta-analysis Henard and Szymanski performed across 60 empirical studies of NPD in 2001 and confirmed by Page and Schirr 2008, more examination of product characteristics, marketplace characteristics, strategy synergy and product quality is required in order to predict new product performance more accurately (Henard & Szymanski 2001; Page & Schirr 2008).

Likewise, the main deliverables of product testing are examination of product characteristics and ensuring product quality during a NPD. The testing activities during NPD do also contribute to the speed of a development and require interaction between different groups. Hence, the topic of this thesis work on testing as part of a new product development process has a strong mandate to be investigated.

(30)

3.2 Product Testing as Part of New Product Development

After the previous introduction into new product development the following section will lead to the scope of this work, the product testing as a part of a new product development.

In this work the term ‘testing’ is used as a general description of activities performed to verify and validate products and as functional unit within the new product development organization. For validation and verification the following definitions were applied:

Validation: “The assurance that a product, service, or system meets the needs of

the customer and other identiﬁed stakeholders. It often involves acceptance and suitability with external customers. Contrast with veriﬁcation.”(PMI 2011, p.452)

Veriﬁcation: “The evaluation of whether or not a product, service, or system

complies with a regulation, requirement, speciﬁcation, or imposed condition. It is often an internal process. Contrast with validation.” (PMI 2011, p.452)

As discussed in the previous chapter an essential phase of most product development processes is testing. Being often the first realization of design in real world product or system, makes the testing phase a critical element for a successful product release. A central problem of NPD management is the timing and frequency of conducting tests. Early studies have shown that the effort related to testing can contribute almost fifty percent to the overall development effort (Shooman 1983). The high cost and resources related to testing, especially if physical prototypes are involved, leads to the ‘not unusual’ scenario of receiving prototypes late in the program and single big “killer” testing (Reinertsen 1997).

A major value of testing is defined by its inherent function to reduce technical uncertainty combined with the serious economic impact late problem discovery can have during a product development (Thomke & Bell 2001).

(31)

To ensure the delivery of functional requirements in a specified duration, predefined testing strategies are common industrial practice. The testing strategy has to define the number and kind of required prototypes as well as the scheduling of test with each of them (Zakarian 2010).

In an environment where product requirements are changing, designs become more complex and the available development time is reduced, performing and managing the required testing becomes more challenging. Thomke and Bell 2001 pointed out that “the academic literature provides some help in formulating optimal testing strategies”(Thomke & Bell 2001, p.308). They referred to solely two sources and hence argued that further research of test strategy and planning would be required. Khalaf and Yang (2006) as well argued that rich scientific literature on the overall product development process is available. Planning and execution of downstream product optimization and testing, however, is treated less frequent in the literature (Khalaf & Yang 2006). The planning aspect of NPD testing became a more investigated research stream in the last years and will be presented in chapter 3.3.1 of this thesis. The managerial aspect of test execution instead remained less treated and will be focus of the case investigation.

The emerging focus on test planning and execution as part of a NPD process can also be seen at Coopers work, founder of the stage-gate® process (Cooper 1990). He observed that iterative testing became a key factor of success for development teams. Handling unstable product specifications and project scope creep, Cooper and Edgett documented a spiral development process in parallel to a stage-gate® process. (Cooper & Edgett 2008).

(32)

The Spiral development process of Cooper and Edgett shows similarities with the regular schedule of prototype builds called “periodic prototyping” proposed by Wheelwright and Clark(1992).

Even earlier concepts of spiral development exist. Boehm (1988) described an iterative waterfall process of software development in which each step consists of:

 Determining objectives

 Identification and resolve risk

 Development and test

 Plan the next iteration

Through each iteration of the development steps the product is further developed and enhanced which creates the spiral of the model. Figure 3-4 shows the phases of Boehms development model. The strategic importance of test planning and execution (also referred to as validation and verification) were fundamental part of Boehms model.

(33)

A further popular model of product development and systems engineering is the V-Model (Clark 2009; S. Eppinger & Browning 2012). As well as Boehms spiral model it is often referred to as a software development process model. It found, however, also application as visualization for development project in general (Forsberg et al. 2005).

Figure 3-5, The basic V-model of the Systems Engineering Process (Forsberg & Mooz 1991; Forsberg et al. 2005)

A top-down development process described by the V-Model starts in the upper left and ends in the upper right. A decomposition and definition phase starts with the understanding of user requirements, a basic system or product concept and a testing plan, allowing the measurement of performance. Steps of specification, design and build, lead to the fabrication, assembly and software coding of a prototype (Forsberg & Mooz 1991). The V shape of the model allows visualizing the relation between development steps and verification steps.

Forsberg et al. underline the need to have a planned process of verification by stating: “Many very expensive systems have failed after deployment due to built-in errors. In every case, there were two failures. First the failure to build the system correctly and second the failure of the verification process to detect the defect.” (Forsberg et al. 2005, p.366)

(34)

Various adaptation of the V-Model have been made to match specific needs of development project management (FHWA 2009; IABG 2009). The central role of verification and validation testing in the V-Model are similar to the approaches of Boehm (1988) and Cooper (2008). The execution of iterative software development summarized Boehm (2002) as a contrast between low planning agile methods and detail planned milestone plan-driven methods.

The agile methods were summarized in a shared value proposition called the “Manifesto for Agile Software Development” (Beck et al. 2001) as:

- individuals and interactions over processes and tools, - working software over comprehensive documentation, - customer collaboration over contract negotiation, - responding to change over following a plan.

Agile development was a reaction on corporate bureaucracy and other dis-advantages of detailed plan driven development. The practitioner driven methods of Extreme Programming, Scrum and others could be summarized under the name of agile software development (Boehm 2002; Highsmith & Chckburn 2001). It would exceed the frame of this thesis to go into details, but the agile methods are an important pillar of today’s software development projects (Estler et al. 2012).

Several applications of agile methods in non-software related product development have been documented. The knowledge base, however, is much less developed as for software development (Carlson & Turner 2013; Mazzani 2012).

Thomke and Bell (2001, p.309) underlined that although “the empirical evidence suggests that early and frequent testing is desirable in some projects, it will certainly not be the most effective strategy for all projects and for all design problems.” A statement that summarizes the variability of context in which testing can be part of a new product development process.

(35)

3.3 Major Consideration for Product Testing Operations

This section aims to give an overview on the different aspect of product testing relevant for the case study research. As structure for the research the activities of a testing and refinement phase according to (Ulrich & Eppinger 2004) were chosen: reliability- and life testing, performance- and functional testing, obtaining regulatory approvals, implementation of design changes. The aspect of test planning and execution was added to provide a connection to the NPD process discussed in the previous chapter.

Focus was put on research from manufacturing industry perspective. Pharmaceuticals development, pure software development, service industry or other non-related field of research were excluded from the review. However, it should be noted that further insights could be expected even from these industries, especially through the organization of services and the interaction within expert organizations, which could be addressed by further studies.

3.3.1 Planning of Product Testing

Chapter 3.2 identified verification and testing as essential phases or stage in various product development process models. The planning effort, however, was not particularly mentioned by various authors in the past (Cooper 1990; Pahl & Beitz 1996; Ulrich & Eppinger 2004; Wheelwright & Clark 1992a). Whereas the effort to manufacturing planning was given a high focus as: “Assess production feasibility”, “Define final assembly scheme”, “Begin procurement of long lead tooling” (Ulrich & Eppinger 2004). As discussed in chapter 3.1 some authors referred to this lack of research (Khalaf & Yang 2006; Thomke & Bell 2001). The NPD models: spiral development, V-Model and agile with a higher focus on test planning and execution were also discussed in chapter 3.1.

(36)

Thomke and Bell (2001) have proposed a model to define optimal test strategies. They incorporated several variables like increasing cost of redesign, the cost of test as function of fidelity and the sequential test correlations. Similar to the model of economic order quantity (EOQ) Thomke and Bell (2001) developed a model to define the optimal number of tests, called Economic Testing Frequency or ETF (Equation 1).

Equation 1, Economic Test Frequency (Thomke & Bell 2001)

According to Thomke and Bell the ETF model provides a “robust solution to the question of how many tests to order. The only situation in which more tests are in fact economical is when the fixed cost component of a test is low and the tests partially overlap.” (Thomke & Bell 2001, p.320) Focus of the work by Thomke and Bell was the optimization of test strategies from economic perspective. They do not consider the managerial aspect of testing execution. However, Thomke and Bell see interesting further research on how testing impacts coordination and communication within companies.

A micro level approach, based on design for six sigma, axiomatic design, Failure Mode and Effect Analysis (FMEA) and robust design was presented by Khalaf & Yang to gain a “Lean approach to validating a product” (Khalaf & Yang 2006, p.18). In a resource constrained case their objective was the reduction of completion time. By developing a mathematical model and conducting different optimization approaches Khalaf & Yang concluded that late in program design and manufacturing changes should be avoided. Furthermore they suggest a test planning in a sequence that allows studying “families of failure modes” in contrast to ‘ad hoc’ testing

(Khalaf & Yang 2006).

Among other aspects Khalaf & Yang worked out the role of variation in a testing process, the within variation of each time contributing factor of testing and the variation of the milestones in a NPD program (see: Figure 3-6).

An in-depth analysis of reasons for variation in product testing execution was not provided.

√

ETF : Economic Test Frequency n* : Number of tests

dv/2 : Avoidable costs

m(f) : Cost of test as function of test fidelity f : Fidelity of test measures the fraction

(37)

Figure 3-6, “Total time variation due to variation in process milestones” (Khalaf & Yang 2006, p.22) Zakarian attested in 2010 that “companies in a variety of industries that develop complex products have well-defined processes for developing product testing plans (to determine the number and types of prototypes needed for new product development) and test plans (to determine how tests are scheduled on each prototype)” (Zakarian 2010, p.370) The large number of requirements combined with the high cost of prototypes and testing resources require test plans to balance the tradeoffs between cost, testing lead-time and requirements validated. Zakarian underlines the uncertainty involved in product validation and testing, impacting test execution times and availability of prototypes most heavily (Zakarian 2010). He developed an analytical bases approach for analysis and optimization of test schedules, providing manager and engineers with a tool to examine critical tradeoffs. The work of Zakarian provides a method for uncertainty driven test planning to:

- Evaluate tradeoff between: number of prototypes, percentage of completed test procedures and reward gained from testing

(38)

Kleyner and Sandborn (2008) applied an analysis of Life cycle cost (LCC) to guide the tradeoff during test planning. They compare warranty and service cost with the cost of development to minimize the total LCC. (Figure 3-7)

Figure 3-7, “Theoretical product development cost versus reliability curve” (Kleyner & Sandborn 2008, p.798).

Validating requirements can improve the quality of the product, if the information about problems is fed back into the design process. It also allows a more accurate reliability prediction. Both with a descending effect on the warranty cost curve. The development costs, however, are ascending due to the test equipment cost of

ownership, the test duration and the cost for the prototype samples (Kleyner & Sandborn 2008).

Figure 3-8; Life cycle cost methodology flow for testing planning (Kleyner & Sandborn 2008, p.800)

(39)

A further method considering an overlapped design processes of test planning has been discusses in the literature. In this method downstream activities are started before upstream activities are completed in order to shorten the overall development time (Krishnan et al. 1998; Qian et al. 2010; Tahera et al. 2013). Based on a model Krishnan et al. (1998) showed the tradeoff between upstream evolution and downstream sensitivity. In this model a fast evolution describes an activity that reaches the final values early in the process, whilst slow evolution provides late final values. A low sensitivity describes a downstream process that can cope with upstream changes easily, in other words is flexible. A high sensitivity, in contrast, is characteristic for activities that require high amount of rework in case of upstream changes (Krishnan et al. 1998). As analytical result Krishnan et al. conclude that low sensitivity and fast evolution activities are well suited for overlapped planning.

Tahera et al. pointed out that “it is observed that engineering companies overlap testing and design as essential practice; regardless of the situation with respect to evolution, sensitivities and resolution” (Tahera et al. 2013, p.202). The impact on communication and workflow has to be well understood.

The literature research on the topic of test planning and execution has shown several different approaches. A focus has been found on the area of planning and integration into a design process. Several modeling efforts are focused on finding optimal solutions considering different boundaries and parameters. The lead time and variability of test execution and the related internal process, however, are viewed as given constraints.

3.3.2 Basic Overview on Different Test Types

The following chapter will give a basic overview on different test types in order to show the portfolio of different activities required in product development testing of a manufacturing company.

Reliability testing and life testing

Reliability verification has to prove that a design will meet specified requirements over time, hence, failure rate or frequency of repair have to stay under acceptable values. A proven mean time between failure (MTBF) are verification results.

(40)

In cases where real time replication testing would exceed time or cost boundaries

acceleration can be achieved by increasing load level or frequency (Forsberg et al. 2005). The acceleration of life testing requires an in-depth knowledge

about load and stress factors for the design on the one hand, but also knowledge about expected failure modes. In particular designs that suffer erosion, chemical or electrical reactions, vibration, static and thermal stress in combination, life testing provides highly valuable knowledge about critical failure modes. Klyatis (2012) refers to the complexity of proper reliability testing as a source for inaccurate prediction of product reliability resulting in recalls and high life cycle cost. He argues that accelerated test approaches like vibration testing, corrosion testing, step-stress testing, thermo shock testing, highly accelerated life testing (HALT), highly accelerated stress screening (HASS), accelerated aging, mechanical crack propagation and growth testing and environmental stress screening (ESS) isolate only few influencing failure factors, hence, do not account for complicated interactions in products and systems. In his book Klyatis presents a framework for accelerated reliably testing (ART) and accelerated durability testing (ADT), that combines various types of accelerated tests together with human and safety factors in a systematic way (Klyatis 2012). The focus is to identify and quantify problems in order to allow design engineers to work on solutions.

Also the Lean literature refers to the learning about failure or testing to failure as the major value adding compared to testing to specification (Millard 2001). “A Lean design engineer wants to test to failure because more can be learned through failure.” (Locher 2008, p.51)

Although reliability and life testing are major contributors to the time and budget of an overall product testing, they are not sorely able to ensure customer satisfaction, a working system and fulfilling of regulatory requirements. Hence, further testing methods are discussed in the following paragraphs.

Performance and functional testing

(41)

performs entirely according to a specification, meaning that all positive aspects and events of the specification are fulfilled and negative events are absent. Functional testing is also referred as verification in general (Forsberg et al. 2005). In case of not fulfilling requirements design corrective actions have to be addressed.

It is important to notice that design verification and performance testing are usually performed with prototype products under nominal conditions in preselected environments. However, the impact of variability has to be considered resulting in design margin verification. Figure 3-9 shows that the resulting proven margin of the design has to exceed the quality verification range during production and the expected operational range during the life cycle of the product (Forsberg et al. 2005).

Figure 3-9, Relation between design verification ranges (Forsberg et al. 2005, p.370)

An important aspect of performance and functional testing of complete products is the role of the tester itself. Aspects of the design that might not have been fully specified as customer value might cause dissatisfaction from a customer point of view. Although traditionally the proof of customer satisfaction or testing appears at

the end of development projects, more recent development process relay on in-process testing (Forsberg et al. 2005). Hence, the phase of functional and

performance testing should be also considered aspects of user satisfaction regardless of whether they have been specified or not.

(42)

Obtaining regulatory approvals

Regulatory approvals are often referred to as certification, compliance or homologation testing. A signed certificate has to prove that one or several standards are met (Forsberg et al. 2005). An example is the certification according to Conformité Européenne (CE), which indicates a product’s compliance with EU legislation and is required to sell a product on the European market. By performing checks and tests the manufacturer has to ensure the conformity of the product (EUCommission 2011).

Virtual Prototyping and Simulation Model Validation

A major effort to reduce the product development time and to allow fast learning cycles for the development of more sophisticated and higher performance designs and systems is the development of simulation models and virtual prototypes (Thomke & Bell 2001). The models have to return the ability to evaluate different designs and scenarios rapidly, hence, to allow optimization and robustness studies in a high pace (Kokkolaras et al. 2013; Tahera et al. 2013).

The presence of uncertainty in both simulation models and test data requires a model validation as basis of product testing (Kokkolaras et al. 2013). Out of a wide design space with a large number of input variables a systematic model validation can help to reduce the total number of require test data, hence safes cost and test time. Although such a model validation does not necessarily create or specify customer value and could be considered as waste in Lean framework (Bauch 2004; Locher 2008; McManus 2005) the risk reducing aspect and the increased confidence in resulting designs represents an enabling value (Chase 2001; Kato 2005).

Application of Lean Methods in Product Development Testing

Application of Lean

Methods in Product

Development Testing

A Case Study from the

Manufacturing Industry

Daniel Walew

I.

Abstract

II.

Acknowledgements

III.

Table of Contents

IV.

List of Tables

V.

List of Figures

VI.

List of Acronyms and Specific Vocabulary

1 Introduction

2 Research Methodology

3 New Product Development and the Role of Testing