TVP – Metric 35 Gate Effectiveness

Background | User Guide | Program Contents | Stakeholders | List of Metrics

1. Metric Definition

The percent of projects in the total portfolio having moved through a defined project management system with defined milestones, or the percent of R&D expenditure on projects using a defined project management system with defined milestones.

a. Metric 1: Quantitative Yield

This metric is designed to measure how effective the design review process is by measuring the yield (passage rate) at each gate or milestone from project initiation to project commercialization. The assumption is that the higher the yield, the more effective the stage-gate or milestone design review process. However, a very high number on this metric indicates that the gate is not working very effectively, i.e., there is no need for a filtering process.
The metric is most accurately used by the percent of the value of R&D projects passing each gate or milestone which meet the criteria for passing the next gate or milestone. The simpler metric of percent of the number of projects may also be used, but runs the risk that it might be excessively influenced by small, easy to understand projects, and therefore misrepresent the true picture.

b. Metric 2: Income Contributors

This metric is the percent that pass through the gate that become significant income contributors. This is a more retrospective number than metric 1, but better reflects what the stage-gate process intends.

c. Metric 3: Costly Failures

Another metric is to document the reasons costly failures passed through the gates and were not screened out. This would identify retrospectively what criteria were missing.

2. Advantages and Limitations

The advantage of these metrics is to calibrate the stage-gate, milestone, and design review process, thus enabling diagnosis of the innovation management process. While metric one is a somewhat retrospective metric, the delay time is only one gate or milestone period long. Thus it is a more prompt feedback metric than overall success or failure in the market place indicated by metrics two and three. Too low a yield at any gate or milestone indicates that too many projects are getting through for further expenditure which do not deserve to have been supported since they did not pass the subsequent gate or milestone.

The limitation is that this is a subjective metric, and needs some historical data within the firm on which to base a benchmark percentage.  The metric for passage of each milestone or stage gate should also be established by the company based on their industry and business strategy.  Too high of a rate of passage across all stages may indicate that not enough revolutionary or step-out ideas are being evaluated in the early stages, while too low may indicate an inefficient innovation process or decision making.

3. How to use the Metric

The only way to use this metric is to keep records for a period of time, and then use them to evaluate trends. This is somewhat of an administrative burden, justified by its use in continuous improvement of the innovation management process.

4. Options and Variations

For the final gate or milestone before commercialization, metrics one and two become the same as the “Percent or value of R&D” metric, which is commercially successful.

5. Champions and Contacts

6. References

Van Remoortere, F., and Cotterman, R. 1993. Project Tracking System Serves as Research Management Tool, Research-Technology Management, 36 (2), March-April, pp. 32- 37. This reference covers milestones in general.

Ellis, L. W. 1984. The Financial Side of Industrial Research Management. New York: Wiley. This reference, on page 105, covers the histogram approach to measuring milestone, gate and project completion.

Patterson, M. L. 1993. Accelerating Innovation: Improving the Process of Product Development. New York: Van Nostrand Reinhold.  This reference introduces the measurement at each originally forecast milestone date of the percent actually completed of the originally estimated work.