TVP – Metric 34 Information Technology Use in R&D

Background | User Guide | Program Contents | Stakeholders | List of Metrics

1. Metric Definition

A measure of the extent of use and the ways in which information technology is used within R&D and the innovation process, for example, by the extent of IT-related expenditures in the R&D budget, ways in which IT is employed, or the extent of IT use in managing R&D.

a. Extent of IT Use in expenditures in the R&D budget.

IT expenditures are a measure of the enhancement of R&D staff effectiveness by the use of information technology. The thrust of the metric is also how to justify this IT spending to higher management, and how to determine an optimum amount. This may be a non-linear effect; while too little use may not match the capability of your competition, too much use may involve R&D staff in activities less optimal than the use of their time directly on R&D. The form of the metric is usually to measure the amount, or ratio of the total R&D expenses of the firm, spent on information technology hardware and software.

b. Ways in which IT is Employed – Impact of IT on R&D

The intent of this metric is to measure how much impact information technology (IT) has on the innovation chain of your company. It is usually measured on a scale of one to four. Level one is when IT is only used to keep track of staff times and costs of R&D activities. In level two, IT is also used to provide tools for technical computing and/or hardware and software for the company’s products and services. In level three, IT also enhances the effectiveness of innovation management by substituting for human effort in R&D. In level four, IT collaborates with R&D to rethink how R&D is done by enabling doing things that can not be done any other way. Where unit scores vary considerably from one unit to another, managers should consider using a median score from all the sub-units.

c. Extent of IT use in Managing R&D

The intent of this metric is to measure how far along one is at using IT in managing the innovation process. Level one is when IT is only used by R&D groups on an individual basis. In level two, IT is integrated between the IT function and a few groups in the R&D activity. In level three, IT is fully integrated in the process of managing technology within the R&D department. Level four is when IT is fully integrated in the process of managing technology with the all department managers from marketing through R&D through manufacturing and operations.

d. Other possible metrics that might be developed

Other possible metrics include breadth of usage by application category vs. benchmarks; quality of usage by application category vs. benchmarks; and percent of R&D staff using IT at benchmark levels.

2. Advantages and Limitations

This metric enables the justification of IT in R&D both at financial budget time, and in benchmarking IT use and impact in R&D against competition and strategic goals. Thus, the stakeholders most interested would be financial officers and managers, and strategic planners.

One limitation is that these are relatively untried metrics, with minimal research support as to their effectiveness. Thus, it is not yet clear how many R&D groups track IT costs. Another limitation is that it may cost more to follow than the benefit derived from it. Information technology is evolving and developing so rapidly that its dimensions are not fully appreciated and some of the measures may be difficult to make.

3. How to use the Metric

The amount of IT expenditure in R&D should be tracked over time, usually the annual budget cycle. The ratings might need to be done with input from IT professionals. The rating can be compared to other measures of R&D effectiveness to see if the change in IT expenses has enhanced effectiveness. If information is available on competitors, this too can be tracked and used to justify future IT budgets.

The ratings of extent of impact and use should also be tracked over a time period for comparison with competition and strategic needs.

4. Options and Variations

Service companies and companies with a high IT content in their operations may wish to use alternatively any of the software effectiveness metrics such as the Capability Maturity Model developed by the Software Engineering Institute at Carnegie-Mellon.

5. Champions and Contacts

6. References

There is an IT project of the R-o-R Committee started in 1995. This is a new area with little published references.

The most general reference is U. S. National Research Council, Information Technology in the Service Sector, Washington, DC: National Academy Press, 1994.

The SEI Capability Maturity Model is reviewed, and compared with Total Quality Management (TQM) and other evaluation techniques, in H. Saiedian & R. Kuzara, “SEI Capability Maturity Model’s Impact on Contractors,” Computer (IEEE), 28(1), Jan., 1995, pp. 16-26. This source has 12 additional references.