Performance benchmarking and measuring program impacts on customers: lessons from the Midwest Manufacturing Technology Center

Abstract

Most attempts to evaluate the impact of manufacturing extension suffer from at least one of three weaknesses: (1) subjectivity of the outcome data, (2) inattention to industry (leading to inappropriate outcome measures), and (3) lack of controls. This article reports on our work on the Michigan Manufacturing Technology Center (MMTC) evaluation effort, and our attempts to redress these weaknesses. Since 1991, the MMTC has established a database of performance metrics, tracking changes over time for both MMTC clients and a non-client control group. These data, coupled with a detailed scheme for characterizing heterogeneous MMTC interventions, provide the raw inputs for an analysis of the impact of MMTC operations. Analysis of the relative change in client performance, from 1990–1992 and from 1991–1993 have demonstrated that MMTC clients are, indeed, taking some actions that lead to faster improvement relative to the control group. But a notable exception is the area of employee computer use: client firms are increasing their use of computers throughout the workforce at a slower rate than the control group. This is of particular concern, as employee computer use is strongly correlated with labor productivity, which is closely related to improved profits and wages. The article outlines our approach and analysis, summarizes our recommendations to the MMTC for program improvements, and reflects briefly on the lessons for manufacturing extension policy more generally.