A new methodology for aggregating tables

Abstract

Often successive studies are conducted to rank items across time or across different raters. For instance, different consumers may be asked to rank products, or banks may be ranked at different points of time based on their lending practices. In such cases, a simple model would suggest arriving at a summary set of rankings by averaging across the sets. However, when the rank of an item is missing in some of the sets, a simple mean can produce a biased result. This is because the absence of an item in a particular set may be important information that is being ignored.MacMillan conducted a series of studies to identify a forum for publishing business policy research, the most recent of which appeared in this journal. The design of the study involved two stages of selection: first, the journals were to be voted into a set, and second, they were to be ranked by a panel of scholars. This procedure resulted in some of the journals not appearing in every study. Thus, taking a simple average across the studies would result in misleading information, because the absence of a journal in a particular study is information that is important (because the panel chose not to vote it into consideration) but lost in averaging.One of the objectives of the MacMillan studies was to provide scholars with information about the importance of the journal as a publication outlet. Thus, it is important to be able to arrive at a summary set of rankings to provide cumulative information. To achieve this, a left-censored model was developed. Assuming that the ranks follow a normal distribution, with unknown mean and the same variance, maximum likelihood estimates of the means were calculated.. The summary set of ranks were then calculated using these estimates. The summary ranks were compared with those arrived at from an independent estimate of journal quality and found to have validity.Our model helps enhance the value of the MacMillan studies by providing cumulative information on the journals and facilitating their comparison with other studies. We see the applicability of the model in other areas too. Known by the generic term of league tables, such sets of data are generated in marketing research, educational studies, etc. Thus, the model is an important contribution to the tool kit of empirical research.