Citation Indexes for Evaluation of Research Productivity
Submitted to Harvard Business Review . August 18, 1964.
HARVARD BUSINESS REVIEW
Soldiers Field
Boston, Mass 02163
Dear Mr. Bursk:
I read Hodge's article on rating company research productivity in the November-December 1963 issue of HBR, p. 109, as well as the letters of Reiss, Suits, and Schairer in the March-April 1964 HBR, and Hodge's reply in the May-June 1964 issue, p.44.
The Institute of Scientific Information has conducted research on citation indexing for a bout 10 years(1). Our citation files now exceed 3,000,000 items. Theses files are now growing by the addition of at least 2,000,000 citations per year. A significant aspect of our coverage is the inclusion of all U.S. patents�not covered by the Hodge study. We have done a number of sociometric studies with citation data (2), (3), and several are now in progress (4). Several sociologists are using our files for various studies. We would be glad to assist any of your readers interested in similar studies.
Citation indexing can be used to facilitate evaluation of individual scientists or laboratories, but especially individual discoveries or inventions. "Impact" factors are in many superior to publication counting, but each has its own special values. For example, publication counting can tell you little about the effect of a man's work on others. Citation indexing can. We recently determined that two chemists, one American, the other Soviet, had each published 117 papers during a four-year period. However, the work of the American chemist was cited hundreds of times; while, during the same period, the Soviet chemist's work was almost completely ignored in the broad literature covered by the 1961 annual SCIENCE CITATION INDEX. Several interesting interpretations can be given to such data.
Hodge claims the use of citations limits its applications due to the inherent time lag involved. This is not generally true. For instance, if a research laboratory is well established, it will have a long record of publication, and its publications will also be cited to the extent that the work was impact. However, if a young firm is involved, then it's publication count will be small in most cases. And yet, a single great breakthrough will be cited frequently�even within a short time. For sociometric purposes, this time lag will be inconsequential.
While Hodge is correct in stating that it was not previously a practical matter for the individual administrator to make citation counts, this s no longer true do to the availability of the SCIENCE CITATION INDEX. Data obtained from this index, would among other things, not suffer from the bias inherent in Hodge's study. While Randall (HBR, May-June 1964, p. 184) points out the consequences of Hodge's failure to cover the biological sciences, there are non-random factors involved which make any sampling procedures suspect. This may appear to be a strong statement, but the size of the populations involved is such that the addition or omission of any one journal might significantly affect on of Hodge's tables. Manu journals of this type were omitted from his study. One the other hand, his list was well chosen in that it included many of the top-ranking journals�those which contain large numbers of articles. Indeed, if anything, his study shows that people rate journals as important in proportion to the number of articles they contain�a valid measure�but not as revealing a measure as impact factor (3) which ranks a journal on the basis of the average number of citations the to the average article. These qualitative differences in journals are similar to those observed for individual papers.
Sincerely yours,
Eugene Garfield, Ph.D
DirectorReferences
1. back to text E. Garfield, "Citation Indexes for Science," SCIENCE 122(3159), 108-111(1955). PDF File
2. back to text E. Garfield, "Citation Indexes in Sociological and Historical Research," AMERICAN DOCUMENTATION 14(4), 289-291(1963). PDF File
3. back to text E. Garfield and I. H. "New factors in the Evaluation of Science Literature through Citation Indexing," AMERICAN DOCUMENTATION 14(3), 195-201(1963). PDF File
4. back to text E. Garfield, "Citation Indexing: A Natural Science Literature Retrieval System for the Social Sciences." AMERICAN BEHAVIORAL SCIENTIST, 7(10), p.58-61,(1964). PDF File