Post-Publication Evaluation: Is There Life after the Impact Factor?
There are eight points I would like to make before showing the tutorial slides for the Journal Citation Reports (JCR) as well as supplementary slides provided by Marie McVeigh, the Manager of (JCR).
The URLs for the two PowerPoint slide series are:
1) When I started Current Contents in the fifties, the basic list of core journals of interest to the pharmaco-medical industry was provided to me by Charlotte Studer of Miles Laboratories. These 125 journals were included in Current Contents from 1958 onwards. So we assumed from the outset of the Science Citation Index (SCI) in the sixties, that the most important biomedical, chemical, and general science journals were covered. The JCR itself was originally called the Journal Citation Index. It is essentially a resorted version of the author-based citation index section of the SCI. Ten years later, in 1975, JCR became the last volume of the printed SCI. Then in 1988 it was spun off as a separate print product, followed later by microfiche and CD-ROM editions. Now it is available online as part of the Web of Knowledge portal. Entries in the SCI are also linked to JCR.
2) JCR provides a current short-term perspective on about 6,000 source journals. If you want a long-term perspective that provides cumulative impact data, you should consider using Journal Performance Indicators (JPI). (http://scientific.thomson.com/products/jpi/ ).
Unlike JCR, it excludes citations to non-substantive material since each source item is linked to the papers that cite it. This aspect of JCR is the subject of much discussion. JPI provides a more precise measure of impact even though this facet of JCR only significantly affects a small number of key journals like JAMA, Lancet, and others which publish numerous editorials, letters, and other non-substantive items.
3) Impact Factors as surrogates. Per Seglenís highly cited 1997 BMJ paper, mainly concerns the use of impact factors as surrogates in evaluating individuals. In general I agree with his admonitions but there are plenty of situations where it is relevant to use impact journal factors even for that purpose. For example, the Soros Foundation used impact factors to identify candidates for research grants in an initial screening of Russian applicants. This adversely affected many scientists who published in lower impact fields like Marine Biology. It tended to favor those who had published in English. However, these methods were later modified to take into account discipline and language differences.
4) The Key determinant of impact factor is not the size of the field. References per source item (R/S) and cited half-life are the significant elements in impact measurements. However, large fields will inevitably produce relatively large numbers of citation classics.
5) Users of JCR have often expressed dissatisfaction with ISIís heuristic categorization procedures. It is now possible to obtain a more accurate topical listing of journals by relatedness. For more information on relatedness measures, see the paper by Pudovkin and Garfield and see the new related journals feature in JCR itself.
6) Nobel characteristics. A significant characteristic of Nobel Class scientists is that they have, almost invariably, published citation classics which appeared in high impact journals. Furthermore, the work of Nobelists is cited 30 to 50 times the average author. And they publish five to six times the number of papers per author. Furthermore, their work is cited by other highly cited authors.
7) To obtain another citation perspective on a journal, use the Web of Science (WOS)to do a general search by source work. In seconds, you can rank, by citation frequency, as many as 100,000 articles published in that journal.
To obtain an even more detailed view of a journal, you can export marked records from a WOS search to HistCite software. To see examples of such journal files, go to www.Histcite.com .
8) The SIGMETRICS/Listserve of the American Society for Information Science and Technology (ASIS&T) will be of interest to impact factor aficionados, or those interested in what Stephen Lock called journalology. The archive is available free. We post the full source descriptions and even full-text URLs, when available, for 200-300 published theoretical and practical citation studies per year. http://web.utk.edu/~gwhitney/sigmetrics.html