Frank-Thorsten Krell's final reply of February 28, 2002 in Nature to E. Garfield's letter of September 13,2001
#eg
13 September 2001
Nature 413, 107 (2001) © Macmillan Publishers Ltd.
Taxonomy is small, but it has its citation classics
by Eugene Garfield in response to Frank-Thorsten Krell's letter, which follows:
"Impact factors aren't relevant to taxonomy" Nature 405, 507 - 508, 1 June 2000
© Macmillan Publishers Ltd.Sir – F. T. Krell in Correspondence ("Impact factors aren't relevant to
taxonomy", Nature 405, 507–508, 2000) suggests that taxonomy is
subject to different "regularities" from other fields. He uses the allegedly
small number of entomology, biodiversity and taxonomy journals
covered in the Science Citation Index (SCI) as the reason for low
impact factors. But he provides only anecdotal data on the size of this
literature. How many articles are published by the low-impact journals
at the Natural History Museum?Krell refers to Bradford's law of scattering, but he needs instead to
provide data to show that entomology–taxonomy diversity is somehow
different from other fields. Bradford's law simply suggests that if there
are, say, 1,000 journals in a field, then one-third of the papers are to be
found in each of three zones containing about 10, 100 and 1,000
journals, respectively.A critical element in determining the impact factor of a field is not the
number of papers it publishes, but the citation density of the average
paper and the half-life of the references cited. Adding more journals to
the SCI database would not increase the impact because the increased
number of cited references would have to be shared by more published
papers.The data on the 65 entomology journals covered in the SCI indicate
that their impact factors are not significantly different from other fields
with long half-lives (see the Institute for Scientific Information's Journal
Citation Reports,
http://www.isinet.com/isi/products/citation/jcr/?version=1). However,
the size of a field does affect the number of super-cited papers that will
be published.Krell states: "Qualified referees must evaluate the scientific work itself."
But this is equally true of any field. Why shouldn't such evaluation be
supplemented by citation analysis? The question is whether the referee
has any basis for comparing articles, authors or journals across a wider
horizon. Taxonomy, small as it may be, is not without its 'citation
classics', as the work of R. Sokal, E. O. Wilson and others
demonstrates. Their work is cited by thousands of papers covered in
the SCI, by taxonomists and by other scientists.Eugene Garfield
Institute for Scientific Information, 3501 Market Street,
Philadelphia, Pennsylvania 19104, USA
_____________________________________________________________
LETTER FROM: Frank-Thorsten Krell
01 June 2000
Nature 405, 507 - 508 (2000) © Macmillan Publishers Ltd.
Impact factors aren't relevant to taxonomy
Sir – Evaluating research by means of ISI impact factors — those
determined by the Institute for Scientific Information in Philadelphia —
gives rise to obvious problems in the case of basic biodiversity
research. Valdecasas et al. in Correspondence (Nature 403, 698;
2000) suggested that one cause is the low citation rate of taxonomic
articles compared to other fields.A more important reason, however, is the inapplicability of this index to
basic biodiversity research on principle.The impact factor generally underestimates the number of citations
because of the limited number of scanned journals. In fields such as
molecular genetics, neuroscience or cancer research — where, as
Bradford's law states, most of the relevant work is published in a few
core journals — this selection does not severely affect the comparability
of impact factors.However, Bradford's law does not apply to taxonomy and other areas
of basic biodiversity research. Generally, taxonomic papers contain
details of nomenclature, which must be considered and discussed
whether the paper is excellent or poor. Similarly, reports of flora and
fauna records need to be debated in further studies before being
accepted or rejected. Quality and relevance are independent
parameters.Therefore, it is impossible to classify taxonomic or ecology journals as
more or less important. They can only be classified as of high or low
quality, which does not affect the number of citations. Many natural
history journals contain taxonomic or similar articles. The library of
London's Natural History Museum alone holds 11,000 serial titles,
most of which come under this category, and each may contain relevant
information. About 1,000 entomological journals are held by the
Natural History Museum's entomology library, but only 65 are covered
by the expanded Science Citation Index (SCI). Because some 93 per
cent of potentially relevant journals are not considered, it is good luck
rather than a sign of good quality to be included in the SCI.The impact factor is a powerful and welcome tool in some disciplines.
In basic biodiversity research, however, it is not applicable. Qualified
referees must evaluate the scientific work itself.Frank-Thorsten Krell
Entomology Department, Natural History Museum, Cromwell
Road, London SW3 5BD, UK
___________________________________________________________
#Frank-Thorsten Krell's final reply of February 28,
28 February 2002
Nature 415, 957 (2002) © Macmillan Publishers Ltd.
Why impact factors don't work for taxonomy
Sir – In an earlier Correspondence1, I explained why impact factors are
irrelevant for judging the quality of taxonomy. E. Garfield2 requested
quantitative arguments to support my hypothesis that taxonomy follows
different "regularities" from most sciences, making the impact factors as
calculated by the ISI (formerly the Institute of Scientific Information)
inapplicable.If ISI impact factors are to judge research meaningfully, as discussed in a
recent News Feature3, they have to be a roughly accurate estimate of the
real impact of publications. This requires that: (1) the impact of a paper is
expressed by citations and the citation impact is positively correlated with
the quality and/or relevance of the paper; (2) most cited and citing journals
are considered (easily possible if Bradford's law of scattering applies to a
field, as most cited papers are published in a few core journals); and (3) a
paper gets most of its citations in the first few years after publication. These
requirements are not met by taxonomy, for the following reasons.First, the number of taxonomists is declining. Consequently, taxonomy does
not follow the 'exponential curve' of most sciences. The old literature is not
overwhelmed by an avalanche of new papers. The peak of species
descriptions (a rough surrogate for relevant publications) was before 1900
for most groups, so the average age of references in taxonomic publications
is much greater than those in other scientific disciplines.I have analysed 2,091 references from seven randomly chosen,
comprehensive taxonomic papers. The mean age of the references is 61
years, the median 36 years (details available direct from F.-T. K.), with
98.5% of cited papers being more than two years old. In shorter taxonomic
journal papers, F. Köhler4 found the average age of citations to be 46.7
years (s.d. 30.5 yr) and 41.8 years (s.d. 22.7 yr). It is therefore pointless to
judge taxonomists according to the ISI method of analysing citations over
the preceding two years.Second, the relevance of descriptive publications in this field remains the
same over time; original descriptions have to be referred to for ever,
independent of the paper's quality. Outside taxonomy, referring to original
descriptions becomes superfluous with time and/or revisions. Einstein's
papers are not always cited for his theory, unlike the original descriptions of,
say, Escherichia coli or Drosophila melanogaster. This obsolescence
underestimates impact only for authors of well-studied species5. However, in
taxonomy this obsolescence effect is negligible.Third, for any group of organisms there are at best a handful of (or
frequently no) extant specialists. Therefore the chance to become cited by
colleagues is relatively rare compared with other fields. The number of
taxonomists and consequently the number of publications is this low for one
reason, which has nothing to do with need or quality: decision-makers are
generally more enthusiastic about other fields. For these reasons, taxonomic
papers have a long-term impact. Sometimes taxonomists have to wait a
generation to be heavily cited. The number of citations of their empirical
taxonomic publications depends on the number of taxonomists working on
the same field and whether these colleagues publish in the few taxonomic
journals covered by the Science Citation Index (SCI). These things are a
matter of luck.Fourth, there are no core journals for general taxonomy. These exist for
cladistics, biogeography, chemical systematics, and so on, but not for
species descriptions, revisions of genera, identification keys or inventories.
Where to publish depends on which museum the material is in; which
institute or learned society the taxonomist is affiliated to; or which serial has
the funding to accept long monographs. Only 27 (42%) of the 64
entomological journals covered by the SCI Expanded publish taxonomic
papers, whereas 898 (82%) of the 1,100 serials held by the entomology
library of the Natural History Museum in London probably contain
taxonomic information. Because of the less-developed relevance hierarchy
of taxonomic journals, the low proportion covered by the ISI puts taxonomy
at a disadvantage.Finally, the most important impact of taxonomy is the usage of identification
keys which enable non-taxonomists to identify and work with a group of
organisms. The use of such keys is generally not documented in reference
lists, hence a crucial impact of taxonomy is missed by citation analysis.Consequently, taxonomy has no citation classics which the ISI would
uncover. E. O. Wilson's citation classics aren't his taxonomic works, and R.
Sokal's treatises on biometry and on the method of numerical taxonomy (not
an empirical taxonomic work, but a methodological textbook on
classification) have far more citations than his taxonomic work. The SCI is
not an appropriate means to judge taxonomy because taxonomy does not
meet its requirements for a meaningful judgement.Frank-Thorsten Krell
Department of Entomology, The Natural History Museum, London
SW7 5BD, UK
f.krell@nhm.ac.uk
References
1. Krell, F.-T. Nature 405, 507-508 (2000). | Article | ISI |
2. Garfield, E. Nature 413, 107 (2001). | Article | PubMed | ISI |
3. Adam, D. Nature 415, 726-729 (2002). | Article | PubMed | ISI |
4. Köhler, F. Amateurwissenschaft: Entwicklung, Beschreibung und
wissenschaftssoziologische Analyse am Beispiel der Koleopterologie p.
195 (Diploma thesis: Sociology, Univ. Köln, 1988).
5. van der Velde, G. Nature 414, 148 (2001). | Article | PubMed | ISI |
#back to E.Garfield's letter 28 February 2002
Nature 415, 957 (2002) © Macmillan Publishers Ltd.
Why impact factors don't work for taxonomy
Frank-Thorsten KrellSir – In an earlier Correspondence1, I explained why impact factors are
irrelevant for judging the quality of taxonomy. E. Garfield2 requested
quantitative arguments to support my hypothesis that taxonomy follows
different "regularities" from most sciences, making the impact factors as
calculated by the ISI (formerly the Institute of Scientific Information)
inapplicable.If ISI impact factors are to judge research meaningfully, as discussed in a
recent News Feature3, they have to be a roughly accurate estimate of the
real impact of publications. This requires that: (1) the impact of a paper is
expressed by citations and the citation impact is positively correlated with
the quality and/or relevance of the paper; (2) most cited and citing journals
are considered (easily possible if Bradford's law of scattering applies to a
field, as most cited papers are published in a few core journals); and (3) a
paper gets most of its citations in the first few years after publication. These
requirements are not met by taxonomy, for the following reasons.First, the number of taxonomists is declining. Consequently, taxonomy does
not follow the 'exponential curve' of most sciences. The old literature is not
overwhelmed by an avalanche of new papers. The peak of species
descriptions (a rough surrogate for relevant publications) was before 1900
for most groups, so the average age of references in taxonomic publications
is much greater than those in other scientific disciplines.I have analysed 2,091 references from seven randomly chosen,
comprehensive taxonomic papers. The mean age of the references is 61
years, the median 36 years (details available direct from F.-T. K.), with
98.5% of cited papers being more than two years old. In shorter taxonomic
journal papers, F. Köhler4 found the average age of citations to be 46.7
years (s.d. 30.5 yr) and 41.8 years (s.d. 22.7 yr). It is therefore pointless to
judge taxonomists according to the ISI method of analysing citations over
the preceding two years.Second, the relevance of descriptive publications in this field remains the
same over time; original descriptions have to be referred to for ever,
independent of the paper's quality. Outside taxonomy, referring to original
descriptions becomes superfluous with time and/or revisions. Einstein's
papers are not always cited for his theory, unlike the original descriptions of,
say, Escherichia coli or Drosophila melanogaster. This obsolescence
underestimates impact only for authors of well-studied species5. However, in
taxonomy this obsolescence effect is negligible.Third, for any group of organisms there are at best a handful of (or
frequently no) extant specialists. Therefore the chance to become cited by
colleagues is relatively rare compared with other fields. The number of
taxonomists and consequently the number of publications is this low for one
reason, which has nothing to do with need or quality: decision-makers are
generally more enthusiastic about other fields. For these reasons, taxonomic
papers have a long-term impact. Sometimes taxonomists have to wait a
generation to be heavily cited. The number of citations of their empirical
taxonomic publications depends on the number of taxonomists working on
the same field and whether these colleagues publish in the few taxonomic
journals covered by the Science Citation Index (SCI). These things are a
matter of luck.Fourth, there are no core journals for general taxonomy. These exist for
cladistics, biogeography, chemical systematics, and so on, but not for
species descriptions, revisions of genera, identification keys or inventories.
Where to publish depends on which museum the material is in; which
institute or learned society the taxonomist is affiliated to; or which serial has
the funding to accept long monographs. Only 27 (42%) of the 64
entomological journals covered by the SCI Expanded publish taxonomic
papers, whereas 898 (82%) of the 1,100 serials held by the entomology
library of the Natural History Museum in London probably contain
taxonomic information. Because of the less-developed relevance hierarchy
of taxonomic journals, the low proportion covered by the ISI puts taxonomy
at a disadvantage.Finally, the most important impact of taxonomy is the usage of identification
keys which enable non-taxonomists to identify and work with a group of
organisms. The use of such keys is generally not documented in reference
lists, hence a crucial impact of taxonomy is missed by citation analysis.Consequently, taxonomy has no citation classics which the ISI would
uncover. E. O. Wilson's citation classics aren't his taxonomic works, and R.
Sokal's treatises on biometry and on the method of numerical taxonomy (not
an empirical taxonomic work, but a methodological textbook on
classification) have far more citations than his taxonomic work. The SCI is
not an appropriate means to judge taxonomy because taxonomy does not
meet its requirements for a meaningful judgement.Frank-Thorsten Krell
Department of Entomology, The Natural History Museum, London
SW7 5BD, UK
f.krell@nhm.ac.uk
References
1. Krell, F.-T. Nature 405, 507-508 (2000). | Article | ISI |
2. Garfield, E. Nature 413, 107 (2001). | Article | PubMed | ISI |
3. Adam, D. Nature 415, 726-729 (2002). | Article | PubMed | ISI |
4. Köhler, F. Amateurwissenschaft: Entwicklung, Beschreibung und
wissenschaftssoziologische Analyse am Beispiel der Koleopterologie p.
195 (Diploma thesis: Sociology, Univ. Köln, 1988).
5. van der Velde, G. Nature 414, 148 (2001). | Article | PubMed | ISI |