But ISI's databanks are increasingly being used for other purposes-namely, the assessment and evaluation of scientific performance. Indeed, they have enabled the emergence of a new field of quantitative analysis, the "science of science"- scientometrics. This new use of ISI data is of particular interest to science administrators, policymakers, funding agencies, legislators, editors, journalists, and others.
My remarks will focus on what citation data can
show about science in Spain. The time period of analysis is the most recent
12 years, 1981-1992. This will give a current citation perspective
on Spain's research. To begin, data will be presented on the number of
ISI-indexed papers by authors based in Spain, and their average citation
frequency or impact. Spain's impact trends will be compared with other
European Community
nations for an international context. Following
this, we'll identify the most-cited papers from Spain as well as the highest
output and impact institutions and authors.
Before we begin, it should be stressed that this analysis is not an exhaustive survey of all research papers by every Spanish author. Instead, it examines that portion of Spanish research which was published in the leading international journals indexed by ISI. As it turns out, ISI's data consistently show that a relatively small set of journals accounts for the vast majority of both papers and citations.
This is shown in the following slide.
SLIDE 1: SOURCE/CITE DISTRIBUTION, 1989 JCR
The data shown here are based on about 4,500 journals that were covered in the 1989 Journal Citation Reports® (JCR®) volumes of the SCI. The dashed line shows that just 100 journals accounted for more than 20 percent of what is published. Just as interesting, the solid line shows that 100 journals account for more than 40 percent of what is cited. Only 600 journals account for more than half of what is published--and over 75 percent of what is cited. Thus, by covering over 7,000 journals, ISI is indeed capturing far more than the most significant journals of international research.
Let's begin our analysis by examining the number
of papers from Spain annually indexed in ISI's databases from 1981 to 1992.
SLIDE 2: ANNUAL SPAIN OUTPUT, 1981-92 SCI
The solid bars represent the numbers of papers that included at least one author listing an address in Spain in each year. It is obvious that Spain's output has increased sharply--from 3900 in 1981 to 14,000 in 1992, a growth of 260 percent. In comparison, the entire SCI annual output grew by 18 percent over the same period. Spain's remarkable output trend is not necessarily an artifact of ISI journal coverage changes. In 1981, ISI indexed 11 journals from Spain, and 7 in 1992. Thus, the data indicate that authors from Spain published substantially more papers in the international literature. Increased national funding, multinational collaboration, or other factors may be involved. In any case, even if ISI had covered 50 journals from Spain, this "domestic" output would still be far less than Spain's output in the international literature.
ISI's data can also indicate how often Spain's papers were cited over this 12-year period. This is shown in the following slide.
SLIDE 3: ANNUAL CITATIONS TO SPAIN PAPERS, 1981-92
SCI
The solid bars indicate the number of papers from Spain. The shaded bars show citations received from year of publication to 1992.
The data show an interesting trend. For example,
1986 papers accrued more citations in 7 years than earlier papers did over
more years. This illustrates a well-known phenomenon-authors tend to cite
relatively current papers published within the past 5 years or so, and
citations to older literature past that point tends to fall off. This citation
"attention span" varies between fields, with the molecular biology/genetics
literature reaching a citation peak at say 3-4 years while
mathematics papers may take 10 or more years.
By dividing numbers of citations by numbers of papers, we can calculate the average impact of Spanish research. In the following examples, we use five-year time periods that advance one year at a time--from 1981-1985 through 1988-1992. That is, we aggregate papers published in each five-year period and calculate impact based on the number of citations they received in that period. This effectively gives all papers an equal opportunity to accrue citations regardless of their "vintage".
Another way to calculate impact is by dividing citations by just the papers that were actually cited during these five-year periods. That is, uncited papers are excluded from the calculation. As a result, the so-called "cited" impact factors will be higher than the total impact.
The resulting trends in both impact and cited impact factors are shown in the following slide.
SLIDE 4: IMPACT & CITED IMPACT OF SPAIN PAPERS,
1981-1992 SCI
The bottom line shows total impact trends and the top line shows cited impact trends. Note that both are increasing. That is, authors from Spain were publishing more papers of higher impact over time.
The total impact of Spain's papers increased by 48 percent, from 1.2 in 1981 to 1.8 in 1992. In comparison, cited impact increased by 17 percent, from 3.5 in 1981 to 4.1 in 1992. That Spain's cited impact increased at a slower rate than total impact is a positive sign. It indicates that a higher proportion of Spain's papers were being cited. For example, of the 27,000 papers from Spain in the period 1981-1985, 34 percent were cited and 66 percent uncited. But in 1988-1992, 43 percent of Spain's 54,000 papers were cited.
While it is interesting to examine Spain's impact trends, it is perhaps more meaningful to compare Spain's performance with other nations. The following slide shows the cited impact trends of Spain in relation to the US, UK, Germany, France, and Italy.
SLIDE 5: SPAIN'S CITED IMPACT vs OTHER NATIONS,
1981-1992 SCI
In this graph, the relative rankings of the six nations is not necessarily as significant as each nation's trend in cited impact. For example, Germany ranked third throughout the 12-year period, but it showed the highest gain in cited impact--21 percent. And it rivals the UK in the most recent 1988-1992 period for second place. Spain, which ranked last, had the second highest growth rate--17 percent.
Instead of using absolute cited impact factors, another way to view Spain's performance is to compare it against an average baseline. In the following example, we've aggregated cited impact data for the 12 European Community nations--Belgium, Denmark, France, Germany, Greece, Ireland, Italy, Luxembourg, The Netherlands, Portugal, Spain, and the UK. Spain's performance against this regional average is shown in the following slide.
SLIDE 6: SPAIN vs EC CITED IMPACT, 1981-1992
In this graph, the EC average cited impact is defined as 1.0 for the entire time period shown here. Spain's cited impact is below that of the EC baseline by about 30 percent. That is, its papers were cited about 30 percent less frequently as the average EC paper. However, note that Spain's performance has improved slightly over the 12-year period--its relative cited impact was 0.68 in 1981-1985 and .71 in 1988-1992, a 4 percent increase.
These data are for all of science as a whole. Of course, ISI's databases can also show relative cited impact trends for different fields. Before we present field-specific comparisons, let us first examine the proportional distribution of Spain's papers by field, shown in the next slide.
SLIDE 7: DISTRIBUTION OF SPAIN PAPERS BY FIELD,
1981-1992
Of the 117,000 ISI-indexed papers from Spain, 32 percent were in the life sciences. This field includes molecular biology and genetics, biochemistry and biophysics, microbiology and cell biology, and so on.
The physical and chemical sciences, which also includes the earth sciences and mathematics, accounted for 29 percent. Clinical medicine is next with 24 percent. This field includes virtually all medical specialties, from anesthesiology to urology.
The agricultural and biological sciences accounted for 10 percent of Spain's output. This field includes agriculture, agronomy, plant sciences, aquatic and environmental sciences, etc.
The remaining 5 percent of Spain's papers were
in engineering and technology. This includes material science,
optics, acoustics, mining, and virtually all
engineering specialties-aerospace, chemical, civil, mechanical, nuclear,
etc.
The following slides compare Spain's performance in each of these fields against the average EC 5-year cited impact. They will be presented in order of proportional share, starting with the life sciences and ending with engineering and technology.
SLIDE 8: LIFE SCIENCES, SPAIN vs EC CITED IMPACT,
1981-1992
Throughout the 12-year period, the cited impact of Spain's life sciences papers remained virtually constant relative to the EC average in this field. There is only a slight 2 percent increase, from .65 in 1981-1985 to .66 in 1988-1992.
In absolute terms, the cited impact of Spain's life sciences papers increased by 19 percent, from 4.0 in 1981-1985 to 4.8 in 1988-1992. The cited impact of all EC life sciences papers also increased by 17 percent, from 6.2 to 7.2 over this period.
Let's now examine the physical and chemical sciences.
SLIDE 9: PHYS/CHEM, SPAIN vs EC CITED IMPACT,
1981-1992
While Spain's cited impact in the physical and chemical sciences was consistently below the EC average, it has improved significantly. Spain's relative cited impact in this field increased by 14 percent, from .65 in 1981-1985 to .74 in 1988- 1992.
In absolute terms, the cited impact of Spain's
physical and chemical sciences papers increased by 21 percent, from 3.5
to 4.2 over this period. In comparison, the EC's average cited impact increased
by 6 percent, from 5.4 to 5.7.
Let's now move on to clinical medicine.
SLIDE 10: CLIN MED, SPAIN vs EC CITED IMPACT,
1981-1992
Spain's cited impact trend relative to the EC average in this field has also improved. It increased by 12 percent, from 0.73 in 1981-1985 to .82 in 1988-1992. In this most recent period, Spain's clinical medicine papers were cited only 20 percent less frequently than the average EC paper.
In absolute terms, the cited impact of Spain's papers in this field increased by 28 percent, from 3.2 to 4.1 over this period. The EC's average cited impact increased by 14 percent, from 4.4 to 5.0.
Let's now examine the agricultural and biological sciences.
SLIDE 11: AGRO/BIOL, SPAIN vs EC CITED IMPACT,
1981-1992
Here the situation is reversed. Spain's relative cited impact in this field decreased by 16 percent, from .68 in 1981 1985 to .57 in 1988-1992. In absolute terms, the cited impact of Spain's agricultural and biological sciences papers decreased slightly by 3 percent, from 3.2 to 3.0. In comparison, the EC's average cited impact in this field increased by 16 percent, from 4.6 to 5.4.
Let's now examine the last field of comparison--engineering and technology.
SLIDE 12: ENG/TECH, SPAIN vs EC CITED IMPACT,
1981-1992
At the beginning of the 12-year period, Spain's
cited impact in engineering and technology virtually equaled the EC average.
For example, in 1982-1986, Spain's papers were cited just 5
percent less frequently than the average EC paper.
After that, Spain's relative cited impact declined through 1986-1990, when
it began to rebound. Overall, Spain's cited impact declined by just 4 percent,
from .92 in 1981-1985 to .88 in 1988-1992.
In absolute terms, Spain's cited impact increased slightly by just 2 percent, from 2.8 at the beginning to 2.9 at the end. In comparison, the average EC cited impact increased by 6 percent, from 3.1 to 3.3.
Having examined broad statistical trends for Spain and the EC, we will now focus at a more specific level on Spain by identifying its highest impact papers that were published and cited from 1981 through 1992.
The following slides list 15 papers that were cited at least 180 times in this period. Before we review them, let me put this into context. In the 1945-1988 SCI database, over 32.7 million items were cited. Of these, just 0.1 percent were cited 200 or more times. That is, the highest impact papers from Spain are in the 99.9th percentile of all cited items. Thus, the papers we will now review are a very select sample of the entire scientific literature.
SLIDE 13A: MOST-CITED SPAIN PAPERS, 1981-1992
The highest impact paper--with 538 citations--concerns the regulation of T-cell growth by a transforming growth factor produced by T-lymphocytes. Published in 1986, it was authored by M. Alvarezmon of the Puerta Hierro Clinic, Madrid, and colleagues based in the US. The co-authors include Anthony Fauci, head of the National Cancer Institute.
The next paper, with 498 citations, is a large multinational collaboration in physics involving 16 authors from 13 institutions in 7 nations. These include authors based at the Universidad de Valencia and the Centro de Investigacion Energetica, Medio-ambiental y Tecnologia, Madrid. Published in 1990, this review paper has received about 500 citations in 3 years. This certainly qualifies it as a "hot" paper.
The third paper is another physics review article by basically the same team of authors, who call themselves the Particle Data Group. Published in 1986, the paper has received about 430 citations.
The fourth paper, with 399 citations, is an epidemiological analysis of hepatitis C virus antibodies in the Spanish population. It involved 14 authors from the Universidad Autonoma de Barcelona and the Chiron Corporation.
The last paper on the screen is on a plant sciences topic-- the nitrate reducing system and its regulation. All three authors of this 1981 review paper were based at the Universidad de Sevilla.
Additional papers are shown on the following slide.
SLIDE 13B: MOST-CITED SPAIN PAPERS (cont.)
The first paper here is by 8 authors based at the Universidad de Barcelona and concerns prognostic factors in patients who survived heart attacks. Published in 1985, it has received 235 citations.
The next paper concerns brain-derived acidic growth
factor. It involved six authors based at the Centro de Investigaciones
Biologicas, Madrid, and Merck Sharp & Dohme
in the US. Another paper on this topic by essentially the same team of
authors also ranks among the most-cited papers from Spain, and will be
shown on the next slide.
The next two papers are both on physics topics, and the last is a molecular biology and genetics paper.
Let's move on to the final papers on the list.
SLIDE 13C: MOST-CITED SPAIN PAPERS (cont)
The second paper here was mentioned earlier and concerns fibroblast growth factor. Also note that the last paper is another epidemiological study of hepatitis C prevalence in Spain.
From ISI's database of papers from Spain, it is fairly straightforward to derive lists of institutions and authors ranked by total papers, total citations, impact, etc. The following slides show the highest output institutions for the period 1981-1992.
SLIDE 14A: HIGHEST OUTPUT INSTITUTIONS, 1981-1992
These slides include 29 institutions that produced at least 500 papers during this 12-year period. The first two institutions produced virtually the same number of papers--the Universidad Autonoma de Madrid with 7200 and the Universidad de Barcelona with 7100. But note that their impact factors differ. Madrid's papers received about 42,000 citations, giving an average impact of 5.8, compared to 3.9 for Barcelona.
The Universidad Complutense de Madrid ranks third
with about 6600 papers. They received 20,200 citations, giving an impact
of
3.1.
The list is fairly simple and self-explanatory. Since time is running short, I won't discuss the rankings in detail. Copies of this information will be made available to you to review at your convenience. Let's now move on to see the remaining institutions.
SLIDE 14B: HIGHEST OUTPUT INSTITUTIONS (cont.)
Output or productivity is perhaps the most traditional quantitative indicator used to compare the research performance of institutions, departments, authors, etc. But the most productive institutions are not necessarily those with the highest impact. The following slides rank Spain institutions by impact in all sciences.
Only those institutions that produced at least 100 papers are listed. This effectively "censors" the occasional citation outlier--that is, institutions which may achieve high impact on the basis of just a few very highly cited papers. Thus, by setting an arbitrary threshold of 100 or more papers, the list will include those institutions that consistently produced papers throughout the 12-year period.
Let's now see how Spain's institutions compare in terms of impact.
SLIDE 15A: HIGHEST IMPACT INSTITUTIONS, 1981-1992
These slides show 24 institutions with an impact of at least 3.0 for papers published and cited from 1981 through 1992. Asterisks indicate those that also appeared on the list of highest output institutions.
The Junta de Energia Nuclear shows the highest impact by a wide margin. It produced 170 papers that received 2100 citations for an impact of 12.6.
Also, the list is dominated by research centers and institutes of Spain's national research council--the Consejo Superior de Investigaciones Cientificas (CSIC). All had impacts greater than 4.0.
Two universities rank among the top 10 in terms of impact - the Universidad Autonoma de Madrid (# 5, 5.8) and the Universidad de Barcelona (# 10, 3.9).
The following slide shows the remaining highest impact institutions in Spain.
SLIDE 15B: HIGHEST IMPACT INSTITUTIONS (cont.)
As we move down the impact rankings, we see the universities that also ranked among the most productive in Spain. Of the 24 highest impact institutions shown here, 15 also appeared on the list of highest output institutions.
I should point out that it is important for authors
to be consistent in the way they list their institution's names on papers.
A good example is the Universitat de les Illes Balears. Many list it under
that name, but others have also used Universitat Balears, Universitat Islas
Balears, Universitat Balears Isles, and so on. In these lists, we have
unified all such variants. However, authors may simply give Faculty of
Medicine without any indication of the university. Faculty deans or department
heads should define how their institutions should be listed on papers and
ensure that researchers follow this
recommendation.
Let's now move on to identify the highest output authors in Spain, based on 1981-1992 papers.
SLIDE 16A: HIGHEST OUTPUT AUTHORS FROM SPAIN,
1981-1992
This series of slides shows 20 authors who produced at least 150 papers from 1981 through 1992.
Two authors published over 300 papers-J. Rodes, Universidad de Barcelona Hospital, with 315 papers and J. Elguero, CSIC Instituto de Quimica Medica, Madrid, with 313.
The list is self-explanatory, so I'll make a few general points. One of the problems with compiling these ranked lists is homographs. That is, two or more different authors with the same surname and initials. In most cases, they can be identified by checking institutional addresses-usually, they are based at different universities. Even when they are at the same institution, they usually are in different departments-physics versus biomedicine, for example. However, the best way to discriminate between homographs is to refer to the individual authors' c.v. and personal bibliography. In these lists, every effort was made to ensure that homographs were deleted from the rankings. This was done by checking addresses in several annual files of the SCI.
Of the 20 authors, five are based at the Universidad de Barcelona. Four were based at CSIC institutes-2 at the Instituto de Ciencia de Materiales de Aragon (Universidad de Zaragoza), and one each at the Instituto de Quimica Fisica (also Universidad de Zaragoza) and the Instituto de Quimica Medica, Madrid. The Universidad de Cordoba accounts for 3 authors, and 2 are based at the Universidad Autonoma de Madrid. The following universities each accounted for 1 author: Alcala de Henares, Complutense-Madrid, Extremadura-Badaj oz, Murcia, Oveido, and Santiago de Compostela.
Let's now move on to the final slide in this presentation, which shows the highest impact authors from Spain.
SLIDE 17A: HIGHEST IMPACT AUTHORS FROM SPAIN,
1981-1992
This list includes only those authors who produced at least 50 papers from 1981 through 1992. By doing so, we effectively exclude authors who may achieve high impact rankings based on just one or two highly cited papers.
Several of the names will be familiar because they appeared as authors of the most-cited papers reviewed earlier.
The highest impact author is M. Aguilarbenitez of the Centro de Investigacion Energetica, Medio-Ambiental y Tecnologica, with an impact of about 28-based on 65 papers that received about 1800 citations.
Of the 19 authors listed, 7 are based at the Universidad
Autonoma de Madrid. Five are based at CSIC institutes: 1 each at the Centro
de Investigaciones Energetica, Medio-Ambiental y Tecnologica (Madrid);
Centro de Investigacion y Desarrollo (Barcelona); Instituto de Ciencias
de Materiales de Aragon (Universidad de Zaragoza); Instituto de Investigaciones
Biomedicas (Madrid); and the Instituto de Fisica Corpuscular (Universidad
de Valencia). The Universidad de Barcelona accounts for 3 authors, and
the following universities each account for 1:
Cordoba, Murcia, and Sevilla. In addition, 1
author is based at the Hospital Clinica de Madrid.
The following slide shows the remaining high impact authors.
SLIDE 17B: HIGHEST IMPACT AUTHORS (cont.)
It is interesting to note that of the 19 highest impact authors on this list, only 2 also appeared on the list of most productive authors. They are indicated by asterisks-R. Uson, Universidad de Zaragoza, and J. Rodes, Universidad de Barcelona.
This concludes our citationist perspective on current research in Spain. I hope this presentation has demonstrated that citation data provide interesting insights on national and institutional scientific performance. Of course, like any quantitative indicator, citation data have their advantages-and limitations. And the appropriate and balanced interpretation of citation data requires the input of experts familiar with these applications.
In particular, conclusions drawn from citation data on individuals require special care and attention. Citations are formal acknowledgments of intellectual debt, indicating that an idea or method has influenced later research in some way. But citation data do not necessarily indicate the quality, importance, or significance of an author's work. These are subjective judgments that require peer review and assessment by specialists familiar with the research topic in question.
Thank you for your attention.