The Ian P. Sharp Lecture on Information Science was established in 1990 with an endowment from Reuters Information Services (Canada) Limited in honour of its founding president and former chi ef executive officer. The lectureship is intended to provide a forum for distinguished figures in information science and related fields. LP. Sharp Associates, one of the world�s leading numeric database com-panies, was founded by Ian P. Sharp and seven colleagues. The Canadian company soon expanded, establishing a timesharing service and pioneering the use of electronic mail. In 1976, the company installed its own private, packet-switched network and today it supplies the world�s major financial and economic centres with historical information and financial products. In June 1987, LP. Sharp Associates was acquired by Reuter Holdings PLC of London, the world�s largest electronic publisher.

Dr. Garfield, the fourth I.P. Sharp lecturer, delivered the address that follows at the University of Toronto on April 8, 1993. He was introduced by his longtime colleagueProfessor Charles Meadow, Faculty of Library andlnformation Science, University of Toronto.
 

Introduction

Many of you are no doubt familiar with the Institute for Scientific Information® (ISI®) and its many products and services. These include Current Contents® (CC®), our primary current awareness service, which is available in print, online, magnetic tape, and floppy diskette. In addition, the Science Citation Index® (SCI®), Social Sciences Citation Index® (SSCI®), and Arts & Humanities Citation Index® (A&HCI®) are our primary information retrieval services. Like CC, they are available in a variety of fonnats.

In essence, ISI�s products and services are tools for navigating the scholarly literature. That is, they help researchers, students, administrators, and others to discover relevant information in the flood of papers published every week in thousands of journals worldwide. And they help to recover infonnation you know�or think�is out there somewhere in the vast archive of published research.

But the topic of this lecture involves the application of ISI�s databases for purposes other than current awareness and information retrieval. That is, citation databases are increasingly being used as indicators for science evaluation (Garfield 1983a; Garfield 1983b; Garfield and Welljams-Dorof 1992).

I cannot provide an exhaustive analysis of Canadian science in the time available. This lecture will serve to illustrate the variety of perspectives on Canadian science that are possible with citation data. The examples will range from the general to the specific, including rankings of nations, institutions, fields, papers, and authors.

Output and citation of Canadian papers

To begin, let�s examine Canadian output in the SCI, shown in Figure 1. There is a clear upward trend in the output of Canadian authors. In 1981 theSCI indexed about 20,500 papers with at least one Canadian address, compared to over 27,000 in 1991. This represents a growth of 32.4% over the eleven-year period. Increased collabora-tion could account for this growth. Collaboration increases absolute output but not necessarily the average number of papers per author.

Canada�s growth in output is comparable to that of other nations�for example, papers from Australia grew by 32.4%, from Germany by 25.8%, from all of Europe by 3 1.0%, and from Japan by 44.5%. And Canada�s output growth is about three times greater than that of the SCIas a whole (9.7%), the United States (11.7%), and the United Kingdom (11.3%).

Notice that the growth in Canadian papers is not smooth and steady. For example, there are slight declines in the years 1989 through 1991 compared with the previous four years. This is due in part to changes in SCI coverage�the total number of SCI papers also declined during this three-year period primarily because the SCI reduced the number of indexed journals since 1986.

Of course, ISI�s databases can also track the number of citations to these Canadian papers, which is shown in Figure 2. The clear declining trend in number of citations in recent years has an obvious explanation�that is, papers published in earlier years have had more time to accrue citations than those published more recently. For example, 1981 papers received about 91,600 citations over the eleven-year period from 1981 through 1991. However, 1991 papers received about 7,000 citations in that single year.

Impact of Canadian papers vs world averages

We have developed a method for smoothing out these annual fluctuations in both numbers of citations and papers. It involves aggregating publication and citation data over five-year periods. These five-year �windows� overlap and advance a year at a time.

Figure 3 shows the numbers of Canadian papers and citations in these five-year periods. The number of papers is shown in the solid bars, and citations in the shaded bars. In these five-year windows, the number of Canadian papers steadily increases from about 106,000 in 1981�85 to about 142,000 in 1987�91. This is a growth of 38.8%. The number of citations grew by 5 1.7%, from 262,000 to 397,000 over the same time period.

From these data one can calculate the so-called �impact factor��that is, the average number of citations per paper published during these five-year periods. Actually, there are two ways to calculate impact. Citations can be divided by the total number of papers to give the standard impact factor. Alternatively, citations can be divided by only the number of papers that were actually cited. That is, uncited papers are excluded from the calculation. This results in the so-called �cited� impact factor.

Both impact factors are shown in Figure 4. The top line shows trends in the five-year �cited� impact of Canadian science, and the bottom line shows overall impact factors. Both trend lines were stable until the period 1984�88, when each began to steadily increase. Cited impact factors rose by 7.8% from the beginning to the end of the period, and overall impact increased by 12.0%. This difference is probably due to lower uncitedness. One could speculate on the reasons.

It would be interesting to determine if these trends reflect initiatives by Canadian science policymakers, or changes in the level and allocation of research funding. Then again, the trends may simply reflect a general impact �inflation� in other leading research nations. Answers to these questions require the input of information specialists working closely with knowledgeable scientists, administrators, and policymakers.

While reviewing impact factor trends for Canadian papers is interesting, it is perhaps more meaningful to compare them with other nations. One way to do so is to define a �baseline� for comparison. We use the average impact factors for the entire SCI file as a baseline.

In Figure 5, the citation impact of Canadian papers is compared with those of the other so-called Group of Seven (G7) nations�the U.S., the U.K, France, Germany, Italy, and Japan. This graph is taken from a recent issue of Science Watch, ISI�s monthly newsletter that tracks trends in research performance (Garfield 1990). The world average is arbitrarily set at 1.0 throughout the eleven-year period. That is, if a nation�s impact is higher than the world average, it will have a value greater than 1.0.

The U.S. shows the highest impact relative to the world average, exceeding it by about 45% to 50% throughout the period. The U.K. is next, followed by Canada.

How much greater a nation�s impact is than the world average at any given point in time is relevant for analysis. However, the trend is perhaps more important. For example, the U.S. is steadily increasing its relative impact against the world average. On the other hand, the U.K. is declining�which should be of some concern to British policymakers.

Canada�s relative impact is fairly stable, with a slightly increasing trend. Note that Germany�s performance is increasing at a greater rate, and its relative impact equalled Canada�s by 1987�91. Note also that Japan matched the world average for the first time in the period 1986�90. It will be interesting to see if Japan�s scientific progress continues in future years.

Using the G7 nations as a group for comparison is arbitrary. It is possible to rank all nations by impact. While the G7 nations still appear in these lists, their impact often is less than that of comparatively smaller nations producing fewer papers. This is shown in Table 1, which lists the twenty highest-impact nations for 1981�90. It includes only those nations producing at least 10,000 papers.

Switzerland ranks first with an impact of 7.3, based on 66,000 papers that received 485,000 citations. Sweden and the U.S. rank second, each with an impact of 6.7. Denmark follows (6.2) and the Netherlands is next (6.0). Thus, four of the top five are comparatively small nations, in terms of absolute output. These smaller nations achieved higher impact than the largest producers�including the U.K., Germany, France, Japan, Italy, and Canada.

Field distribution and relative impact of Canadian papers

The data reviewed so far are on science as a whole. ISI �s databases can also provide insights into the output and impact in broad fields of science as well as narrow subspecialties. Before we review comparative impact data for Canadian science in different fields, let�s first examine how Canadian papers are distributed by field.

Figure 6 shows the field distribution of about 142,000 Canadian papers in the most recent five-year period, 1987�91. Keep in mind that these groupings have some redundancy�that is, overlap in journal coverage. For example, the Canadian Medical Association Journal is covered in both the life sciences and the clinical medicine editions of Current Contents. However, this overlap is consistent for all nations in the database.

Following the Current Contents journal groupings, the physical and chemical sciences include mathematics and the earth sciences. This group accounts for the largest share�33.5%.

The life sciences, with 21.2% of papers, include molecular biology, genetics, biochemistry, microbiology, cell biology, and so on. This group is followed by agriculture and biology (18.9%), which includes biotechnology, plant science, aquatic and environmental science, and clinical medicine (14.4%). Of course, all three of these fields could be considered as the life sciences in general, and they would account for 55% of Canadian papers. The last group�engineering and technology (11.9%)�includes materials science, optics, acoustics, mining, etc.

Figure 7 shows impact trends for Canadian papers in these fields. Canadian life sciences papers show the highest impact factors. Their impact increased by 8.7%, from 4.6 to 5.0 over the eleven-year period. However, clinical medicine papers showed the highest impact gain�a 10.7% increase.

This graph reinforces an important point that deserves to be stressed: impact factors vary by field and subspecialty. Typically, the life sciences have the highest impact. This is due, in part, both to the larger population of researchers and papers in this field, as well as the higher average number of references per life sciences paper, compared with other fields. Thus, when conducting citation evaluations of nations, institutions, departments, authors, or whatever, analysts ought to use field-specific output, citation, and impact averages to make valid comparisons and draw appropriate conclusions.

As noted earlier, while it is interesting to review absolute impact factors for a nation, it is perhaps more meaningful to compare them with groups of other nations. Figure 8 compares Canada �s impact versus the world average for each of these fields. The impact of Canadian papers consistently exceeded the world average in three fields�clinical medicine, engineering and technology, and the physical and chemical sciences.

As noted previously, however, the trend in a nation�s impact may be more meaningful than the actual percentage above or below the world average. For example, the impact of Canadian papers in clinical medicine, relative to the world average, increased by 9.2% from the beginning to the end of the period. It is reasonable to ask whether the growth of multinational clinical studies have some bearing on this trend. On the other hand, agriculture and biology papers declined in relative impact by 9.5%.

Of course, it is possible within each of these fields to compare Canada�s performance with that of other specific nations as well as the world average. This is illustrated in Figure 9, which shows impact trends, relative to the world average, for the G7 nations in the life sciences. The U.S. had a higher relative impact than the world average by 30 to 40%, and its impact increased by 3% over the eleven-year period. The performance of the U.K. was relatively unchanged, while Germany�s relative impact increased by 6.1%, more than any other nation shown here.

As noted earlier, Canada�s relative impact declined by 2.1%, shown by the dotted line. France improved by 5.7%, sufficient to overtake Canada in the most recent period, 1986�90. It is possible that the French improvement may be due to increased publication in English-language journals.

Highest-impact Canadian institutions

Having reviewed citation data at the national level, let�s move on to the next level of specificity�that is, institutions. Table 2 identifies the twenty highest-impact Canadian institutions in the life sciences. It includes institutions that produced at least 1,000 life sciences papers from 1981 through 1991. The boxed line of data shows the average impact of all Canadian institutions for the period 1981�91. Eight of the twenty institutions exceeded the national average in the life sciences. Three have impacts greater than 10�McMaster University (10.7), the University of Toronto (10.6), and Queen�s University (10.4).

A different ranking of universities is obtained for the physical and chemical sciences, as shown in Table 3. Ten institutions exceed the national average impact in this field. The only one with an impact higher than 7.0 is the University of Guelph (7.4), which ranked 19th in the life sciences. The University of Toronto is a close second (6.9), followed by the University of British Columbia (6.6) and the Université de Montréal (6.6).
 

Most-cited and highest-impact authors

One can also obtain impact rankings of authors as well as institutions, nations, etc. Table 4 identifies the twenty most-cited Canadian authors who produced at least twenty-five papers from 1981 through 1990. It also shows their current institutional affiliations.

Two authors received more than 8,000 citations�M. Cantin (8,530)and J. Genest (8,494), both of the Clinical Research Institute, Montreal. In fact, this institute dominates the list. Six of the ten most-cited authors are based there.

Another way to view authors is in terms of impact, or average citations per paper. Typically, impact rankings identify a different group than lists of the most cited or most productive authors, institutions, nations, or whatever. This is seen in Table 5, which lists the twenty highest-impact Canadian authors for papers published and cited from 1981 through 1990. Only those authors who produced at least twenty-five papers are included.

The highest-impact author is A.J. de Bold, University of Ottawa, with forty-three papers that received 3,700 citations for an impact of 85.5. De Bold also appeared on the list of most-cited authors, as indicated by the asterisk. Of the twenty highest-impact authors, only four also were among the most-cited.

Of the twenty authors listed, four were based at the Hospital for Sick Children, Toronto. The Ontario Cancer Institute and McGill University are represented by three authors each. Two authors were from the University of Toronto�H. Sonnenberg and N. Isgur.

The impact of these authors ranged from 30 to 85.5, which is five to fifteen times greater than the national average impact during this period (5.6).

Most-cited Canadian papers

Let�s now examine the most-cited Canadian papers from 1981 to 1992. Not surprisingly, the names of some of the authors reviewed on these lists will appear on these papers. Table 6 identifies eleven papers cited at least 500 times, which by my definition makes them Citation Classics (Garfield 1977). The first paper, with 1,665citations, includes two authors who were on the lists of most-cited and highest-impact authors�de Bold and Sonnenberg. As shown previously, de Bold received 3,676 total citations during this period, and Sonnenberg had 3,343. Thus, this single paper accounts for about half of their total citations.

This illustrates an important point that should be kept in mind in citation analyses of authors, departments, journals, and so on. That is, citation distributions tend to be "skewed." For example, a comparatively small proportion of papers may account for the majority of an author�s or journal�s citations. Similarly, a relatively small group of authors or research teams may account for the majority of an institution�s citations. And a nation�s total citation frequency may be concentrated in a small number of high-impact institutions.

It is obvious from the titles of these papers that all are related to topics in the life sciences. The interesting exception is the third paper shown here with 627 citations. It concerns decision-making and the psychology of choice. The life sciences tend to dominate undifferentiated citation rankings of papers, journals, authors, and so on. As noted earlier, average impact in the life sciences is due to the higher number of cited references per paper, compared with other fields. However, the much larger occurrence of high-frequency Citation Classics is due to the relative size of the fields.

Lest we overlook high-impact work in other fields, Table 7 identifies the most-cited Canadian papers in the physical, chemical, and engineering sciences. It lists eight papers published from 1981 through 1992 that received at least 250 citations. The second paper on the list is the most recent, a 1988 Physical Review B paper concerning superconductivity. Co-authored by Y. LePage, National Research Council of Canada, Ottawa, and colleagues at Belleore Research Laboratories, Red Bank, New Jersey, it received 392 citations in five years.

Two authors also appeared on the twenty highest-impact authors list: D.A. Vandenberg, University of Victoria, whose 1983 paper on star clusters and stellar evolution received 279 citations; and N. Isgur, University of Toronto, whose 1985 paper with S. Godfrey concerning mesons and quark models was cited 268 times.

In addition to identifying the most-cited papers from the past decade or longer timne periods, ISI�s data can also identify recently published papers that have achieved high impact. These so-called "hot" papers, accompanied by brief commentaries by the authors, are a regular feature in The Scientist, the biweekly newspaper for science professionals I founded several years ago (Garfield 1986).

Table 8 identifies the ten "hottest" Canadian papers published since January 1990. The first column of data shows the number of citations each paper received in the two-month period January�February 1993. The second column shows total citations from publication through February 1993.

The first three papers all include authors from the University of Toronto. In fact, the University of Toronto appears on seven of the ten hottest Canadian papers. The second paper on the list is interesting to note. A 1991 Nature paper by H.H.M. Vantol and colleagues, it concerns genetic cloning for a humati dopamine receptor. Another paper published in the same issue of Nature by essentially the same team of authors also appears on the list.
 

Mapping science by co-citation analysis

The difficulty of classifying papers by journal groups was illustrated earlier. To overcotne the arbitrariness of all manual classification systems, we long ago began using co-citation analysis to create a dynamic systetn for classifyitig papers. Co-citation analysis enables us to literally map the structure of research specialties and fields. These structural relationships emerge when co-citation analysis and multidimensional scaling methods are applied to our database (Small 1973; Small and Garfield 1985).

Explained simply, co-citation analysis involves tracking pairs of papers that are cited together in the source articles we index. With single-link clustering techniques, a cluster of research begins to form when the same pairs of papers are co-cited with many others. The co-cited papers in these clusters usually share some common thread. Indeed, papers at the "core" of these clusters turn out to be the foundation or key publications of a particular research specialty. They are highly cited by researchers currently working at the forefront of the specialty.

Figures 10�15 demonstrate this research front mapping capability of ISI�s new SciMap system. This special file includes all ISI-indexed papers published from 1989 through 1991 that were cited at least ten times. In effect, this current file identifies "hot" research front specialties.

Figure 10 shows a typical opening screen on the SciMap system. The SciMap system can be searched by author, address, journal, or title words. In this example, we start our mapping exercise by using the author dictionary. We�ve selected Y. LePage, a Canadian author working on superconductivity who was identified earlier in the list of most-cited physics and chemistry papers. By selecting the "OK" option from the menu of choices at the bottom of the box, the SciMap system plots this paper as the starting point of the cluster.

As Figure 11 illustrates, the SciMap program provides a bibliographic reference for the paper in the information box at the left�first author, journal, volume, starting page, year, paper title, and author address. In addition, the number of 1989�91 citations is provided on the top line.

The SciMap program also plots the LePage paper as a hollow circle and provides an identifying label. At this point, we simply select the "ADD ONE" option from the menu at the top of the screen to see what other paper is linked to the LePage paper by co-citation.
 
 

Figure 12 shows that the computer has retrieved the second paper in this growing cluster, a 1990 Physica C paper by J.M. Tarascon, Bell Communications Research Laboratories, Red Bank, New Jersey, and colleagues. As it turns out, LePage is a co-author of this paper. The paper received fifteen citations, of which nine also cited the LePage paper. The distatice between the circles reflects the co-citation "strength" of association between these papers. And the size of the circles reflects their relative citation frequency.

The next item added to the cluster is another paper by Tarascon with LePage as a co-author, shown in Figure 13. Published in the 1989 Physical Review B, it was cited ten times. It was co-cited with the LePage paper eight times, and co-cited five times with the first Tarascon paper.

In Figure 14, we see that the next item added to the cluster is a 1989 Physica C paper by E.A. Hewat, Centre for Nuclear Studies, Grenoble, France. It was cited thirteen times; in eleven instances the LePage paper was co-cited, and in four the 1990 Tarascon paper was co-cited.

We can continue to add new papers until the cluster is completed, as is shown in Figure 15. The SciMap program indicates that the cluster is complete now. That is, no other papers are linked with it by co-citation at the specified strength threshold. The last to be added was a 1989 Physica C paper by M.H. Whangbo, North Carolina State University, Raleigh, and colleagues.

In effect, the SciMap program has identified seven papers concerning various structural, magnetic, and other properties of superconducting materials. As stated previously, these papers can be considered the "core" or foundation papers on this topic. Of course, these are not the only papers relevant to superconductivity. Our database includes many other clusters related to the structural and magnetic properties of superconductors. But these papers are most closely associated with the original "seed" paper by LePage.

Conclusion

This concludes our presentation on what citation data can tell us about a nation�s scientific performance. The examples we�ve reviewed are intended to illustrate the range of perspectives that citation-based indicators can provide. Tracking trends in output and impact, identifying the highest impact papers or institutions or authors, and mapping the core publications on a given topic by co-citation analysis each provide a unique view on different aspects of Canadian science.

While citation data create new tools for analyses of research performance, it should be stressed that they supplement rather than replace other quantitative�and qualitative�indicators. And like other indicators, the appropriate and balanced interpretation of citation data requires the input of information specialists working with knowledgeable scientists.

Note All illustrations © Institute for Scientific Information.

References

back to textGarfield, Eugene. 1977 (January 3). Introducing Citation Classics: The human side of scientific reports. Current Contents no.1: 5-6. Reprinted in Eugene Garfield, Essays of an information scientist, Vol.3, 1�2 (Philadelphia: ISI Press, 1980).

back to textGarfield, Eugene. 1983a. Citation indexing: Its theory and application in science, technology, and humanities. Philadelphia: ISI Press. 274p.

back to textGarfield, Eugene. 1983b. How to use citation analysis for faculty evaluations, and when is it relevant. Part 1�Current Contents no.44 (October 31): 5�13; Part 2�Current Contents no.45 (November 7): 5�14. Reprinted in Eugene Garfield, Essays of an information scientist, Vol.6, 354�72 (Philadelphia: ISI Press, 1984).

back to textGarfield, Eugene. 1986 (July 21). Introducing The Scientist: At last, a newspaper for the science professional.Current Contents no.29: 3�6. Reprinted in Eugene Garfield, Essays of an information scientist: Towards scientography, Vol.9,222�25 (Philadelphia: ISI Press, 1988).

back to textGarfield, Eugene. 1990 (January 22). Announcing Science Watch: A unique newsletter that tracks trends and performance in scientific research.Current Contents no.4: 3�6. Reprinted in Eugene Garfield, Essays of an information scientist: Journalology, Key-Words Plus, and other essays, Vol.13, 25�28 (Philadelphia: ISI Press, 1991).

back to textGarfield, Eugene, and Alfred Welljams-Dorof. 1992. Citation data: Their use as quantitative indicators for science and technology evaluation and policy-making. Science and Public Policy 19: 321�27. pdf file

back to textSmall, Henry. 1973. Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science 24: 265�69.

back to textSmall, Henry, and Eugene Garfield. 1985. The geography of science: Disciplinary and national mappings. Journal of Information Science 11: 147�59.