From a comparison of 1000+ references to 20 papers in four fields of astronomy (solar, stellar, nebular, galaxy), we found that the citation counts in Science Citation Index (SCI) and Astronomical Data System (ADS) agree for 85% of the citations. ADS gives 15% more citation counts than SCI. SCI has more citations among physics and chemistry journals, while ADS includes more from conferences. Each one misses less than 1% of the citations.

Astronomers now have two independent sources for citation counts to papers, namely the Science Citation Index (SCI) and the NASA Astronomical Data System (ADS). Do they give the same results?

The pioneering Science Citation Index was started in 1961 by Eugene Garfield at the Institute for Scientific Information in Philadelphia, PA to help people locate scientific papers by subjects and by authors in the rapidly-growing field of scientific publication (Abt 2003). However, a main use quickly became to collect citations and citation counts for individual papers, authors, institutions, and journals.

Pertinent to the following discussion, the SCI has several criteria for the inclusion of journals. One is that they must have a proven record of prompt publication. When the SCI wishes to close a year, it does not want to delay publication because of tardy journal issues. Another criterion is that each journal be often cited in the field. Of the estimated 100,000 serial publication in Ulrich’s International Periodicals Directory, only the most-used 8700 journals in all the sciences are listed in SCI and only 42 of those are in astronomy and astrophysics. The SCI was initially available in printed form only (roughly several meters of volumes per year) but is now available on-line by subscription as the “Web of Science.”

The other source for citation counts had a different origin. Many astronomers and institutions cannot afford the huge libraries needed for full scientific searches, but it is now possible to put volumes on-line. Drs. Guenther Eichhorn and Stephen D. Murray at the Smithsonian Astrophysical Observatory undertook the large job with NASA funding of copying onto a computerized base all of the major astronomical journals. It became the ADS (see Eichhorn 2005). Those reproductions are like photographic images called bitmaps; they are not like the on-line versions of  The Astrophysical Journal (ApJ) and Astronomical Journal (AJ) in which one can click on references in the text and they will appear in full on the screen. However, those bitmaps are now available for all of the major astronomical journals, many conference volumes, and some books; they are now copying many observatory publications. They make doing astronomical searches nearly as complete as in a major observatory library.

Furthermore, this service is free whereas the SCI is expensive and can be afforded only by large libraries. An added attraction of these on-line astronomical publications is that through optically scanning reference lists, the citations within this body of publications to any paper in the set is given.

Thus we have two sources (SCI and ADS) for obtaining citation counts for astronomical papers. However, they are based on different bodies of material. The SCI includes all the major journals in physics, chemistry, mathematics, and related sciences, while ADS is not that broad. SCIADSADS is working on those. However, few observatory publications are being published now so they are of decreasing importance. Increasing numbers of monographs (books) are being published so it becomes increasingly difficult to buy or scan them. A minor comment is that for Spanish names that include the mother’s maiden name as an apparent second family name, the SCI lists papers under the first one and ADS under the second one.

Some astronomers have access only to the ADS, while others have access to both the SCI and ADS. How do citation counts from the two compare? This study involves comparing citation lists for sample papers and provides lists of which citations are missing in each.

2. The survey
We copied lists for five papers in each of four fields: solar physics, stellar astronomy, gaseous nebulae and the ISM, and galaxies. The papers were generally the first five published in 1997 in the appropriate journals: solar papers in Solar Physics and the others in ApJ. I wanted to include other journals (AJ and Astronomy and Astrophysics) but most of them yielded too few citations in eight years to give statistically-valid results. Table 1 gives the four fields, five papers in each, and citation counts until February 2005 in SCI and ADS, and the citations in common for the two.

From the sums of the last three columns of Table 1 we see that ADS lists 15.0 ±4.2% more citations than ADS. From the first and third columns of numbers, we see that SCI and ADS have 85% of their citations in common (91% of the SCI citations and 79% of the ADS citations). Therefore at the 15% level of accuracy, the two sources give similar counts, but within that accuracy there are selection differences.

Now consider the 1044 - 953 = 91 citations listed in SCI but not in ADS. Our sample of 20 papers has a total of 28 journals that are not cited very often (maximum of 7 for Izvestiya Akademii Nauk Series Fizicheskaya) plus 32 citations to IAU Symposia. Those journals plus symposia are listed in Table 2. The numbers following the titles are the numbers of times they cited the 20 papers. In addition, there are 8 citations too recent for SCI and 3 errors in ADS.

Now consider the 1201 - 954 = 247 citations listed in ADS but not in SCI. Those are listed in Table 3, and are primarily conference papers. The differences between SCI and ADS counts would be nearly halved if SCI included ASP conference papers; like the criterion used to include IAU symposia, all ASP conference papers are published by the same publisher. Therefore users of these two sources of citation counts have to decide whether they wish to have included the many secondary physics and astronomy journals (in SCI) or the many conference papers (that are not refereed papers) in the ADS.

We found 10 journal papers listed in ADS but not in SCI. Those omissions in SCI are listed in Table 4. That omission of 10 papers relative to the 1044 papers found constitutes a 1.0% error, which agrees with the similar error found by Abt (2005) in a survey of Solar Physics papers.

We conclude that (1) SCI and ADS give the same citation counts for 85% of the papers, (2) that that percentage would increase to 92% if SCIADS has 15% more citations than SCI, and (4) the primary difference is that SCI includes citations from a wide variety of physics, chemistry, and mathematics journals while ADS includes many more conference papers.


TABLE 1. Citation Counts in SCI and ADS for 20 Sample Papers.
includes only the IAU symposia (because they are all published by one publisher), but not the IAU colloquia (which have a large variety of publishers and therefore are hard to find) or the Astronomical Society of the Pacific (ASP) conference series or most other conferences. includes only those conferences for which it has received printed volumes. In its process of scanning pages in an optical reader, it must physically cut the pages from the volumes and therefore destroy the volumes. Neither source includes observatory publications, although included ASP conference papers, (3) that ADS has 15% more citations than SCI, and (4) the primary difference is that SCI includes citations from a wide variety of physics, chemistry, and mathematics journals while ADS includes many more conference papers.