Impact Metrics

This guide includes traditonal and more recently developed metrics such as alt-metrics which can be used to determine research impact.

Research Impact

  • h-index: The h-index for a researcher is the number of papers with citation number ≥ to h (Hirsch, 2005). An h-index of 25 would mean that the author has published 25 papers that have been cited at least 25 times each. An advantage of the h-index is that it is very easy to calculate but it does not account well for a few highly cited papers. The h-index has also been used for journals. Recently a future h-index has been proposed.
  • g-index: The g-index has been proposed as an improvement of the h-index as a measure of citation performance. For a set of articles "ranked in decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received (together) at least g2 citations" (Egghe, 2006).
  • e-index: The e-index has been designed to correct for problems of excess citations and low resolution (failing to differentiate two researchers with the same h-index) (Zhang, 2009).

Journal Impact

  • Journal Impact Factor: There are various measures of a journal's impact or relevance but probably the best known measure is the Impact Factor which can be obtained from Journal Citation Reports. It is the number of cites in a given year (e.g., 2011) to articles published in the two preceding years (e.g., 2010 and 2009) divided by the number of published articles in that same time period (2010 and 2009). It should be used to compare relative citedness of various journals within a given subject area. Journal Citation Reports provides two and five year journal impact factors, immediacy index, cited half life, and Eigenfactor Metrics.
  • Eigenfactor: Provides Eigenfactor and Article Influence scores. It adjusts for citations differences among subjects, uses five year citation history, and also provides price information.
  • Journal Cost Effectiveness provides price per citation and price per article information.
  • Source-Normalized Impact Per Paper (SNIP): SNIP is freely available and is based on data from Scopus. It allows for the comparison of journals from different subject areas where citation patterns may be different. A journal's subject field is taken to be the papers citing that journal. SNIP is "the ratio of the journal's citation count per paper and the citation potential in its subject field" (Moed, 2009). SNIP uses a three year citation window.
  • SCImago Journal Rank (SJR): The SJR indicator measures the scientific influence of the average article in a journal, it expresses how central to theglobal scientific discussion an average article of the journal is." Citations are weighted so that a citation from a highly cited journal counts more than a citation from a poorly cited journal.
  • Usage Factor: The Usage Factor for journals is being developed from usage statistics as a measure of impact and quality (in draft).

Price Information and Economic Studies

Altmetrics Tools

  • Impact Story: aggregates altmetrics from numerous sources and generates a single report.
  • ReaderMeter: adapts the H-index and G-index for bookmarks instead of citations. It uses readership data from Mendeley.
  • PLoS Impact Explorer: collects discussions related to specific articles from Twitter, Facebook, science blogs, mainstream news outlets and other sources, and facilitates browsing or searching and filtering of the data to monitor the attention that an article is receiving (several pricing plans, free bookmarklet).
  • PLoS Article Level Metrics:: a suite of measures that can be used to track the research impact of individual articles (instead of journals) over time. PLoS provides usage (downloads), citations, social networking activity (e.g., social bookmarks, facebook, twitter), blogs and media coverage,and readers' comments for articles published in its journals.
  • PaperCritic: uses Mendeley API and allows users to rate and review articles in a publication library.

Citation Tracking

  • CitedIn uses the PubMed identifier to find where a specific article has been cited, e.g., blogs, databases, Wikipedia.
  • Publish or Perish is a software program that must be downloaded and installed on a computer. It allows researchers to provide evidence of their research impact. Citations are obtained from Google Scholar. Besides basic statictics it calculates H-index, G-index, and E-index, among others.
  • Scholarometer is a browser extension that queries Google Scholar. Network visualizations are based on crowdsourced discipline annotations of the queried authors.
  • Web of Science provides times cited data for indexed articles.
  • Scopus (not subscribed) also provides times cited data for indexed articles.
  • Google Scholar and CiteSeerX: In addition to the Web of Science and Scopus, citation counts for articles can also be obtained from Google Scholar and CiteSeerx. Google Scholar Citations also allows authors to keep track of citations to their articles.

Further Reading

Aarssen, L. W. et al. 2008. Bang for your buck: rejection rates and impact factors in ecological journals. Open Ecology Journal 1: 14-19.

Abbott, A. et al. 2010. Do metrics matter? Nature 465: 860-862.

Acuna, D.E., Allesina S., and Kording K.P. 2012. Future impact: Predicting scientific success. Nature 489: 201–202.

Bollen, J. et al. 2009. A principal component analysis of 39 scientific impact measures. PLoS ONE 4: e6022.

Colledge, L. et al. 2010. SJR and SNIP: two new journal metrics in Elsevier's Scopus. Serials 23(3): 215-221.

Cronin, B. 1984. The Citation Process: the Role and Significance of Citations in Scientific Communication. London: Taylor Graham.

Egghe, L. 2006. Theory and practise of the g-index. Scientometrics 69: 131–152

Falagas, M. E. and Alexiou, V. G. 2008. The top-ten in journal impact factor manipulation. Archivum Immunologiae et Therapiae Experimentalis 56: 223-226.

Good, B., Tennis, J. and Wilkinson, M.2009. Social tagging in the life sciences: Characterizing a new metadata resource for bioinformatics. BMC Bioinformatics 10(1) doi:10.1186/1471-2105-10-313.

Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 102(46): 16569–16572.

Krauss, J. 2007. Journal self-citation rates in ecological sciences. Scientometrics 73: 79-89.

Krell, F-T. 2010. Should editors influence journal impact factors? Learned Publishing 23: 59-62.

Oppenheim, C. and S. Renn, S. 1978. Highly cited old papers and the reasons why they continue to be cited. Journal of the American Society for Information Science 29(5): 225–31.

Priem, J. and Hemminger, B. 2010. Scientometrics 2.0: Toward New metrics of scholarly impact on the social Web. First Monday [Online], Volume 15 (7)

Thelwall, M. 2008. Bibliometrics to webometrics. Journal of Information Science. 34(4):605

Van Noorden, R. 2010. A profusion of measures. Nature 465: 864-866.

Zhang, C.-T. 2009. The e-index, complementing the h-index for excess citations. PLoS ONE 5:(5)e5429.