The Journal Impact Factor is often perceived as a universal indicator of the quality of scientific publications. However, it actually reflects the average citation rate of articles in a journal rather than the scientific value of each individual paper. Why is an article published in a journal with a high impact factor not always influential? Which indicators should be considered for an objective assessment of a study? We explore these questions in our article.
What is the Journal Impact Factor and how is it calculated?
Impact Factor (IF) is calculated by Clarivate for journals included in the Journal Citation Reports (JCR) database.
The IF for a given year is the number of citations that the journal's articles from the two preceding years received in that year. This number is divided by the number of those articles.
The indicator was originally created to select journals for the Science Citation Index and to help libraries choose which titles to subscribe to. Its creator, Eugene Garfield, did not intend it as a way of assessing an individual piece of research.
Impact Factor is a journal-level average, not an article-level one
The indicator averages the citation rate of all the journal's articles over two years. But citations within a single journal are distributed very unevenly.
A small share of articles receives most of the citations and pulls up the journal's average. Most other publications are cited noticeably less often than the journal's IF suggests.
This limits the indicator's use for assessing a specific piece of work:
- represents an average across all the journal's articles over two years
- is built on a highly uneven distribution of citations between articles
- does not account for the methodology, novelty or reproducibility of a specific work
- can rise because of review articles, which on average are cited more often than original research
- is affected by self-citation and the journal's editorial policy
Why doesn't a high Impact Factor guarantee the quality of an article?
A link between a journal's impact factor and the actual citation rate of an individual article does exist. But it is usually weak, and its strength varies between scientific fields.
This is why even journals with a high IF publish articles that are barely cited. Conversely, journals with a modest IF sometimes publish work that attracts wide scientific attention.
Impact Factor is officially not recommended for assessing an article
In 2012, the San Francisco Declaration on Research Assessment (DORA) was adopted. It explicitly recommends not using a journal's impact factor as a substitute for assessing the quality of an individual article or a scientist's contribution.
The declaration has been signed by more than 25,000 scientists and research organisations worldwide – from universities to publishers and funding bodies. Signatories include the European Commission. This movement is known as responsible research assessment.
A similar principle applies at the level of national evaluation systems. In Poland, the a list of scientific journals approved by the Ministry of Science and Higher Education. Points in this list are also awarded to the journal as a whole, not to an individual article. The ministry is currently preparing an updated version of the list, so the specific criteria may change.
How to assess an article without relying on Impact Factor?
Scientometrics offers other approaches too. They focus on the specific piece of work rather than the journal as a whole, and complement expert review rather than replace it:
- Look at the actual number of citations the article itself has received, rather than the journal's average.
- Check whether the study's methodology meets the standards accepted in the discipline.
- Assess the reproducibility of the results and the availability of the underlying data, where relevant to the topic.
- Look at the content of peer reviews, if the journal publishes them openly.
- Take altmetrics into account – mentions of the article in professional databases and open discussions within the scientific community.
Impact Factor remains a useful reference point when choosing a journal to submit to. But its relevance is limited to the level of the journal as a whole.
Assessing a specific piece of work requires separate analysis – of its methodology, its actual citation record and how it is received by the scientific community. This approach gives a more accurate picture of an article's contribution than the journal's overall indicator.
Need help with publishing a scientific article? The specialists at Scientific Publications will help you choose a journal that matches your topic and the level of your research, and will support you at every stage of the publication process. To get a free consultation, fill in the form below and our manager will contact you shortly. Onwards to new scientific achievements!
