- Strengths and Weaknesses of Bibliometric Data Sources
Ranking organizations develop various strategies based on the bibliometric data sources and
scientific performance measurement systems they rely upon. However, every bibliometric data
source has its own unique strengths as well as notable limitations — none of them is flawless or fully
comprehensive. Understanding this fact is crucial in order to clearly explain our rationale for
preferring Google Scholar, and to challenge the widespread perception that other databases are
“absolutely correct” or “always superior.”
Some commonly used citation indexes and bibliometric-based analyses focus on a pool of
approximately 9,000 to 15,000 journals, selected through strict criteria. While the citation data these
systems provide is widely accepted for scientific impact assessment, their coverage is nonetheless
limited. Because they mainly emphasize publications in English and prioritize STEM fields (science,
technology, engineering, mathematics), research produced in social sciences, humanities, arts, or in
local/regional languages is often systematically underrepresented. For example, in some subfields of
the social sciences, the coverage ratio of these databases can remain as low as 5–20%, making a large
number of valuable studies essentially invisible.
In addition, scientific communication formats such as books, book chapters, and conference
proceedings — which are especially important in the humanities, law, education, or computer
science — often fail to find sufficient space in these systems. However, disregarding such
publications or systematically pushing them to the background is not compatible with the principles
of academic opportunity equality and respect for scientific diversity. Academic output cannot be
defined solely by a particular publication type or language; each discipline has its own knowledge
production culture and publishing traditions. For example, ignoring a book or a conference paper —
both of which are key academic communication tools in many fields — means disregarding a
legitimate and valuable part of scientific labor. Such an approach can undermine the scientific
visibility of certain disciplines or regions, leading to an unfair overall assessment. Yet all scientific
outputs enrich the collective academic knowledge; therefore, their exclusion or underrepresentation
should be questioned both ethically and methodologically. - Inclusivity and Accessibility in Bibliometric Evaluation
Another limiting factor is access costs. Traditional bibliometric data sources mostly operate with
high-priced subscription models. As a result, only well-funded institutions and researchers can
benefit from these databases, while scholars or universities with more modest resources cannot
access these services. Consequently, scientific performance measurement cannot be carried out
fairly on a global scale. Moreover, the lack of price transparency and uncertainty about what
subscription fees might be in the coming year creates a serious problem for sustainability.
In terms of coverage, most ranking systems built upon these databases encompass approximately
80–90 countries and are limited to around 1,500–2,500 institutions. Over many years, there has been
no significant increase in these figures. Such a narrow scope fails to adequately reflect the true
distribution of scientific production and academic visibility at the global level.
Additionally, persistent problems in standardizing institution names, author names, and affiliation
information over many years continue to cause serious consistency issues in the data. Many critics
have also noted that these systems fall short on matters such as publication ethics, peer review
processes, and fairness of coverage.
- Fairness and Inclusivity in Scientific Performance Measurement
In contrast, Google Scholar indexes any academic-looking content available on the internet as an
open-access and free platform. By covering different types of publications — including journal
articles, theses, books, reports, and conference papers — without regard to discipline or language, it
significantly improves the scientific visibility of academic production in social sciences, arts,
humanities, education, and local languages. Studies have shown that Google Scholar captures much
higher levels of citation data in these areas. Furthermore, by also tracking citations from books and
proceedings, it provides a more inclusive impact evaluation from the perspective of publication
metrics.
Another advantage of Google Scholar is that it continuously and rapidly updates its data, meaning
there is no specific “data freeze” date. This enables a more current, transparent, and openly
accessible structure for research evaluation processes. Any academic or institution can track their
own bibliometric data through free tools (e.g., Publish or Perish); this helps to reduce the access
inequalities created by paid databases and contributes to the democratization of bibliometric
knowledge.
Of course, Google Scholar also has errors. However, most of these errors are random, and there is no
evidence that they systematically favor certain people or institutions. Thanks to its open-access
structure, certain unethical behaviors (such as excessive self-citation or fabricated publications) can
be detected more quickly. Compared to the systematically excluded content in other databases,
Google Scholar’s broader approach offers a fairer and more meaningful indicator for scientific impact
analyses within similar contexts. - The Strategic Position of Google Scholar in Research Evaluation Methods
Our primary reason for preferring Google Scholar is that it allows us to reflect the scientific visibility
of researchers and institutions under fairer conditions, without discriminating based on geography,
language, or budget. At the same time, we openly acknowledge this tool’s limitations, and we strive
to minimize these weaknesses through multilayered data cleaning, continuous quality improvement,
and strong auditing processes. In addition, thanks to the high visibility provided by Google Scholar,
the awareness of individuals, institutions, and professional associations on this matter has increased
significantly; as a result, hundreds of thousands of researchers have focused on organizing their
profiles in a more careful and consistent manner. Simultaneously, a large amount of improper data
has been removed from the system through strict rules, thereby further strengthening overall quality
and reliability. Through this process, institutions have also become able to detect unethical practices
and faulty activities much earlier, gaining the opportunity to take necessary precautions in time.
Conclusion
No bibliometric data source is flawless or fully comprehensive. Each has its strengths and
weaknesses. Recognizing this fact is critical for all stakeholders who wish to develop a balanced and
inclusive approach to research evaluation, scientific impact analysis, and scientific performance
measurement on a global scale. Our reason for preferring Google Scholar is its ability to reflect the
scientific visibility of researchers and institutions under fairer conditions, without discrimination
based on geography, language, or budget. At the same time, we openly acknowledge its limitations
and try to minimize them through multilayered data cleaning, continuous quality improvement, and
strong auditing processes.
In conclusion, the idea that “a single bibliometric source is perfect” does not reflect reality. No data
source today is capable of perfectly reflecting the full diversity of global academic production on its
own. Bibliometric data sources continue to evolve with contributions from the academic community.
Therefore, the most accurate approach is to analyze the boundaries of each source well, interpret
the data appropriately in context, and establish a more fair, accurate, and inclusive scientific
measurement and evaluation system using complementary methods.
✅ Our Approach
- A global, practical, and inclusive methodology
- Robust auditing processes to mitigate the limitations of data sources (approximately 2 million
profiles reviewed and inappropriate ones removed) - Continuous data cleaning and updating for near-real-time, accurate, and up-to-date rankings
https://www.adscientificindex.com
