In August of this year, the Coordination for the Improvement of Higher Education Personnel (Capes) and Clarivate released another report from this company on Brazilian scientific production.[1]. The document, entitled Overview of Changes in Research in Brazil, was sponsored by Capes, the institution that takes care of all postgraduate studies in the country and which is the loci responsible for generating most of Brazil's scientific production. As we know, everything that Capes does, supports and disseminates has repercussions and signals research values for the country.
Hence the interest in knowing what the report shows.
This is an analysis of publications in the Web of Science (WoS), complemented by other databases, from Clarivate itself.
Basically, all the information in the report can be obtained by anyone who has access to these databases (and several organizations in Brazil have this access). WoS is certainly a global reference, as much as Scopus (Elsevier) and Dimension (Digital Science), or even open and free databases such as OpenAlex and Scielo.
A first observation is that these databases vary in coverage and some are more comprehensive than others. A second observation is that open and free access databases already exist and offer material that leaves little or nothing to be desired compared to closed and paid databases. The coverage of scientific journals varies greatly between them, and can reach double or triple, depending on the reference.[2]
The use of different sources, with important differences between them, often leads to different data and different interpretations. The idea that science is global should be true, but this is a vanishing point: it depends on the observer. As has been discussed and demonstrated in the specialized literature, there are themes, approaches, geographies, institutions and publishers that are “more global” than others.
Thus, the first thing the reader should be careful about is that the Clarivate/Capes report is based on a data section identified by someone as important. A frame that selects a certain angle, with a certain resolution. The document is far from (and could not be otherwise) offering a comprehensive and in-depth portrait of Brazilian research.
This warning should be on the cover page of the report.
The document provides updated information on some indicators known to the community. It also provides some original information that draws attention and sparks interest in future research, as this is the only way to understand what they mean.
According to the WoS scale, we are maintaining our 13th position globally (in number of publications), but the relative importance measured by number of citations (normalized by area of knowledge and period) has remained below comparative medians with the G7 (which is expected), with the BRICS (expected, but for some of them) and with Latin America (not very expected).
Still on this subject, the report, in Figure 26, analyzes Brazil's “performance” in ten “main” global citation topics. The figure shows that, of the selected WoS topics, six are classified as weaknesses and two as relative threat points. This leaves only two as the country's strengths: antibiotic resistance and coronavirus.[3]
The question remains: does this composition of information mean that we are becoming more productive (or maintaining production) and less relevant? Does this conclusion make sense? Are the data and analysis used the best way to look at the evolution of research in Brazil?
In terms of open access publications, according to the report, we are relatively stable, with a slight increase in Gold access, which means that the article is open, but a processing fee (APC) has been paid to the publisher. It is a pity that the report does not separate what is totally open and what is free of APC (known as Diamond access). According to the report, among 20 countries selected for having the largest scientific production in the world,[4] Open access rates range from 36% (India) to 84% (Netherlands), with Brazil recording 54%. The median for this group is 60%.
Another question: should we be publishing more in open access? Since several studies show that it has a greater impact, not only in terms of number of citations, but also and perhaps mainly in the diversity of those who read, mention, cite and use the knowledge?[5]
In international collaboration, measured by co-authorship with researchers from other countries, Brazil has made progress. From less than 30% ten years ago, we have now reached 38%, with 13% with the United States, 6% with the United Kingdom, 4% with Spain and Germany. The median for the same 20 countries is around 45%, ranging from 23% to 75%.
One more question: should we have made more progress in international collaborations? Both in total and in the diversity of countries and profiles of collaborators?
These questions lose clarity when we come to Table 15, which presents bibliometric indicators comparing Brazilian publications financed by national and foreign agencies.
Basically, the table leads the reader to believe that Brazilian research only has an impact when it is funded by foreign agencies. The differences are so great that one would expect a warning to the reader, such as:
"Attention: these data, presented as such, without considerations and without discussion, do not mean that Brazilian agencies finance research with less impact than foreign ones.”
I know this, of course, and the authors of the report know it too, but in times of abundant information and opinion, which flow swiftly and are freely used, one cannot be too careful about what one writes and says. Even more so when it comes to science.
The item titled “Collaboration and commercialization” uses three indicators as proxies: articles co-authored with industry, number of patents and citations of articles in patents. No data is on commercialization. We do not know where we stand in this scenario of the research landscape.
Here is a concrete example of how the data source can change the result of the analysis. According to the WoS scale, Brazil would have 1,5% of articles co-authored by companies between 2019 and 2023. A quick access (less than five minutes) to a competitor database shows 2,2% for the same period. The difference, considering the amplitude presented in Figure 12 of the report, is large and would change the country's position, as well as the global comparative conclusions.
The regional and state analyses shown in the report do not bring many surprises. In addition to the “usual suspects”, states outside the traditional geographic axis appear with indicators that attract attention. For example, the state of Amazonas appears well positioned for indicators such as international collaboration and articles among the 10% most cited.
Another question: what does this mean? Certainly, we have good researchers in that state – as we do in all of them. But what else do we need to know to design better-informed policies for the Amazon? There are currently several programs focused on the Amazon that could benefit from good evidence produced by research and research on research.
Finally, the report shows a map of Brazil with the ten universities with the highest production of articles in environmental sciences.[6] Interesting! It would be illustrative to know, in addition to the total number of publications per university, the number relative to the size of the university (for example, per professor/researcher).
The report, therefore, provides indicators and analyses that are worth more for what they raise doubts than for what can be concluded from them.
Bibliometrics is an important, useful and powerful tool, which is why it should be used with care, so as not to induce hasty analyses and conclusions that could potentially cause damage to the country's science.
Capes, which has been rightly changing its model for evaluating postgraduate programs, including quality indicators beyond the number of publications, citations and international collaborations, in line with what has been happening in the world, certainly knows the limits and scope of the report.
As a national reference for Brazilian research, it is important that the report be accompanied by a warning that much more needs to be done to compose a portrait – and a panorama – of Brazilian research. After all, everything that Capes does, supports and disseminates has repercussions and signals values for research in the country, right?
Our own ability to design and analyze the situation of Brazilian research can and should go further.
We continue in search of a contemporary panorama and, above all, a clear photo.
This text does not necessarily reflect the opinion of Unicamp.
[1] https://www.abcd.usp.br/wp-content/uploads/2024/08/Relatorio_panorama_da_pesquisa_brasil_clarivate-capes-agosto-2024.pdf
[2] For those who want to examine comparisons between scientific databases: https://arxiv.org/html/2401.16359v1 e https://openalex.org/about. For magazine coverage: https://link.springer.com/article/10.1007/s11192-021-03948-5 e https://link.springer.com/article/10.1007/s11192-024-04948-x?fromPaywallRec=false
[3] The report also presents research topics in which Brazil stands out, in Tables 11 and 12. There are ten topics in each, bringing together all areas of knowledge. It is known there that “Zika Virus, Liquid-Liquid Phase Separation”, “Galaxy: Kinematics and Dynamics, Pyroptosis”, “Classification, Corporate Social Responsibility” and “Forest Restoration, Knowledge Graph” are topics in which Brazilian research has weight in the world. Very good, but what does this mean?
[4] Saudi Arabia, Switzerland, Netherlands, United Kingdom, France, Australia, Canada, Germany, Spain, Italy, United States, Poland, Brazil, Japan, South Korea, Iran, Russia, India, Turkey and China.
[5] See, for example: https://www.science.org/content/article/open-access-papers-draw-more-citations-broader-readership.
[6] An important topic, without a doubt, but which one isn't?
