This is a project aimed at measuring, disseminating, studying and fostering scientific consensus on matters of scientific and social conversy surrounding the COVID-19 pandemic.

By collecting and displaying anonymized responses from invited experts it aims to meet multiple objectives, including:

• learn what published researchers are thinking about controversial topics in which they are competent
• inform the public about the current scientific consensus on contentious issues
• where consensus is high:
  • fight the spread of conspiracy theories and disinformation
• where consensus is low:
  • divulge a more nuanced understanding of complex problems
  • allow experts to connect and discuss in a moderated forum, fostering scientific consensus and progress
  • watch how opinions change over time
  • study the nature, causes and diversity of scientific opinions
  • learn how to help scientific progress by fostering scientific consensus

An example of a preliminary analysis can be found in this LSE blog post .

To track the level of consensus, it uses a simple measure of "proportional entropy explained":

where H(Y) is the Shannon entropy (information content) of the distribution of answers Y (e.g. "none", "little", etc. but excluding "don't know"). This is a simplified version of a K function that has been elsewhere proposed as a general metric of knowledge.

Consensus is full when k=1, and all respondents give the same answer, whatever that answer is. Conversely, k=0 means that all answers are equally likely - in other words, we have no idea what any one thinks.
Note that this measure is most useful as the sample of respondents grows larger - because the probability to disagree increases. Conversely, if only a single opinion is taken, then consensus will be trivially maximal (there no possibility of disagreement).

How is the data obtained?

• A search in the recent literature identifies the names of authors of peer-reviewed articles that are relevant to a particular topic. Selection criteria and the list of invited authors are available on the website, to ensure full transparency of the selection process.
• These authors are sent an invitation email that includes a unique key, known only to them, which allows them to cast one vote and to write down their motivations.
• Votes and comments are collected and displayed with no identifying information. The total anonymity prevents any personal concerns about expresssing controverisal opinions.
• Votes and comments are moderated before being displayed, to prevent any form of abuse of the platform (e.g. profanity etc.). But absolutely no other filetring takes place.
• Whenever a voter changes opinion, they can post a new vote, which replaces the old one, and they can add new comments. They can also withdraw from the project and, apart from just not voting, they can ask to have their data removed from the list of invited experts.
• Members of the public can view and explore, on the website, how consensus varies across various categories and over time. The data is updated every few hours, and presented in various graphs (e.g. aggregated as total, by country of affiliation, by discipline, or over time) only to the extent that it does not compromise the anonymity of voters.
• Members of the public can also of course view all names of invited voters, and are welcome suggest the names of individuals or categories of experts to be invited (for example, other keywords to include in the literature search), or new questions they would like us to ask. Such decisions are at discretion of the PI, but will be communicated and displayed transparently.

This project is a creation of Daniele Fanelli

I am a methodologist and meta-scientist, currently a Fellow in Quantitative Methodology at the London School of Economics and Political Science .

Most of my research is devoted to understanding and quantifying the nature of knowledge , differences between disciplines, and the prevalence and causes of bias and misconduct in science. I am a member of two national research integrity committees (CNR in Italy and LARI in Luxembourg) and I have been involved in numerous projects and policy reports to improve scientific practices.

I am increasingly concerned by the polarization and politicization of scientific debates and the propensity of social media to discourage or even actively suppress dissenting but honest opinions. With this project, I hope to contribute to a solution. You can read more about me at my personal website, check the webpages of the wonderful Department of Methodolgy, and please do feel free to contact me.