PREreview partnered with the non-profit organization Outbreak Science to develop open infrastructure to help researchers read, provide, and request rapid feedback on outbreak-related preprints.
We are proud to announce the beta release of Outbreak Science rapid PREreview (OSrPRE), an application for rapid, structured reviews of outbreak-related preprints.
What does Outbreak Science Rapid PREreview do?
Login with ORCID iD
Similar to PREreview, OSrPRE allows for anyone to read content without creating an account. However, if you want to contribute with a rapid review of an outbreak-related preprint or request one on a preprint, you need to create an account using ORCID credentials. To register, you first need to read and agree to our Code of Conduct and login via the ORCID API.
Anonymous/Public profile settings
Also similar to PREreview, you can set your profile to “Anonymous” if you wish your identity to be kept private, or to “Public” if you wish your identity to be disclosed to everyone.
We are currently trying out a slightly different anonymity model from PREreview, where you are able to fluidly switch between having your profile anonymous or public. There are controls in place that allow administrators to restrict users from "double dipping" on reviews and requests with their personas on the platform.
If a violation of the Code of Conduct is found (see below), depending on the severity of the violation, the review and/or the user will be removed from the platform regardless of their profile settings.
Requesting/adding a rapid review
A rapid review is a review consisting of a few, multiple-choice questions and two optional open-ended questions. The possible answers to the structured questions are “Yes”, “No”, “N.A”, and “Unsure”.
You can solicit feedback on a preprint from the community by simply adding the preprint’s digital object identifier (DOI) – or, if it’s a preprint from the arXiv server, its arXiv iD, and by then clicking on the “Add request” button. Similarly, you can submit a rapid review of a preprint by clicking on the “Add Review” button.
Aggregated rapid reviews visualization
The multiple-choice answers of each rapid review question are aggregated across all reviews and visualized in a bar chart.
The questions are formulated such that “Yes” refers to positive characteristics (e.g., data availability), while “No” refers to the absence of such aspects. The visualization is designed to quickly identify questions with numerous positive (blue) or negative (red) responses from community reviewers.
You can also choose to visualize rapid reviews from one reviewer or from a subset of the reviewers by dragging and dropping their icons in the reviewer filter box above the visualization. Each combination of rapid reviews authored by users has a unique permanent link associated with it so that you can share it with your network, should you desire to.
Are you into data visualization? If you have ideas about how to visualize the data, we’d love to hear them — reach out to us on GitHub, via the feedback form on the website, or email us at outbreaksci@prereview.org.
Web extension
OSrPRE is also available as a browser extension for Firefox and Chrome. This means that as you read preprints online, you can add reviews and request reviews directly on the preprint sites you visit without having to navigate to the OSrPRE homepage.
Code of Conduct violation reporting and content moderation
OSrPRE is a tool to facilitate constructive evaluation of preprints. As detailed in the Code of Conduct, constructive feedback is not a synonym for positive feedback, but it means expressing a concern or highlighting a possible issue with the content of a preprint in a way that is respectful and helps the authors improve the work.
If you come across a potential violation of the Code of Conduct, you can log in and flag the problematic content specifying the reason for reporting it as problematic (e.g., you identify derogatory comments in the optional open-ended answers of a given review, or you think that the review author may be an author of the preprint). Members of our team will review these reports and determine what action to take as a result.
Open API
All data are available programmatically via an open API with full anonymized datasets that can be obtained via replication. Data are available in JSON-LD, a lightweight Linked Data format.
The API also allows other platforms to integrate with OSrPRE, to add the ability to submit requests for reviews and to disseminate review content. This would allow a preprint server, for example, to enable preprint authors to solicit rapid reviews at the time of submission.
If you represent a preprint server or another entity interested in learning more about how to integrate your platform with OSrPRE, please visit the API documentation on our website and/or contact us at outbreaksci@prereview.org with any questions.
Give feedback
With the help of the user experience team at Simply Secure, we are conducting interviews to learn more about specific needs and test the platform’s workflow and features.
We would love to hear from you too!
We invite you to try out the platform and give us feedback on your experience by either emailing us directly at outbreaksci@prereview.org or by filling out this anonymous form, or participating in an interview.
Thank you!
What’s next?
As we collect feedback through the interviews, emails, and the anonymous form, we are making the last little tweaks to the platform in preparation for a full launch in January.
Next, we will focus on outreach to invite communities of researchers around the world to engage with rapid reviews and with each other on OSrPRE, stimulating discussions around emergent scientific outputs in the field of infectious diseases and public health.
We will also work on developing partnerships with other relevant platforms and stakeholders who share the goal of helping to speed up research dissemination and community engagement in peer review.
We have additional funding via the Wellcome Trust Diversity and Inclusion Research Enrichment to support further user research and community development with communities of researchers who are currently under-represented in scholarship. Stay tuned for more updates on this effort.
Acknowledgments
This is an open project funded by the Wellcome Trust Open Research Fund. This project is a collaboration between PREreview and Outbreak Science. Outbreak Science is a non-profit organization aiming to advance the science of outbreak response.
We thank Dr. Naomi Penfold for introducing us to Dr. Michael Johansson, Founder and Director of Outbreak Science and collaborator on this project.
OSrPRE was developed by Sebastien Ballesteros (GitHub) and Erik Wysocan (portfolio website), open source developers from the sci.pe team. We thank them for their professionalism, competence, and for building a platform that reflects our vision.
User research is conducted in partnership with Simply Secure. We thank Simply Secure’s Executive Director, Georgia Bullen for her guidance and support in making researchers the focus of this effort. We also thank all the researchers who generously volunteered their time to help us better understand their needs and improve the platform.
Finally, we thank our fiscal sponsor organization, Code for Science & Society, for their constant guidance and support.
Follow us on Twitter @PREreview_ and @outbreaksci.
