In a nutshell
The PREreview platform is a free and open-source website for the crowdsourcing of preprint reviews (PREreviews). It's where we hope our community members will find a welcoming, resourceful space to compose, read, endorse and comment on PREreviews.
We do not host the preprints as we are not a preprint server. Rather, we fetch the metadata related to preprints which have a findable digital object identifiers (DOIs) on CrossRef and Google Scholar, and we display either the full pdf (when available under open license) or the abstract alongside the PREreviews.
As a registered user (PREreviewer), you are able to:
- Switch between Public and a unique Anonymous Persona for any public action your perform on the platform
- Request a PREreview on any preprint with a registered DOI
- Fill out a rapid PREreview—a structured form of 12 yes/no/not sure/n.a. questions designed to capture the essence of the preprint
- Compose a full PREreview—a text review report that upon submission is assigned a DOI via Zenodo
- Invite a co-reviewer, another PREreviewer who can write a full PREreview with you and be recognized as a co-author
- Invite a mentor, another PREreviewer who can give you feedback on your full PREreview draft and be recognized as mentor
- Comment on others’ full PREreviews
- Endorse others’ full PREreviews via Plaudit
- Report others’ full PREreviews as violating our Code of Conduct
- Join communities of like-minded reviewing groups
- Start your own community
As an unregistered user, you are able to:
- Search for preprints that have PREreviews or requests for PREreviews on the site
- Read and sort through aggregated rapid and full PREreviews alongside the preprint pdf (if available)
- Search through PREreviewers and view users’ profiles and contributions to preprint reviews and scholarship
PREreview provides a RESTful API documented using the OpenAPI v3.0 standard. Using this API and simple web requests, users have access to all of the same data and functionality as is exposed by the PREreview website. Read more here.
Links to rapid and full PREreviews are displayed on bioRxiv, medRxiv, and Europe PMC infrastructure.
The PREreview platform is an open-source web application available under the MIT License. In order to contribute to the development of the software, visit our Github repository.
In detail
Log in / Sign up
Login and signup to PREreview.org is done via the ORCID Public API. That means that to create an account and log into the platform you will need to use your ORCID credentials. Via the API, we will then import basic information you made publicly available on your ORCID profile to your PREreview.org profile—e.g., your name, your ORCID iD, your biography, your list of publications, and your publicly shared email address(es).
Onboarding
As you first sign up to PREreview, you will be guided through an onboarding process that will give you a quick overview of the site and tell you what you should expect by joining our community.
It is very important to us that users understand what they are agreeing to when creating an account on PREreview. Thus, explicit consent is part of our design.
Set up profile your Personas
Once you have completed the signup process, you’ll land on your profile page. Every PREreviewer will have access to two profiles or personas: an Anonymous and a Public persona. In your profile page you will be able to select which persona you wish to be "active".
What is an Anonymous persona?
An Anonymous persona is a user profile that we create for you at the time of signup combining a random color with a random animal (e.g., Grey Platypus). The Anonymous persona is linked to your Public persona on the back end, but to the public it appears as a separate user profile.
The website display of your Anonymous person's profile will only list your Anonymous persona’s name, your contributions to PREreview using this persona, as well as the communities you may have joined and badges you may have received with this persona. It will NEVER show any activity or information about your real identity and contributions made with your Public persona.
To keep your Anonymous persona private, we recommend that you never share your Anonymous persona's name with anyone.
What is a Public persona?
A Public persona is a second user profile that is created for you at the time of sign up. This profile will contain basic public information imported directly from ORCID, such as your full name, ORCID iD, biography, public email address(es), and list of publications, if any. It will also display your contributions to PREreview signed with your Public persona.
Why two personas for each account?
The two-personas account is a design response to the need for PREreviewers, particularly those belonging to vulnerable communities, to contribute to open preprint review without fearing retribution or judgment that may occur when their full name is attached to the feedback, whilst retaining an element of accountability.
Unconstructive and disrespectful feedback as well as any other violations of our community Code of Conduct will lead to consequences regardless of the persona activated and used by a PREreviewer.
PREreviewers who decide to contribute using their Anonymous persona may request an authenticated report of their contributions to the site to share with third-parties such as potential employers and in grant applications.
The Anonymous persona will be active by default. This means that if you perform any action that requires a registered account before modifying the settings—e.g., publish or comment on a PREreview—you will do so with your Anonymous persona’s name.
If you wish to switch to having your Public persona active—which would lead to having your real name associated with your PREreviews and comments done in that instance—you need to select the Public persona at the top of your profile page. This action can be performed at any time, allowing you to switch between the two personas as you feel appropriate.
Email settings
At the time of signup, any email address made public on your ORCID profile will be automatically imported to your PREreview public persona profile. If no publicly available email address is found on your ORCID profile, the email field on your PREreview profile will be empty.
While we do not require an email address to complete an action on PREreview, we strongly encourage you to provide us with one so that we can contact you with rare but important platform updates and communicate with you for any issues we may find with your account.
If you choose to enable notifications associated with any email address you provide, we will notify you if a PREreview is done in response to your request. This feature may be particularly useful if you are the preprint author, are looking for feedback from the community and you want to know when that feedback lands.
Request PREreviews
Only registered users can ask for feedback on a preprint from the general community.
We have two methods for doing this:
1) If the preprint is not already in the system, click the red Add PREreview | Add Request button on the platform page, and in the dialog box paste in the DOI of the preprint you wish to request a PREreview for (e.g., 10.1101/2021.04.12.439490 or the ArXiv ID, e.g., arXiv:2105.03902, if the preprint is published on the preprint server ArXiv). The system will look for the preprint fetching the metadata from CrossRef and Google Scholar. Once the title of the preprint you are expecting loads, click Request PREreviews.
2) If the preprint is already in the system as it has been already PREreviewed or has other requests, click on the title of a preprint listed on the main search and from the split screen page, click on the Add Request tab. Here you can click on the red Add a request for PREreview button to complete your request.
Fill out a rapid PREreview
A rapid PREreview is a structured review made up of 12 Yes/No/N/A/Unsure questions designed to capture the essence of a preprint.
The questions are formulated such that “Yes” refers to positive characteristics (e.g., data availability), while “No” refers to the absence of such aspects. Aggregated rapid PREreviews authored by individual PREreviewers are displayed in such a way that a reader can quickly identify questions with numerous positive (blue), negative (red), N/A (black), or unsure (grey) responses.
To fill out a rapid PREreview on your preprint of choice, from the main search page click on the red button Add PREreview | Add Request, and in the dialog box paste in the DOI of the preprint you wish to request a PREreview for (e.g., 10.1101/2021.04.12.439490 or the ArXiv ID, e.g., arXiv:2105.03902, if the preprint is published on the preprint server ArXiv). The system will look for the preprint fetching the metadata from CrossRef and Google Scholar. Once the title of the preprint you are expecting loads, click Add PREreviews.
You’ll land on a split screen where the preprint title and abstract or the full pdf, if available, will visualize on the left, and the empty rapid PREreview will appear on the right.
Once all questions have been answered, declare your Conflicting Interests if any, and click Submit. You’ll then be prompted to write a full PREreview.
Write a full PREreview
A full PREreview is a free text review of a preprint. PREreviewers are invited to write any feedback they may have as long as it is constructive. This can vary from a few sentences, to a long report as one that would be requested by a journal editor in a journal-organized peer review process. For some tips on how to write a constructive full PREreview, check out our Resource Center.
Our collaborative editors allow for simultaneous editing of the PREreview. That is because full PREreviews can be composed by an individual PREreviewer or in collaboration with others (see below to learn how to add a co-reviewer and/or a mentor).
Before submitting your review, don't forget to ensure your feedback is constructive and does not violate our community code of conduct. You also get another chance to declare any competing interests you or your co-reviewers may have.
Once submitted, the full PREreview will be assigned a digital object identifier (DOI) and published on Zenodo under CC-BY 4.0 license, thus becoming part of the scholarly record.
Invite co-reviewer(s) and mentors
A co-reviewer is another PREreviewer who is invited to join in writing the full PREreview.
A mentor is another PREreviewer who is invited to give you feedback on the PREreview before you submit it.
To add a co-reviewer or a mentor, click the Save button to save your full PREreview draft and then click on the + Add co-reviewer(s) button. A dialog box will open giving you the option to search users by PREreview persona name or ORCID. Once you found the PREreviewer(s) to invite, click Send invitations and the PREreviewers will receive an email inviting them to collaborate with you on the full PREreview draft.
Load full PREreview template(s)
Directly above the collaborative editor for your full PREreview is a Load Templates button. If you are part of a community, your moderator may have pre-loaded templates for you. If no templates are showing, then you are welcome to use the templates found in our Resource Center (select “PREreview Templates” in the left-hand side bar to view options).
Access full PREreview drafts
When you Save a full PREreview draft you can quickly find it to return to it by clicking User Drafts under your profile picture or avatar. Here you will also find any preprint draft invites that have been sent to you from co-reviewers.
Read PREreviews
Everyone who lands on our platform can read PREreviews. To do so, from the main search page click on the title of a preprint linked in our system that has either a rapid PREreview and/or a full PREreview. You can see the number of reviews in red below the listed preprint server and associated DOI.
By clicking on the title, you should land on the Read PREreviews page in a split screen format showing either the aggregated rapid PREreview results or a single rapid PREreview if that is all that has been filled out.
Scroll down to the full PREreview section and click Read more to read the full text.
Endorse reviews via Plaudit
Plaudit.pub is a light-weight plugin that links researchers, identified by their ORCID, to research they endorse, identified by its DOI.
We want a culture in which we are encouraging each other to provide constructive feedback to authors. When there is a full PREreview that you appreciate or want to give praise to, then use Plaudit to endorse the PREreview.
Report Code of Conduct violations
PREreview is committed to supporting and nourishing a healthy and harassment-free community of PREreviewers in which everyone feels safe to express their feedback honestly and constructively.
While the content PREreviewers decide to publish on the platform is not moderated prior to its publication, our team reserves the right to remove content that violates the Code of Conduct and, depending on the circumstances, temporarily or permanently block the account of the offender who will not be able to rejoin the PREreview family and any communities they were connected with via the same ORCID iD.
If you experience or witness a violation of the Code of Conduct, you are invited to report it by either clicking the red flag Report button on full PREreviews, by emailing us at report@prereview.org, or by submitting an anonymous report via this form.
All complaints will be taken seriously, reviewed and investigated, and will result in a response that is deemed necessary and appropriate to the circumstances.
Comment on PREreviews
Under each full PREreview there is an option to add a comment. The comment box can be used by authors, for example, to respond to the PREreview.
To add a comment to a full PREreview, type in your text in the comment box, and click on Comment.
PREreview Communities
One important innovation of this latest version of the PREreview platform are communities.
PREreview communities are groups that form around an identity, research topic, an institution, a laboratory, a journal club, or program. Examples of PREreview communities are Outbreak Science and Open Reviewers.
Each community has one or more moderators. Moderators can add new members, remove members, add events to the community calendar, and set up other basic settings for the community such as full PREreview templates.
Our first community on PREreview is made up of 597 PREreviewers who came together to rapidly review outbreak-related preprints. This community was migrated from what was once Outbreak Science Rapid PREreview, a separate platform that was developed in collaboration with Outbreak Science and whose code has now been archived.
Start your community
To start a community, from the Communities page click on Start Your Own Community. You will land on a Google form where you can fill out some basic information about the community you want to start. We will reach out to you promptly with more information about what to expect and the moderator's responsibilities.
Manage and moderate your community
After receiving your inquiry via the google form, our staff will set up your community and add you (or whoever was identified as such in the form) as the moderator.
Community moderators have access to community settings via the gear icon in the community page.
Under community settings, you can:
- Add a banner image
- Add a community description
- Add a Twitter handle and/or #hashtag related to your community
- Add additional moderator(s) and new community member(s)
- All additional people must be registered users, aka PREreviewers
- Add event(s)
- Events can be live-streamed preprint journal clubs, meetup calls, or any other event you want to get your community members involved with. Events get displayed on the main page and can be private (only visible to community members) or public.
- Add full PREreview templates
- Templates are suggested preprint review formats for your community members to select and use as they write their full PREreview
A moderator’s responsibility:
Your primary responsibility as a moderator is to make sure the PREreviews coming through your community are aligned with the PREreview’s Code of Conduct. Our platform is a tool to facilitate constructive evaluation of preprints. As detailed in the Code of Conduct, constructive feedback is not a synonym for positive feedback, but it means expressing a concern or highlighting a possible issue with the content of a preprint in a way that is respectful and helps the authors improve the work.
Instances of abusive, harassing, or otherwise, unacceptable behavior may be reported directly to the PREreview leadership team by emailing contact@prereview.org, or by filling out this anonymous form. The PREreview leadership team will consult with you to determine what action to take as a result.
Open Code
The PREreview source code is openly available under MIT License on our organization's GitHub. To report a bug, ask for help, or any other platform-related concerns, please email us at engineering@prereview.org.
Open API
PREreview provides a RESTful API documented using the OpenAPI v3.0 standard. Using this API and simple web requests, users have access to all of the same data and functionality as is exposed by the PREreview.org website.
A basic overview of the API's functionality is provided here (with examples using the common open-source command-line utility cURL), while the OpenAPI specification is available in JSON format is available here and detailed, automatically generated documentation of all available endpoints and methods is available here.
Acknowledgments
The design and development of this platform has received funding from the Sloan P. Foundation, from Mozilla, and the Wellcome Trust.
The latest version of the PREreview preprint review platform was developed by Josh King, Rae Gaines, and Harum Helmy, open-source developers from the Throneless Tech team. We thank them for their perseverance, professionalism, competence, and for building a platform that reflects our vision.
Design guidance was provided by Simply Secure. We thank Simply Secure’s Executive Director, Georgia Bullen for her knowledge and support in making the users the focus of this effort. We also thank Simply Secure’s intern, Carrisa Yao for her design work including the brilliance in Figma that communicated PREreview’s vision and guided the Throneless Tech team in the design of the platform.
Finally, we thank our fiscal sponsor organization, Code for Science & Society, for their constant guidance and support.
Follow us on Twitter @PREreview_ and sign up to our quarterly newsletter.
For any questions or concerns, don't hesitate to contact us at help@prereview.org. Thank you!