Project Summary
PREreview Open Reviewers is a cohort-based and interactive peer review mentoring program that empowers early-career researchers (ECRs) to contribute to scholarly peer review in a way that is rewarding to them.
Our long-term goal is to make this program accessible and available to anyone completely free of charge, building on the model established by Mozilla Open Leaders. However, for the pilot that we aim to launch in October 2020 (see roadmap below), we will focus on a small group of selected participants so that we can gather feedback to improve the program and scale it to global communities.
Why a peer review mentoring program?
The current system of scientific peer review is flawed. Research is evaluated by a handful of unpaid reviewers selected by journal editors as “experts” in the field. Two or three reviewers are chosen opaquely - often based on personal or established connections - to decide the fate of a research article behind closed doors. Paradoxically, while peer review is a key component for scientific dissemination, very few scientists receive any formal training prior to their first invitation to review.
Today, in the context of a global emergency that is affecting millions of lives, scientists are recognizing the power of collaborating and opening up discovery in ways that have not been seen before. The SARS-CoV-2 pandemic has led to an unprecedented uptake of preprints (more than 3,000 are published as of April 13), with researchers from all over the globe collaborating and sharing information at record speed. We need to mobilize the entire research community to provide timely feedback to these fast-emerging research outputs now more than ever.
To expand and diversify the pool of peer reviewers we need to tap into communities of ECRs who are currently either not engaged in peer review, or who are via their supervisors but in ways that often go unrecognized.
With this program we hope to achieve the following goals:
- Build a community of trained ECRs that can enrich and diversify the current pool of peer reviewers.
- Offer equal opportunities for ECRs to access mentorship-based peer review training focused on promoting diversity and inclusion in scholarship.
- Create opportunities for ECRs to network with peers and more senior researchers working in their field worldwide.
- Increase awareness around preprints through innovative and open ways of doing peer review.
During the program, mentees will:
- Be paired with a mentor who will help them set up goals and expectations, as well as guide them in the writing of 2 preprint reviews;
- Produce 2 edited preprint reviews that will be published on PREreview with a digital object identifier, allowing mentees to build their peer reviewing profile. Although we will encourage mentees to author their reviews openly, we will also provide a way for mentees to author their reviews anonymously if they are not comfortable with signing their work;
- Meet peers from other parts of the world with diverse cultural and professional backgrounds;
- Make connections with journal editors and more senior researchers in their field of research.
Curriculum outline 📖
Over the course of 14 weeks, ECRs with little to no experience in peer review will be paired with more experienced reviewers and engage in community calls, peer-to-peer support chats, and one-to-one mentor/mentee calls.
The curriculum focuses on the critical evaluation of emerging scientific literature, strategies to write constructive reviews, and helping mentees recognize and challenge their own biases. In addition, we will provide opportunities for peer-to-peer support, and mentorship and leadership training for those who wish to be mentors in future cohorts, an effort that will help make the program grow in impact and reach.
Below is a high-level map of the curriculum as it is developed so far:
Week 1: Introduction Call + Ethics and Bias in peer review (cohort + guest speaker)
🎈🎈🎈Welcome to all mentees and mentors!!! Overview of the program's syllabus, goals and expectations, ✋🏽 ⚖️ conflict of interest, plagiarism, anonymity, confidentiality, ethics and bias in current models of peer review
Week 2: Introduction Call for Mentor + Mentee (1:1)
🤝 🎯 Get to know each other and set your goals and expectations for the program
Week 3: Constructive Criticism & Language (cohort + guest speaker)
🙌 🌐 What do we mean by "constructive" criticism? Implications of language and cultural background in peer review
Week 4: The review process: Part I (cohort + guest speaker)
🔎 📄 How to review a research manuscript part I, overview of the structure of a research manuscript, introducing the bias self-assessment tool
Week 5: The review process: Part II (cohort)
🧐 📄 How to review a research manuscript part II, cohort discussion and reflections on review writing experience and bias self-assessment
Week 6: Peer-to-peer support coffee hour (cohort, optional)
☕ 💁🏻♀️ Mentee-led call, opportunity for skill/experience sharing between cohort members
Week 7: Peer-to-peer review sharing: Part I (cohort)
👐 📃 Mentees exchange reviews among themselves, discuss successes and challenges
Week 8: Mentor + Mentee Call (1:1)
🙋🏾♀️ 🤝 Mentees get direct, 1:1 time with their mentors who will provide feedback on their first review assignment
Week 9: Peer-to-peer review sharing: Part II (cohort)
👐 📃 Mentees exchange reviews among themselves, discuss successes and challenges and feedback from mentors
Week 10: Expert call (cohort + guest speaker)
👩🏾🏫 🧑🏻🔬 Topic to be decided
Week 11: Careers: How does one become a professional editor (cohort + guest speaker)
👩🏼💻 🖋️ Learn the career paths to become a professional editor
Week 12: Mentor + Mentee Call (1:1)
🙋🏻♂️ 🙌🏽 Mentees get direct, 1:1 time with their mentors who will provide feedback on their second review assignment
Week 13: Peer-to-peer support coffee hours (cohort, optional)
☕ 💁🏽 Mentee-led call, opportunity for skill/experience sharing between cohort members
Week 14: Wrap up call and next steps (cohort)
🔗 📬 Final reflections, how to stay connected, future programs and how to become a mentor
Program Development Timeline & Milestones 📍
As an open project, we would like to share with you our roadmap for developing our peer review mentoring program. We welcome constructive feedback to help make this program a success. Thank you!
Phase I: January-February 2020
- Define the goals of the program and the unique value proposition
- Research and consolidate information about existing peer review resources available online
- Outline the curriculum
- Structure the format for the program
- Develop and write mentor expectations
Phase II: March-May 2020
- Write an Open Roadmap and share it with the community
- Complete first/second guide and peer review rubric
- Develop bias/assumptions training resources (guide and survey)
- Design and develop a rubric for preprint reviews
Phase III: June-September 2020
- Build the program website page and add relevant content
- Develop application form
- Compile a list and contact potential mentors
- Compile a list and contact potential mentees
- Compile a list and contact potential experts to engage as guest speakers
Phase IV: October 2020-January 2021
- 🎉🎉 LAUNCH PILOT PROGRAM!!! 🎉🎉
How can you help? 👐🏾 👐🏼 👐🏿
Funding: We are currently looking for funding to support a part-time or full-time position to help our team run this program. If you are a funder or know of a funder whose mission aligns with our goals, or you are an organization interested in partnering with us to run this program or a future version of it, please let us know by emailing us at mentoring@prereview.org.
Feedback: If you have ideas 💡 or feedback 🗣 on this program and its content, we are all ears!👂 Please contact us at mentoring@prereview.org.
Participation: If you are an early-career researcher interested in being considered as a mentee for our pilot starting in October, please contact us at mentoring@prereview.org.
People
These are the people who have so far contributed to building this program:
Antoinette Foster is a Ph.D. candidate at Oregon Health & Science University, in Portland OR, where she studies the transcriptional and translational profiles of glial brain cells called oligodendrocytes. She is also the Co-Founder of a campus affinity group, Alliance for Visible Diversity in Science, which works to increase racial diversity and inclusion visible diversity within science. She joined the PREreview team in January 2020 and has developed most of the content for the program.
Samantha Hindle, Ph.D., is PREreview Co-Founder and lead of this Peer Review Mentoring Program. She has provided project direction and content development. Her day job is as Senior Content Lead at bioRxiv and medRxiv where she guides and supports the preprint screening teams, helps to implement innovative updates to the server, and engages in advocacy efforts to promote equity in access to scientific discoveries through preprints.
Daniela Saderi, Ph.D., is PREreview Co-Founder and Director. She provides suggestions and input to the program implementation and coordinates external partnerships and fundraising for the program.
Katrina Murphy joined the PREreview team as a Project Manager in April 2020. She provides support to all three pillars of PREreview including being part of the team behind this Peer Review Mentoring Program.
Acknowledgments
We thank the Mozilla Open Leaders community, in particular Abigail Cabunoc-Mayes and Chad Sansing for pioneering a completely online and interactive program that empowered people to be leaders in their communities from all over the world.
We thank Emmy Tsang, Yo Yehudi, and Julia Lawndes for advising us on how to frame and organize a spin-off of the Mozilla Open Leaders, adapting it to our needs and those of the communities we want to serve.
We thank Monica Granados, a member of the PREreview leadership team, as well as all other members of the PREreview Advisory Committee/Advisors for their advice and support in building this program.
