DataSeer fills the urgent need for low-cost, scalable solutions to measure open science, show researchers how to comply with policy, and deploy just-in-time interventions.

Antibody transparency report from the Only Good Antibodies community: This service evaluates the quality of information about the choice and use of antibodies within a research protocol or manuscript. It provides a table that shows which experiments relied (or will rely) on antibodies, and information about the suitability of the antibodies for these specific experiments – both within the submission/experimental plan, and from other sources where available. The report suggests where further experimental work to confirm antibody specificity is needed, where it is desirable, or where the use of an alternative antibody is suggested because of poor suitability of the antibody for the purpose used. This service is provided by experts within the OGA community, a non-profit organisation with the mission to enhance the quality of research that relies on antibodies. Validation: This is an experimental platform that relies on a combination of human checking assisted by AI to enhance speed and accuracy. Evaluations provided for Lifecycle Journal submissions are part of our validation study.

Paper-Wizard is a cutting-edge AI-powered evaluation service that delivers comprehensive, actionable feedback on academic papers in minutes, not months. The system harnesses advanced processing to provide detailed insights on theoretical soundness, methodological rigor, statistical appropriateness, and writing quality, helping authors critical evaluate and enhance their manuscripts with unprecedented speed and precision. Operating as a fully autonomous digital peer reviewer, Paper-Wizard helps researchers polish their work to meet the highest academic standards. Validation: Paper-Wizard leverages developments in generative AI to deliver manuscript evaluation capabilities. The system is in active development, with an established user base of over 1,500 researchers. While formal studies are upcoming, participation in Lifecycle Journal serves as part of our systematic validation process.

“Peer Community in” (PCI) is a non-profit and non-commercial scientific organisation gathering specific communities of researchers reviewing and recommending, for free, preprints in their field, e.g. PCI Ecology or PCI Archaeology. PCI is thus a peer-review and endorsement service that publicly evaluates – through peer review – and recommends/highlights high-quality preprints deposited in open archives. In the case of positive evaluation, PCI publishes a recommendation text and the whole evaluation process. Validation: So far, we have published over 4500 public peer reviews and over 800 public recommendations.

Participating PCI Communities:

• PCI Animal Science
• PCI Archaeology
• PCI Ecology
• PCI Ecotoxicology and Environmental Chemistry
• PCI Evolutionary Biology
• PCI Genomics
• PCI Health and Movement Science
• PCI Infection
• PCI Microbiology
• PCI Neuroscience
• PCI Organisation Studies
• PCI Palaeontology

Peer Community in Registered Reports (PCI RR) is a free, non-profit, non-commercial arm of the broader Peer Community In platform, dedicated to reviewing and recommending Registered Reports preprints across the full spectrum of STEM, medicine, the social sciences and humanities. Established in 2021 by the founders of the Registered Reports format, PCI RR has publicly recommended hundreds of Stage 1 and Stage 2 preprints.
More info

PREreview supports and empowers diverse and historically excluded communities of researchers (particularly those at the early stages of their careers) and other expert reviewers to engage in open preprint peer review. In addition to training and collaborative review facilitation services, we offer a free-to-use online review publication platform with a request-a-review feature that helps users find preprints to review. Reviewers can publish their preprint reviews with us for free by registering with their ORCID iD and then receive DOIs for published reviews and credit for their public review activity in the peer review sections of their ORCID profiles. We support dozens of preprint servers and offer COAR Notify Protocol integrations to partnering servers, like bioRxiv and SciELO Preprints, that want to give their preprint authors the ability to request reviews from PREreview’s community with the click of a button.

RegCheck leverages large-language models and AI to automatically compare preregistration plans with scientific publications. This lets researchers effortlessly identify if and how executed studies deviated from the initial plan.

ResearchHub provides an innovative open peer review marketplace for authors and journals, dramatically reducing review timelines from months to an average of 10 days. We financially incentivize reviewers with ResearchCoin (RSC), promoting rapid, high-quality feedback while promoting open science. With a transparent and efficient process, ResearchHub accelerates knowledge dissemination, empowers researchers with fair compensation, and ultimately advances the pace of scientific progress. Validation: 1) Our editorial team evaluates each peer review against a rigorous internal rubric as part of the validation process. And only high quality and scientifically rigorous reviews are awarded. 2) To date, we’ve completed over 4,000 peer reviews, with a current weekly output exceeding 150 reviews. 3) Majority of authors found the reviews to be of high quality (4/5 and 5/5 on 1-5 rating).

SciScore is an scientific methods integrity checking tool covering several checklists, including MDAR (Material Design Analysis Reporting) and STAR Methods. We provide authors a report that points out which of the facets of these guidelines were found and which were not found in the submitted text, while attempting to assess whether the criterion is applicable to the specific manuscript (see also Menke et al 2022). Validation: Babic, et al., 2019; Menke et al., 2020; Menke et al., 2002).

Statcheck can be considered a “spellchecker for statistics”: it searches the text for results of null hypothesis significance tests and checks whether the reported p-value matches its accompanying test statistic and degrees of freedom. If the p-value does not match, statcheck flags it as an inconsistency. If the reported p-value is < .05, but the recalculated p-value is > .05, or vice versa, it is flagged as a decision inconsistency. Statcheck is currently designed to find statistics in APA reporting style. It finds about 60% of reported statistics and reaches an accuracy of 96.2% to 99.9% in classifying detected results (Nuijten et al., 2017; doi.org/10.31234/osf.io/tcxaj).

The Unjournal (est. 2022) is a grant-funded nonprofit organization aiming to make research more impactful and rigorous. Our team prioritizes research in economics and quantitative social science based on its potential for global impact. Managers recruit research experts as evaluators and we pay them to write a report providing high-quality feedback and discussion. We elicit percentile ratings across several criteria, emphasizing credibility, transparency, and methodological rigor, as well as an overall assessment benchmarked to standard journal-tier metrics. Authors can respond (and evaluators can adjust in turn) before we publish the evaluation package along with a manager’s synthesis. See our guidelines for evaluators and our public evaluation output (Validation: ~35 evaluation packages).

Do you run an evaluation service for scholarly research, or do you have an innovative idea for how planned or completed research could be evaluated? Contact us to explore partnering in the marketplace of evaluation services.