Moshontz, Hannah, Lorne Campbell, Charles R. Ebersole, Hans IJzerman, Heather L. Urry, Patrick S. Forscher, Jon E. Grahe, et al. 2018. ‘The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network’. Advances in Methods and Practices in Psychological Science 1 (4): 501–15. https://doi.org/10.1177/2515245918797607.
« The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. » (Moshontz et al., 2018, p. 2)
« The initiative quickly grew into a network with over 300 data collection labs, an organized governance structure, and a set of policies for evaluating, preparing, conducting, and disseminating studies. » (Moshontz et al., 2018, p. 3)
« […] the PSA provides an ideal context in which to train early-career psychological scientists, and in which psychological scientists of all career stages can learn about new methodological practices and paradigms. » (Moshontz et al., 2018, p. 18)
Phase 1: Submission & Evaluation
« Proposing authors submit a description of the proposed study background, desired participant characteristics, materials, procedures, hypotheses, effect-size estimates, and data-analysis plan, including an analysis script and simulated data when possible, much like a Stage 1 manuscript submitted under a Registered Reports model. These submissions are then masked and evaluated according to a process overseen by the Study Selection Committee . » (Moshontz et al., 2018, p. 11)
Phase 2: Preparation
« Next, the Methodology and Data Analysis Committee , whose members are selected on the basis of methodological and statistical expertise, evaluates and suggests revisions of the selected studies to help prepare the protocols for implementation. » (Moshontz et al., 2018, p. 12)
« Next, the Logistics Committee identifies specific labs willing and able to run the specific protocols, bundling multiple studies into single laboratory sessions to maximize data collection efficiency when possible. The Logistics Committee then matches data collection labs to projects. Not every network lab participates in every study. » (Moshontz et al., 2018, p. 12‑13)
Phase 3: Implementation
« We expect implementation to be the most time-intensive and variable phase. This process begins with pre-registering the hypotheses and confirmatory or exploratory research questions , the data-collection protocol, and the analysis plan developed in Phase 2, with instructional resources and support provided to the proposing authors as needed by the Project Management Committee. Pre-registration of confirmatory analysis plans, methods, and hypotheses is a minimum requirement of the PSA. The PSA encourages exploratory research and exploratory analyses, as long as these are transparently reported as such.» (Moshontz et al., 2018, p. 13)
Phase 4: Analysis and Dissemination
« The proposing authors will complete confirmatory data analyses, as described in their pre-registration. Once the confirmatory analyses have been conducted, the proposing authors will draft the empirical report . Drafting authors will be encouraged to write the manuscript as a dynamic document , for example using R Markdown. All contributing labs and other authors (e.g., those involved in designing and implementing the project) will be given the opportunity to provide feedback and approve the manuscript with reasonable lead time prior to submission. Following the principle of transparency, the PSA prefers publishing in open-access outlets or as open-access articles.» (Moshontz et al., 2018, p. 14)
« When the project is concluded, all data, analytic code and meta-data will be posted in full and made public on the OSF by default or on another public and stable repository on a case-by-case basis. These data will be available for other researchers to conduct exploratory and planned secondary analyses. Data release will be staged such that a “train” dataset will be publicly released quickly after data collection and preparation, and the remaining “test” dataset will be released later, following a wide and early (e.g., one year out) public announcement (e.g., as in Klein et al., 2018). » (Moshontz et al., 2018, p. 15)