Why should you care about making your research reproducible?
As a researcher, you want to be able to show evidence of the correctness of your results. Documenting your methods, your code, and your analyses will allow you to verify your own findings if you need to revisit a past project. Sharing your data, methods, and analyses enables others to confirm your findings and extend your methods and results into other research.
Some benefits of reproducible research:
Increase the verifiability of your methods and results
Reduce possible data loss and save time for future projects
Enable others researchers to build off of existing research
Reproducibility is defined as “obtaining consistent results using the same input data, computational steps, methods, and code, and conditions of analysis” (National Academies of Sciences, Engineering, and Medicine, 2019. Reproducibility and Replicability in Science. Washington, D.C.: The National Academies Press. https://doi.org).
Reproducibility is the ability to take the code and data from a previous publication, rerun the code and get the same results.
Replicability is the ability to rerun an experiment and get “consistent” results with the original study using new data. Results that are not reproducible are hard to verify and results that do not replicate in new studies are harder to trust.” credit: Jeff Leek
Registration or pre-registration: Pre-registration of research simply refers to a detailed plan about how researchers will conduct a study and analyze its results. Pre-registering your studies and analysis plans ensures that your study can be found and readers can differentiate exploratory from confirmatory research. Pre-registration is a step towards transparency and supports reproducible research practice.
The journal Developmental Science is an example of a journal that is using OSF’s Registries platform for registering research prior to conducting a study. Registered studies go through a two-stage peer review process: (1) peer review of the proposed methods/protocols with “in-principal” acceptance and (2) peer review of final results in the manuscript with publication following as long as authors adhere to the registered protocols.
Version Control & File Management - Using version control and file management will provide a framework for you and your collaborators to differentiate between files used, to have reference points at the different stages of your work, and to provide internal guidance for instances and opportunities to share work.
Organizing your lab - by using things like electronic lab notebooks this can provide a digital back up of materials. This is beneficial in instances of natural disasters or life events, to have an additional location to back up and save active lab work, results, and data.
Coding - using coding languages / platforms like R and Python can make reproduction of code easier due to the open license. This allows access to the code files without using a proprietary software or coding language.
GNU Make - controls the generation of executables and other non-source files of a program from the program’s source files.
Pandoc - helps convert files from one markup format to another.
ReproZip - allows you to pack your research along with necessary data files, libraries, environment variables and options into a self-contained bundle. Then ReproZip can use that bundle to automatically set up the same original environment so anybody can reproduce the research on a different machine, without tracking down and installing the dependencies, or even having to run the same operating system.
Writing - Using editors like those listed below can provide reproducible document creation. This can allow for the integration of code sections, scientific notations, and environments. Using this can also provide literate computing, authoring and publishing so the writing and publishing are self-contained documents that include the narrative and code used.