Making Your Research Reproducible

Why should you care about making your research reproducible?

As a researcher, you want to be able to show evidence of the correctness of your results. Documenting your methods, your code, and your analyses will allow you to verify your own findings if you need to revisit a past project. Sharing your data, methods, and analyses enables others to confirm your findings and extend your methods and results into other research. 

Some benefits of reproducible research:

  • Increase the verifiability of your methods and results 
  • Reduce possible data loss and save time for future projects
  • Enable others researchers to build off of existing research
  • Get credit for sharing and documenting your data
  • Increase the visibility of your research

 

  • What is Reproducibility?
    Reproducibility is defined as “obtaining consistent results using the same input data, computational steps, methods, and code, and conditions of analysis” (National Academies of Sciences, Engineering, and Medicine, 2019. Reproducibility and Replicability in Science. Washington, D.C.: The National Academies Press. https://doi.org).
  • Are there best practices or guidelines for reproducibility?

    FAIR data principles are a great place to start: https://www.go-fair.org/fair-principles/ and cover these main themes:

    • Data are Findable - create and use metadata to support discovery
    • Data are Accessible - make sure that mechanisms to access data are clear
    • Data are Interoperable - structure data so that data can be integrated with other data, applications and workflows
    • Data are Reusable - apply appropriate licenses that enable reuse

    FAIR data cartoon

    Image credit: https://book.fosteropenscience.eu/

  • What does reproducibility look like in practice?
  • What is the difference between Reproducibility and Replicability?
    • Reproducibility is the ability to take the code and data from a previous publication, rerun the code and get the same results. 
    • Replicability is the ability to rerun an experiment and get “consistent” results with the original study using new data. Results that are not reproducible are hard to verify and results that do not replicate in new studies are harder to trust.” credit: Jeff Leek 
  • What is “pre-registration” all about?

    Registration or pre-registration: Pre-registration of research simply refers to a detailed plan about how researchers will conduct a study and analyze its results. Pre-registering your studies and analysis plans ensures that your study can be found and readers can differentiate exploratory from confirmatory research.  Pre-registration is a step towards transparency and supports reproducible research practice.

  • Tools for reproducibility
    • Version Control & File Management - Using version control and file management will provide a framework for you and your collaborators to differentiate between files used, to have reference points at the different stages of your work, and to provide internal guidance for instances and opportunities to share work.
    • Organizing your lab - by using things like electronic lab notebooks this can provide a digital back up of materials. This is beneficial in instances of natural disasters or life events, to have an additional location to back up and save active lab work, results, and data.
    • Coding - using coding languages / platforms like R and Python can make reproduction of code easier due to the open license. This allows access to the code files without using a proprietary software or coding language.
    • Compiling / Packaging
      • GNU Make - controls the generation of executables and other non-source files of a program from the program’s source files. 
      • Pandoc - helps convert files from one markup format to another.
      • ReproZip - allows you to pack your research along with necessary data files, libraries, environment variables and options into a self-contained bundle. Then ReproZip can use that bundle to automatically set up the same original environment so anybody can reproduce the research on a different machine, without tracking down and installing the dependencies, or even having to run the same operating system.
    • Writing - Using editors like those listed below can provide reproducible document creation. This can allow for the integration of code sections, scientific notations, and environments. Using this can also provide literate computing, authoring and publishing so the writing and publishing are self-contained documents that include the narrative and code used. 
  • Where can I learn more?

I have a question - who can I reach out to?

Contact us with the Research Facilitation Services (RFS) intake form.

Credit for much of the content on this page goes to https://www.openaire.eu/how-to-make-your-data-fair