RQM is a research methods course that focuses on modernizing the post-data collection portion of the scientific workflow. The course takes an approach that produces both conventional research products and trains students to make their work more efficient and reproducible. This handbook provides a framework for professors who would like to teach a 14-week class on reproducible quantitative methods, presuming an understanding of open workflows for publication, some intermediate R or (other command-line based data analysis software) skills, and basic GitHub operations and use.

Background There is wide agreement in the biomedical research community that research data sharing is a primary ingredient for ensuring that science is more transparent and reproducible. Publishers could play an important role in facilitating and enforcing data sharing; however, many journals have not yet implemented data sharing policies and the requirements vary widely across journals. This study set out to analyze the pervasiveness and quality of data sharing policies in the biomedical literature. Methods The online author’s instructions and editorial policies for 318 biomedical journals were manually reviewed to analyze the journal’s data sharing requirements and characteristics. The data sharing policies were ranked using a rubric to determine if data sharing was required, recommended, required only for omics data, or not addressed at all. The data sharing method and licensing recommendations were examined, as well any mention of reproducibility or similar concepts. The data was analyzed for patterns relating to publishing volume, Journal Impact Factor, and the publishing model (open access or subscription) of each journal. Results A total of 11.9% of journals analyzed explicitly stated that data sharing was required as a condition of publication. A total of 9.1% of journals required data sharing, but did not state that it would affect publication decisions. 23.3% of journals had a statement encouraging authors to share their data but did not require it. A total of 9.1% of journals mentioned data sharing indirectly, and only 14.8% addressed protein, proteomic, and/or genomic data sharing. There was no mention of data sharing in 31.8% of journals. Impact factors were significantly higher for journals with the strongest data sharing policies compared to all other data sharing criteria. Open access journals were not more likely to require data sharing than subscription journals. Discussion Our study confirmed earlier investigations which observed that only a minority of biomedical journals require data sharing, and a significant association between higher Impact Factors and journals with a data sharing requirement. Moreover, while 65.7% of the journals in our study that required data sharing addressed the concept of reproducibility, as with earlier investigations, we found that most data sharing policies did not provide specific guidance on the practices that ensure data is maximally available and reusable.

The objective of this study was to evaluate the nature and extent of reproducible and transparent research practices in neurology publications. Methods The NLM catalog was used to identify MEDLINE-indexed neurology journals. A PubMed search of these journals was conducted to retrieve publications over a 5-year period from 2014 to 2018. A random sample of publications was extracted. Two authors conducted data extraction in a blinded, duplicate fashion using a pilot-tested Google form. This form prompted data extractors to determine whether publications provided access to items such as study materials, raw data, analysis scripts, and protocols. In addition, we determined if the publication was included in a replication study or systematic review, was preregistered, had a conflict of interest declaration, specified funding sources, and was open access. Results Our search identified 223,932 publications meeting the inclusion criteria, from which 400 were randomly sampled. Only 389 articles were accessible, yielding 271 publications with empirical data for analysis. Our results indicate that 9.4% provided access to materials, 9.2% provided access to raw data, 0.7% provided access to the analysis scripts, 0.7% linked the protocol, and 3.7% were preregistered. A third of sampled publications lacked funding or conflict of interest statements. No publications from our sample were included in replication studies, but a fifth were cited in a systematic review or meta-analysis. Conclusions Currently, published neurology research does not consistently provide information needed for reproducibility. The implications of poor research reporting can both affect patient care and increase research waste. Collaborative intervention by authors, peer reviewers, journals, and funding sources is needed to mitigate this problem.

Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000–2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks. We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]). Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]). Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed. Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.

The Research Data Curation Bibliography includes over 750 selected English-language articles, books, and technical reports that are useful in understanding the curation of digital research data in academic and other research institutions.

MANTRA is a free, online non-assessed course with guidelines to help you understand and reflect on how to manage the digital data you collect throughout your research. It has been crafted for the use of post-graduate students, early career researchers, and also information professionals. It is freely available on the web for anyone to explore on their own.

Through a series of interactive online units you will learn about terminology, key concepts, and best practice in research data management.

There are eight online units in this course and one set of offline (downloadable) data handling tutorials that will help you:

1. Understand the nature of research data in a variety of disciplinary settings
2. Create a data management plan and apply it from the start to the finish of your research project
3. Name, organise, and version your data files effectively
4. Gain familiarity with different kinds of data formats and know how and when to transform your data
5. Document your data well for yourself and others, learn about metadata standards and cite data properly
6. Know how to store and transport your data safely and securely (backup and encryption)
7. Understand legal and ethical requirements for managing data about human subjects; manage intellectual property rights
8. Understand the benefits of sharing, preserving and licensing data for re-use
9. Improve your data handling skills in one of four software environments: R, SPSS, NVivo, or ArcGIS

Each unit takes up to one hour, plus time for further reading and carrying out the data handling exercises. In the units you will find explanations, descriptions, examples, exercises, and video clips in which academics, PhD students and others talk about the challenges of managing research data. The data handling tutorials assume some experience with each software environment and provide exercises in PDF along with open datasets to download and work through using your own installed software.

MANTRA modules and data handling exercises are available for download via Zenodo: https://doi.org/10.5281/zenodo.1035218

The Research Data Management Academy (RDMLA) is a global, free online professional development program for librarians, information professionals, or other professionals who work in a research-intensive environment. The curriculum focuses on the knowledge and skills needed to collaborate with researchers and other stakeholders on data management. RDMLA features a unique partnership between a library and information science academic program, academic health sciences and research libraries, and industry publisher. All of the content is hosted on Canvas Network, freely available, and open for reuse under a CC-BY-NC-SA license.

This webliography is intended for librarians seeking to enhance their own knowledge and assist peers in improving their data management awareness. The webliography is organized by content type, first with more foundational materials such as established data management curricula and then with current awareness and community materials such as social media.

This version of this teaching module was published in Teaching Issues and Experiments in Ecology:

Jim McNeil and Megan A. Jones. April 2018, posting date. Data Management using National Ecological Observatory Network’s (NEON) Small Mammal Data with Accompanying Lesson on Mark Recapture Analysis. Teaching Issues and Experiments in Ecology, Vol. 13: Practice #9 [online]. http://tiee.esa.org/vol/v13/issues/data_sets/mcneil/abstract.html

*** *** ***

Undergraduate STEM students are graduating into professions that require them to manage and work with data at many points of a data management life cycle. Within ecology, students are presented not only with many opportunities to collect data themselves, but increasingly to access and use public data collected by others. This activity introduces the basic concept of data management from the field through to data analysis. The accompanying presentation materials mention the importance of considering long-term data storage and data analysis using public data.

This data set is a subset of small mammal trapping data from the National Ecological Observatory Network (NEON). The accompanying lesson introduces students to proper data management practices including how data moves from collection to analysis. Students perform basic spreadsheet tasks to complete a Lincoln-Peterson mark-recapture calculation to estimate population size for a species of small mammal. Pairs of students will work on different sections of the datasets allowing for comparison between seasons or, if instructors download additional data, between sites and years. Data from six months at NEON’s Smithsonian Conservation Biology Institute (SCBI) field site are included in the materials download. Data from other years or locations can be downloaded directly from the NEON data portal to tailor the activity to a specific location or ecological topic.

In this activity, students will:

- discuss data management practices with the faculty. Presentation slides are provided to guide this discussion.
- view field collection data sheets to understand how organized data sheets can be constructed.
- design a spreadsheet data table for transcription of field collected data using good data management practices.
- view NEON small mammal trapping data to a) see a standardized spreadsheet data table and b) see what data are collected during NEON small mammal trapping.
- use Microsoft Excel or Google Sheets to conduct a simple Lincoln-Peterson Mark-Recapture analysis to estimate plot level species population abundance.

Please note that this lesson was developed while the NEON project was still in construction. There may be future changes to the format of collected and downloaded data. If using data directly from the NEON Data Portal instead of using the data sets accompanying this lesson, we recommend testing out the data each year prior to implementing this lesson in the classroom.

This module was originally taught starting with a field component where students accompanied NEON technicians during the small mammal trapping. As this is not a possibility for most courses, the initial part of the lesson has been modified to include optional videos that instructors can use to show how small mammal trapping is conducted. Instructors are also encouraged to bring small mammal traps and small mammal specimens into the classroom where available.

The Data Sets

The National Ecological Observatory Network is a program sponsored by the National Science Foundation and operated under cooperative agreement by Battelle Memorial Institute. This material is based in part upon work supported by the National Science Foundation through the NEON Program.

The following datasets are posted for educational purposes only. Data for research purposes should be obtained directly from the National Ecological Observatory Network (www.neonscience.org).

Data Citation: National Ecological Observatory Network. 2017. Data Product: NEON.DP1.10072.001. Provisional data downloaded from http://data.neonscience.org. Battelle, Boulder, CO, USA

Notes
Version 2.1: Includes correct Lincoln-Peterson Index formula in PPT, faculty, and student notes.

Version 2.0: This version of the teaching module was published in Teaching Issues and Experiments in Ecology. McNeil and Jones 2018. This version reflects updates based on comments from reviewers.

Version 1.0: This version of the teaching module was prepared as part of the 2017 DIG FMN. It was submitted for publication as part of the DIG Special Issue of TIEE.

Cite this work
Researchers should cite this work as follows:

Jim McNeil, Megan A. Jones (2018). Data Management using National Ecological Observatory Network's (NEON) Small Mammal Data with Accompanying Lesson on Mark Recapture Analysis. NEON - National Ecological Observatory Network, (Version 2.1). QUBES Educational Resources. doi:10.25334/Q4M121

The laboratory notebook is the cornerstone of any laboratory course. Students develop critical thinking, documentation, and communication skills while mastering scientific concepts. Furthermore, the use of electronic lab notebooks (ELNs) for documentation has become the standard for data management in industry and academic labs.

Replacing paper with an interactive research notebook provides students with authentic data management skills. It also offers a medium for easily incorporating quantitative reasoning into the curriculum to address real. Instructors using digital notebooks in their courses reported a significant increase in student engagement and assessment scores.

In this workshop, we will explore the current ELNs landscape and best practices for moving from paper to digital.

Cite this work
Researchers should cite this work as follows:

Stringer, N. (2019). Electronic Lab Notebooks: Options for Building Data Management and Quantitative Reasoning Skills. Evolution of Data in the Classroom: From Data to Data Science (SW 2019), QUBES Educational Resources. doi:10.25334/B1NP-B249

Description
This lesson introduces students to working with metadata, which can be broadly thought of as the data ABOUT existing data. Data isn’t complete without metadata, and this lesson will help students understand both how to work with metadata and how to create their own.

Data used: NEON aquatic macroinvertebrate datasets from multiple stations. It could be adapted to use any data sets or taxonomic groups though.

Activities: The lesson involves three major activities. 1) Querying and downloading datasets and corresponding products from NEON. 2) Reading and answering comprehension questions about metadata files that correspond with data files 3) Combining two datasets based off understanding the metadata in exercise 2 (e.g. understanding which columns indicate sampling dates and in which formats will allow them to appropriately combine multiple data sets).

Programs: No specific programming skills or language is required for this lesson. This lesson is designed to be done entirely in common office/student software programs (e.g. Microsoft Word and Microsoft Office) and could be done using online programs (e.g. my university has student licenses for Google Spreadsheets and Google Docs).

Learning objectives:

1 – Students will be able to define ‘metadata’ and understand how metadata is critical for reproducible research.

2 – Students will be able to correctly answer comprehension questions about a metadata file.

3 – Students will be able to apply their understanding of the metadata file to create a new data file from two data sets.

4 – Students will understand the importance of creating and understanding metadata to go along with datasets.

Timing: This lesson was designed to take place in two – 75 minute class periods that are in a workshop format. This lesson could easily be part of a longer lab, homework, or a remote / online / asynchronous assignment.

Notes
This version is current as of Spring 2019 and was classroom taught. I encourage folks to adapt, modify, and make new versions.

Cite this work
Researchers should cite this work as follows:

Whitney, K. S. (2019). Introduction to Data Management and Metadata using NEON aquatic macroinvertebrate data. NEON Faculty Mentoring Network, QUBES Educational Resources. doi:10.25334/SJX1-F373

Many of today's popular sports are based around the use of balls, yet none of the balls are completely alike. In fact, they are all designed with specific characteristics in mind and are quite varied. Students investigate different balls' abilities to bounce and represent the data they collect graphically.

Review of Answering Questions with Data
https://drive.google.com/open?id=15lxoO1g39m6JICO7eKhiLR_pQqB4iOoJJtcylZJgp2E

This lesson in part of Software Carpentry workshop and teach novice programmers to write modular code and best practices for using R for data analysis. an introduction to R for non-programmers using gapminder data The goal of this lesson is to teach novice programmers to write modular code and best practices for using R for data analysis. R is commonly used in many scientific disciplines for statistical analysis and its array of third-party packages. We find that many scientists who come to Software Carpentry workshops use R and want to learn more. The emphasis of these materials is to give attendees a strong foundation in the fundamentals of R, and to teach best practices for scientific computing: breaking down analyses into modular units, task automation, and encapsulation. Note that this workshop will focus on teaching the fundamentals of the programming language R, and will not teach statistical analysis. The lesson contains more material than can be taught in a day. The instructor notes page has some suggested lesson plans suitable for a one or half day workshop. A variety of third party packages are used throughout this workshop. These are not necessarily the best, nor are they comprehensive, but they are packages we find useful, and have been chosen primarily for their usability.

Data Carpentry lesson part of the Social Sciences curriculum. This lesson teaches how to analyse and visualise data used by social scientists. Data Carpentry’s aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain. The lessons below were designed for those interested in working with social sciences data in R. This is an introduction to R designed for participants with no programming experience. These lessons can be taught in a day (~ 6 hours). They start with some basic information about R syntax, the RStudio interface, and move through how to import CSV files, the structure of data frames, how to deal with factors, how to add/remove rows and columns, how to calculate summary statistics from a data frame, and a brief introduction to plotting.

Una introducción a R utilizando los datos de Gapminder. El objetivo de esta lección es enseñar a las programadoras principiantes a escribir códigos modulares y adoptar buenas prácticas en el uso de R para el análisis de datos. R nos provee un conjunto de paquetes desarrollados por terceros que se usan comúnmente en diversas disciplinas científicas para el análisis estadístico. Encontramos que muchos científicos que asisten a los talleres de Software Carpentry utilizan R y quieren aprender más. Nuestros materiales son relevantes ya que proporcionan a los asistentes una base sólida en los fundamentos de R y enseñan las mejores prácticas del cómputo científico: desglose del análisis en módulos, automatización tareas y encapsulamiento. Ten en cuenta que este taller se enfoca en los fundamentos del lenguaje de programación R y no en el análisis estadístico. A lo largo de este taller se utilizan una variedad de paquetes desarrolados por terceros, los cuales no son necesariamente los mejores ni se encuentran explicadas todas sus funcionalidades, pero son paquetes que consideramos útiles y han sido elegidos principalmente por su facilidad de uso.

SETDA, CCSSO and Project Unicorn collaborate to improve data interoperability in schools. This second brief from their work together outlines considerations for working with vendors, identifies data standards, and shares promising practices.

Developed through SETDA's Data Modernization Working Group, this brief provides the key points for interoperability and data modernization. Data modernization allows state education agencies to leverage data as a strategic asset and drive insights, decision-making, and innovation.

This module is part of the Foundations of School Librarianship on using web resources to enhance collaboration between STEM classroom teachers and school librarians, with special emphasis on STEM subjects. The module is built around the understanding and use of data to support classroom projects. The module describes a process by which the school librarian and teacher will collaborate on a high school-level project to explore how to find, evaluate, and use data to produce an infographic. Infographics are increasingly important as a vehicle for explaining complex subjects. They are a wonderful blend of data and information to create meaning and new knowledge. This module is intended as a 'stretching' exercise for school librarians who often have scant background in STEM. The skills learned by school librarian students revolve around identifying data sources, developing evaluative skills, translating data into an infographic, and working with classroom teachers in STEM subject to match resources with teacher identified learning goals. 

The School of Data aims to make your learning experience as tailored as possible through independent learning modules. Learning modules are all stand-alone and can be taken in any order. To make your learning experience easier, we curated modules into a series of courses - with a focus on data basics as well as specific skills. When you identified the course you're interested in click on "Show Modules" to see all modules you might want to take.