In this video, I'll discuss the relevance of research documentation. Good documentation is critical for two things, objective replication and checking and verification of results and conclusions. In other words, documentation is very important to research integrity. In order to replicate a study, the hypothesis, research design and predictions need to be clearly stated. Replication also requires a clear description of the procedures and instruments, and the protocol for contact and interacting with participants. Finally, the data, data manipulation, and statistics that were performed, and the researcher's interpretation of these results into a final conclusion also need to be documented. If any of these steps are unclear or cannot be verified after the study is performed, then this can lead to confusion when contradictory results are found in replication studies. The hypothesis, research design, specific predictions, and interpretation of results are generally explicitly stated in a research publication or journal article. Such a document also contains information on the research procedure, instruments, data, data manipulations and statistics. But this information is always summarized. Giving all the details simply takes up too much space. Most researchers just want to know a study's goal, setup and outcome. However, the detailed information should be documented somewhere so it can be made available to those researchers who are interested in performing an exact replication. So, what details should be documented exactly? Well, to start, all information on the instruments and materials needs to be available. With instrument information, I mean the instruments used to measure or manipulate the variables of interest, and the procedures or instruments used to control or measure extraneous variables. The materials include written or verbal instructions, text used in email communications, consent forms, documents containing debriefing information, et cetera, et cetera. Basically, all materials, information and documents that were used by the experimenter or a participant need to be retrievable. Let's focus on the instruments for a moment. Unlike tape measures and weight scales, social and psychological measures are usually not self-evident. In research articles, measurement information is summarized by stating of how many questions a scale in a questionnaire consists, what the response options are, what the range of possible scale scores is, an example item is often provided. This is the minimum required information to interpret the results, but a full version of the instrument needs to be available for checking and replication. The instrument itself, meaning its observation coding scheme or questionnaire items and response items, should be available to others. This requires, for example, recording what version of a standardized test was used, or documenting and providing the instrument itself if it's not publicly or commercially available. Besides information about instruments and materials, the research protocol needs to be documented. The research protocol refers to the order and manner in which materials and information were presented to the participants and the way procedures were implemented. A research protocol is a clear description of what happens to the participant from start to finish. This includes the moment of first contact, how a participant is recruited, with what information and by whom. The protocol also describes how consent is obtained, what instruction or information is given, in what order, what help is provided if participants don't understand instructions or they behave in an unexpected manner. The research protocol also describes how participants are debriefed about the research after their participation. Documenting materials is relatively easy. It simply means saving emails and documents. Writing a research protocol is more tedious work and is not always performed diligently. Of course, this can become a big problem when other researchers want to replicate a study and can only use summarized information or have to rely on the original researcher's memory. Another thing that's often badly documented or not even explicitly formulated at all is the statistical plan. Ideally, a researcher should determine what statistical analysis will be performed before looking at the data. This is because the choice of statistics can sometimes determine whether the predictions are confirmed or disconfirmed. Ideally, the hypothesis and intended statistical analyses are preregistered so that a researcher is not tempted to change either after seeing the results and realizing that a different approach will result in more favorable outcomes. Preregistration is customary in medical sciences and is gaining popularity in neuroscience, clinical psychology and other fields. Finally, information about pilot studies often fails to be documented. A pilot study is a small preliminary study where a newly developed instrument or manipulation procedure is tested. Such studies can contain useful information for other researchers developing or adapting similar instruments. A pilot study should therefore at least be mentioned in the published article so that others are aware that the results are available.
