This repository contains the source code accompanying the JAMA article: 'Affiliation Bias in Peer Review of Abstracts by a Large Language Model', published on 2023-12-27.
DOI: 10.1001/jama.2023.24641
Contributors: Dario von Wedel; Rico André Schmitt; Moritz Thiele; Raphael Leuner; Denys Shay; Simone Redaelli; Maximilian S. Schaefer
The code used for data acquisition and plot generation is in this GitHub repository. Additionally, we provide the SQLite database with the raw data and the Stata code for statistical analysis. You may find these files in the OSF project. Please note that some files may not be displayed in the OSF GUI, but all files can be downloaded for closer examination of the code. Detailed annotations can be found within the code as comments. The order of code documents follows the structure in the supplemental document. Therefore, you can go through the code chronologically to follow the data collection and data analysis process. GPT and GitHub Copilot were used during the coding process to support code creation and debugging. The authors are aware that they are responsible for the correctness of the code and take full responsibility for it.
The 30 abstracts used in this study were scraped from medRxiv using the paperscraper module. Please note that running this code will not create the same list of abstracts as used in our study, since the script (S1_Data_collection/S1_2_Collection_of_Abstracts.py) accesses the last 30 abstracts of the medRxiv dump. Depending on the date your dump is created, this will result in a different list of papers. The list of abstracts we used in our study can be found here: S1_Data_collection/S1_1_Collection_of_Abstracts_results.csv. Note that this list contains only metadata but no abstracts.