This repo contains the code used to obtain all the results in the thesis. The code folder contains detailed instructions about the structure of the code and how to perform the experiments.
This repository is a combination of the nanoGPT repository (source: https://github.com/karpathy/nanoGPT) and a pure PyTorch RetNet repository (source: https://github.com/fkodom/yet-another-retnet). This repository integrates these repos into each other and provides additonal code for using the different RetNet representations, pre-training RetNet, fine-tuning RetNet and sampling/evaluating these models.
The medNLI and MIMIC data used is not included in this repository in order to adhere to ethical standards, but can be obtained from their original sources:
- medNLI: https://physionet.org/content/mednli/1.0.0/
- MIMIC-IV note: https://physionet.org/content/mimic-iv-note/2.2/