Skip to content

Commit c77352b

Browse files
authored
Set TOKENIZERS_PARALLELISM to false in train.py (#120)
Avoids the warning about "The current process just got forked. Disabling parallelism to avoid deadlocks"
1 parent 50d16db commit c77352b

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

train.py

+4-1
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,12 @@
1+
# flake8: noqa: E402
12
import argparse
23
import json
34
import os
45

5-
import pytorch_lightning as pl
6+
# Disable tokenizer parallelism to avoid deadlocks due to forking
7+
os.environ["TOKENIZERS_PARALLELISM"] = "false"
68

9+
import pytorch_lightning as pl
710
import src.data_loaders as module_data
811
import torch
912
from pytorch_lightning.callbacks import ModelCheckpoint

0 commit comments

Comments
 (0)