Skip to content

Commit e13c1f3

Browse files
Fixing typo in function name and updating README
1 parent 2f82f21 commit e13c1f3

File tree

6 files changed

+27
-16
lines changed

6 files changed

+27
-16
lines changed

README.md

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
# BERT
22

3+
**\*\*\*\*\* New November 5th, 2018: Third-party PyTorch version of BERT
4+
available \*\*\*\*\***
5+
6+
NLP researchers from HuggingFace made a
7+
[PyTorch version of BERT available](https://github.com/huggingface/pytorch-pretrained-BERT)
8+
which is compatible with our pre-trained checkpoints and is able to reproduce
9+
our results. (Thanks!) We were not involved in the creation or maintenance of
10+
the PyTorch implementation so please direct any questions towards the authors of
11+
that repository.
12+
313
**\*\*\*\*\* New November 3rd, 2018: Multilingual and Chinese models available
414
\*\*\*\*\***
515

@@ -63,8 +73,8 @@ minutes.
6373

6474
## What is BERT?
6575

66-
BERT is method of pre-training language representations, meaning that we train a
67-
general-purpose "language understanding" model on a large text corpus (like
76+
BERT is a method of pre-training language representations, meaning that we train
77+
a general-purpose "language understanding" model on a large text corpus (like
6878
Wikipedia), and then use that model for downstream NLP tasks that we care about
6979
(like question answering). BERT outperforms previous methods because it is the
7080
first *unsupervised*, *deeply bidirectional* system for pre-training NLP.
@@ -778,9 +788,13 @@ information.
778788

779789
#### Is there a PyTorch version available?
780790

781-
There is no official PyTorch implementation. If someone creates a line-for-line
782-
PyTorch reimplementation so that our pre-trained checkpoints can be directly
783-
converted, we would be happy to link to that PyTorch version here.
791+
There is no official PyTorch implementation. However, NLP researchers from
792+
HuggingFace made a
793+
[PyTorch version of BERT available](https://github.com/huggingface/pytorch-pretrained-BERT)
794+
which is compatible with our pre-trained checkpoints and is able to reproduce
795+
our results. We were not involved in the creation or maintenance of the PyTorch
796+
implementation so please direct any questions towards the authors of that
797+
repository.
784798

785799
#### Will models in other languages be released?
786800

extract_features.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ def model_fn(features, labels, mode, params): # pylint: disable=unused-argument
170170

171171
tvars = tf.trainable_variables()
172172
scaffold_fn = None
173-
(assignment_map, _) = modeling.get_assigment_map_from_checkpoint(
173+
(assignment_map, _) = modeling.get_assignment_map_from_checkpoint(
174174
tvars, init_checkpoint)
175175
if use_tpu:
176176

modeling.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@ def get_activation(activation_string):
315315
raise ValueError("Unsupported activation: %s" % act)
316316

317317

318-
def get_assigment_map_from_checkpoint(tvars, init_checkpoint):
318+
def get_assignment_map_from_checkpoint(tvars, init_checkpoint):
319319
"""Compute the union of the current variables and checkpoint variables."""
320320
assignment_map = {}
321321
initialized_variable_names = {}

run_classifier.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -571,9 +571,8 @@ def model_fn(features, labels, mode, params): # pylint: disable=unused-argument
571571

572572
scaffold_fn = None
573573
if init_checkpoint:
574-
(assignment_map,
575-
initialized_variable_names) = modeling.get_assigment_map_from_checkpoint(
576-
tvars, init_checkpoint)
574+
(assignment_map, initialized_variable_names
575+
) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
577576
if use_tpu:
578577

579578
def tpu_scaffold():

run_pretraining.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -152,9 +152,8 @@ def model_fn(features, labels, mode, params): # pylint: disable=unused-argument
152152
initialized_variable_names = {}
153153
scaffold_fn = None
154154
if init_checkpoint:
155-
(assignment_map,
156-
initialized_variable_names) = modeling.get_assigment_map_from_checkpoint(
157-
tvars, init_checkpoint)
155+
(assignment_map, initialized_variable_names
156+
) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
158157
if use_tpu:
159158

160159
def tpu_scaffold():

run_squad.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -576,9 +576,8 @@ def model_fn(features, labels, mode, params): # pylint: disable=unused-argument
576576
initialized_variable_names = {}
577577
scaffold_fn = None
578578
if init_checkpoint:
579-
(assignment_map,
580-
initialized_variable_names) = modeling.get_assigment_map_from_checkpoint(
581-
tvars, init_checkpoint)
579+
(assignment_map, initialized_variable_names
580+
) = modeling.get_assignment_map_from_checkpoint(tvars, init_checkpoint)
582581
if use_tpu:
583582

584583
def tpu_scaffold():

0 commit comments

Comments
 (0)