Image Caption Tensorflow

• Image caption model base on Show and Tell: A Neural Image Caption Generator with some modifications.
• The dataset come from Microsoft COCO 2014 train and valid, and we do some redistribution.
• This model is trained for NTHU CS565600 image caption competition.
• Our model achieved 0.944 CIDEr-D score on single model, which is the 1st place of the Image Caption Kaggle Competition.
• We provide end to end scripts and pretrained weight for reproduction.
• This slides briefly describe the implementation
• If you meet any problem, feel free to contact ([email protected]).

Requirements

Here are some required libraries.

General

• python >= 3.6
• cuda >= 10.0 (or base on your tensorflow version)

Python

• please refer requirements.txt

Reproduce from scratch

Download the data

cd data
sh download.sh

Redistribute the data (Competition required)

python split.py

Generate the image features

We use the NASNet model pretrained by Keras to get the image features. This step may took over one hour.

python nasnet.py

Create tensorflow records

cd ../script
python create_tfrecord.py

Train

python train.py

Evaluate on validation set

python inference.py

Performance

	CIDEr-D
Single Model	0.944
Ensemble Model	0.955