AttributeError: can't set attribute 'eos_token'

[7Alive7]

利用全量训好的chatglm3-6B模型进行predict时，出现了如下问题：
Traceback (most recent call last):
File "/apdcephfs_cq3/share_1567347/share_info/ritchiezhou/LLaMA-Factory-main/src/train_bash.py", line 14, in
main()
File "/apdcephfs_cq3/share_1567347/share_info/ritchiezhou/LLaMA-Factory-main/src/train_bash.py", line 5, in main
run_exp()
File "/apdcephfs_cq3/share_1567347/share_info/ritchiezhou/LLaMA-Factory-main/src/llmtuner/tuner/tune.py", line 26, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "/apdcephfs_cq3/share_1567347/share_info/ritchiezhou/LLaMA-Factory-main/src/llmtuner/tuner/sft/workflow.py", line 28, in run_sft
model, tokenizer = load_model_and_tokenizer(model_args, finetuning_args, training_args.do_train, stage="sft")
File "/apdcephfs_cq3/share_1567347/share_info/ritchiezhou/LLaMA-Factory-main/src/llmtuner/tuner/core/loader.py", line 67, in load_model_and_tokenizer
tokenizer = AutoTokenizer.from_pretrained(
File "/root/anaconda3/envs/zt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 738, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/root/anaconda3/envs/zt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2017, in from_pretrained
return cls._from_pretrained(
File "/root/anaconda3/envs/zt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2249, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/1109_wz_full_wz112K+choose_5K/tokenization_chatglm.py", line 93, in init
super().init(padding_side=padding_side, clean_up_tokenization_spaces=clean_up_tokenization_spaces, **kwargs)
File "/root/anaconda3/envs/zt_env/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 363, in init
super().init(**kwargs)
File "/root/anaconda3/envs/zt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1604, in init
super().init(**kwargs)
File "/root/anaconda3/envs/zt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 861, in init
setattr(self, key, value)
AttributeError: can't set attribute 'eos_token'

训练代码指令如下：
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 deepspeed src/train_bash.py
--stage sft
--model_name_or_path /apdcephfs_cq2/share_1567347/share_info/llm_models/chatglm3-6b
--do_train
--dataset wz_112K+choose_5.3K
--template default
--finetuning_type full
--output_dir output/wz_full_out/1113_wz_full_wz112K+choose_5K
--per_device_train_batch_size 4
--gradient_accumulation_steps 4
--lr_scheduler_type cosine
--logging_steps 10
--save_steps 1000
--learning_rate 1e-5
--num_train_epochs 2.0
--fp16
--deepspeed deep_config.json

预测代码指令如下：
CUDA_VISIBLE_DEVICES=0 python src/train_bash.py
--stage sft
--model_name_or_path /apdcephfs_cq3/share_1567347/share_info/ritchiezhou/LLaMA-Factory-main/output/wz_full_out/1109_wz_full_wz112K+choose_5K
--do_predict
--dataset choose_one
--template default
--finetuning_type full
--output_dir output/wz_choose_out/1109_choose
--per_device_eval_batch_size 8
--max_samples 100
--predict_with_generate \

[hiyouga]

#1307 (comment)