Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Hackathon 8th No.28】RFC:在 PaddleNLP 中复现 Phi3 模型 #1103

Merged
merged 4 commits into from
Apr 7, 2025

Conversation

robinbg
Copy link
Contributor

@robinbg robinbg commented Apr 2, 2025

No description provided.

Copy link

paddle-bot bot commented Apr 2, 2025

你的PR提交成功,感谢你对开源项目的贡献!
请检查PR提交格式和内容是否完备,具体请参考示例模版
Your PR has been submitted. Thanks for your contribution!
Please check its format and content. For this, you can refer to Template and Demo.

@CLAassistant
Copy link

CLAassistant commented Apr 2, 2025

CLA assistant check
All committers have signed the CLA.

@luotao1 luotao1 self-assigned this Apr 3, 2025
@luotao1
Copy link
Collaborator

luotao1 commented Apr 3, 2025

请签署CLA @robinbg

@DrownFish19
Copy link
Collaborator

  1. 需注意实现_get_name_mappings方法用于转换参数,参考llama代码如下
    https://github.com/PaddlePaddle/PaddleNLP/blob/bfd053db0897943f5d4d116dde755dbf21d18b23/paddlenlp/transformers/llama/modeling.py#L1334-L1366
  2. 需实现模型并行相关代码,可暂不实现sequence_parallel相关逻辑。if config.tensor_parallel_degree > 1:相关代码,参考llama代码如下(不仅一处位置,可检索后对应添加):
    https://github.com/PaddlePaddle/PaddleNLP/blob/bfd053db0897943f5d4d116dde755dbf21d18b23/paddlenlp/transformers/llama/modeling.py#L775-L813
  3. 建议优先完成单卡模型组网,后续再支持模型并行。

Copy link
Collaborator

@DrownFish19 DrownFish19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DrownFish19 DrownFish19 merged commit 2f05fd5 into PaddlePaddle:master Apr 7, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants