Skip to content

OpenAI.Request.PromptLossWeight

Jump to bottom
Andrew Lambert edited this page Feb 9, 2023 · 2 revisions

OpenAI.Request.PromptLossWeight

Property declaration

 Dim PromptLossWeight As Single

Remarks

The weight to use for loss on the prompt tokens. This controls how much the model tries to learn to generate the prompt (as compared to the completion which always has a weight of 1.0), and can add a stabilizing effect to training when completions are short.

If prompts are extremely long (relative to completions), it may make sense to reduce this weight so as to avoid over-prioritizing learning the prompt.

Wiki home | Project page | Bugs | Become a sponsor

Text and code examples are Copyright ©2023-24 Andrew Lambert, offered under the CC BY-SA 3.0 License.

Overview

Clone this wiki locally