Skip to content

Conversation

likejazz
Copy link

@likejazz likejazz commented Jun 6, 2024

No description provided.

@jameswdelancey
Copy link
Contributor

These appear to be llama2 ports vs llama3. They need to at least use tiktoken start tokens to work with the base, not 0 like sentencepiece, and if they work with tinystories, i suspect they are using sentencepiece tokenizer. Chat/instruct require a stack of tokens to start dialogue else you will be unimpressed with the output. There is a predictable output at zero temperature for both base/pretrained and chat/inspect that you can use to validate. You can copy this diff for the meat of what you need: jameswdelancey@7815cd3#diff-8935a7a088435e2ddf7315451f07fae16810932fb3a0a5d706a2eead1618af26R850-R854

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants