Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enh(ai/llm2): Document OLMo as a fully open model #12545

Merged
merged 3 commits into from
Jan 31, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion admin_manual/ai/app_llm2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,27 @@ Known Limitations
* Language models are likely to generate false information and should thus only be used in situations that are not critical. It's recommended to only use AI at the beginning of a creation process and not at the end, so that outputs of AI serve as a draft for example and not as final product. Always check the output of language models before using it.
* Make sure to test the language model you are using it for whether it meets the use-case's quality requirements
* Language models notoriously have a high energy consumption, if you want to reduce load on your server you can choose smaller models or quantized models in exchange for lower accuracy
* Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI)
* Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI)

Addendum: Running with a fully open model
-----------------------------------------

If you would like to use a fully open model that scores a green score on our Ethical AI rating, we recommend the following model:

* OLMo 2 (either in 7B or 13B): `<https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct-GGUF>`_

What makes OLMo a fully open model?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The code for training, fine-tuning and inference of the model is publicly available and fully open source
* The training data with which the model is pretrained is publicly available
* The model itself is publicly available and fully open source
* The instruction tuning data is publicly available
* The Reinforcement learning model is publicly available and fully open source

Limitations
~~~~~~~~~~~

* OLMo currently only works well with English language input
* In our tests it sometimes produced hallucinated or garbled output; make sure to thoroughly test the model for your use case
* It cannot use tools, so cannot be used in conjunction with :ref:`Context Agent <ai-app-context_agent>`