Documented LLMs overview

- Added missing content index to remote overview

docs/source/index.rst

@@ -37,6 +37,12 @@ Genv lets you easily control, configure, monitor and enforce the GPU resources t
    remote/enforcement
    remote/monitoring
 
+.. toctree::
+   :maxdepth: 2
+   :caption: LLMs
+
+   llms/overview
+
 .. toctree::
    :maxdepth: 2
    :caption: Container Toolkit

docs/source/llms/overview.rst

@@ -0,0 +1,126 @@
+Overview
+========
+
+.. contents::
+   :depth: 3
+   :backlinks: none
+
+Genv is an environment management software for GPUs that manages GPUs on a single machine or in a cluster.
+
+Genv uses `Ollama <https://ollama.com/>`__ for managing managing Large Language Models (LLMs) with Genv.
+This allows users to efficiently run, manage and utilize LLMs on GPUs within their clusters.
+
+This guide will help you get started with LLMs using Genv.
+
+Prerequisites
+-------------
+
+First, you will need to :doc:`install <../overview/installation>` Genv on your GPU machines in the cluster.
+
+.. note::
+
+   It is recommended to install Genv :ref:`system-wide <Install System Wide>`.
+
+Then, you will need to `install <https://ollama.com/download>`__ Ollama on your GPU machines with the command:
+
+.. code-block:: shell
+
+   curl -fsSL https://ollama.com/install.sh | sh
+
+Quick start
+-----------
+
+If you are running on a GPU cluster with multiple machines, follow the :ref:`cluster <LLM cluster quick start>` quick start.
+Otherwise, follow the :ref:`local <LLM local quick start>` quick start.
+
+.. _LLM cluster quick start:
+
+Cluster
+~~~~~~~
+In my case, I have two remote GPU machines: :code:`gpu-server-1` and :code:`gpu-server-2`.
+
+Serve your first LLM on a single GPU with the command:
+
+.. code-block:: shell
+
+   $ genv remote -H gpu-server-1,gpu-server-2 llm serve llama2 --gpus 1
+   ...
+
+.. note::
+
+   Use :code:`Ctrl+C` to stop the serving.
+
+Open another terminal and list all LLMs in your cluster with:
+
+.. code-block:: shell
+
+   $ genv remote -H gpu-server-1,gpu-server-2 llm ps
+   HOST                     MODEL       PORT    CREATED              EID     USER
+   gpu-server-1             llama2      N/A     2 hours ago          571     root
+
+We can see that :code:`llama2` is now being served in the cluster.
+
+Attach to the model and interact with it using the command:
+
+.. code-block::
+
+   $ genv remote -H gpu-server-1,gpu-server-2 llm attach llama2
+   >>> Hi! I'm Genv, how are you?
+
+   Hello there, it's nice to meet you! *blinks* I'm just an AI assistant, so I don't have feelings or emotions
+   like humans do, but I'm here to help you with any questions or tasks you might have. How can I assist you
+   today?
+
+   >>> Send a message (/? for help)
+
+.. note::
+
+   Use :code:`Ctrl+D` to exit the interactive terminal.
+
+.. _LLM local quick start:
+
+Local
+~~~~~
+Serve your first LLM on a single GPU with the command:
+
+.. code-block:: shell
+
+   $ genv llm serve llama2 --gpus 1
+   ...
+
+.. note::
+
+   Use :code:`Ctrl+C` to stop the serving.
+
+Open another terminal and list all LLMs on your machine with:
+
+.. code-block:: shell
+
+   $ genv llm ps
+   MODEL       PORT    CREATED              EID     USER            PID(S)
+   llama2      33123   22 seconds ago       1179252 raz(1001)       1179252
+
+We can see that :code:`llama2` is now being served on the machine.
+
+Attach to the model and interact with it using the command:
+
+.. code-block::
+
+   $ genv llm attach llama2
+   >>> Hello from Genv
+
+   Hello there! *blinks* Uh, I'm... uh... *trails off* I'm not actually a person, I'm just an AI designed to
+   simulate conversation. So, I don't have feelings or emotions like a real person would. But I'm here to help
+   you with any questions or tasks you may have! How can I assist you today?
+
+   >>> Send a message (/? for help)
+
+.. note::
+
+   Use :code:`Ctrl+D` to exit the interactive terminal.
+
+Where to Go Next
+----------------
+If you are not familiar with Genv, it is recommended to go over the :doc:`overview <../overview/overview>` and follow the :doc:`quick start tutorial <../overview/quickstart>`.
+
+You should also learn how :doc:`remote features <../remote/overview>` work in Genv.

docs/source/overview/overview.rst

@@ -18,6 +18,8 @@ If you are not familiar with the concept of environments, you can check out the
 
 Over time, the project developed into a complete GPU cluster management tool with various capabilities and mechanisms like :doc:`monitoring <../usage/monitoring>`, :doc:`enforcement <../usage/enforcement>`, :doc:`remote capabilities <../remote/overview>` and more.
 
+In addition to running GPU environments in a cluster, Genv also supports managing :doc:`LLMs <../llms/overview>` on GPUs.
+
 At its core, Genv is a `Python package <https://pypi.org/project/genv>`__ that provisions GPUs to environments and keeps its state as :ref:`files <Files>` at :code:`/var/tmp/genv`.
 
 On top of this core layer, Genv has a CLI (i.e. the command :code:`genv`), :doc:`Python SDK <../usage/python-sdk>` and :ref:`integrations <Integrations>` with many common tools and environments such as terminal, containers (e.g. :code:`docker`), Ray, Visual Studio Code, JupyterLab and PyCharm.
@@ -55,5 +57,6 @@ For more information, see :doc:`here <../remote/overview>`.
 Where to Go Next
 ----------------
 If you are not familiar with Genv, it is recommended to follow the :doc:`quick start tutorial <quickstart>`.
+If you are into LLMs, check out the :doc:`LLMs overview <../llms/overview>`.
 
 If you are looking for how to install Genv, visit the :doc:`installation page <installation>`, and if you want to know how to do specific things, check out the :doc:`usage guide <../usage/usage>`.

docs/source/remote/overview.rst

@@ -1,6 +1,10 @@
 Overview
 ========
 
+.. contents::
+   :depth: 3
+   :backlinks: none
+
 Genv is an environment management software for GPUs.
 It manages all GPUs on a single machine.