Loading VectorStoreIndex from StorageContext

[Alnasser0]

how do I load VectorStoreIndex from StorageContext that I persisted?

[dosubot[bot]]

To load a VectorStoreIndex from a persisted StorageContext, you can use the following code:

from llama_index.core import VectorStoreIndex
from llama_index.core.storage.storage_context import StorageContext
from llama_index.core.indices.loading import load_index_from_storage

# Load storage context from persisted directory
storage_context = StorageContext.from_defaults(persist_dir="./path_to_persisted_storage")

# Load your index from the storage context
index = load_index_from_storage(storage_context=storage_context)

Make sure that the persist_dir is correctly set to the directory where your storage context was persisted. This method allows you to retrieve the index without reinitializing it or losing previously stored documents ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Alnasser0]

@dosu how can I check if data persisted on my disk, I don't want to create a vector index. However, if the data is missing, I want to create the vector index. Can you do that in code? I am currently using DeepLakeVectorStore and VectorStoreIndex with local embedding model.

[dosubot[bot]]

To check if data is persisted on your disk and create a vector index only if the data is missing using DeepLakeVectorStore and VectorStoreIndex, you can use the following code:

from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.deeplake import DeepLakeVectorStore
import os

# Define the dataset path
dataset_path = "<dataset_path>"

# Check if the dataset path exists
if not os.path.exists(dataset_path):
    # If the dataset does not exist, create a new vector store and index
    vector_store = DeepLakeVectorStore(dataset_path=dataset_path)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)
    
    # Load documents and build index
    index = VectorStoreIndex.from_documents(
        documents, storage_context=storage_context
    )
else:
    # If the dataset exists, reload the existing index
    vector_store = DeepLakeVectorStore(dataset_path=dataset_path)
    index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

This code snippet checks if the dataset path exists on the disk. If it does not exist, it creates a new DeepLakeVectorStore and builds a new index. If the dataset path exists, it reloads the existing index from the vector store ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}