SupabaseVectorStore lacks metadata filter support

[navjot227]

Describe the bug
Adding metadata filter to supabase vector store doesn't take effect.

To Reproduce
Code to reproduce the behavior:

export async function chatWithLlama(workspaceId: string, userMessage: string, chatHistory: LlamaChatMessage[]) {
    Settings.embedModel = new OpenAIEmbedding();
    Settings.llm = new LlamaOpenAI({ model: process.env.LLM_MODEL! });
    const supabaseClient = createAdminClient();
    const vectorStore = new SupabaseVectorStore({
        client: supabaseClient,
        table: "document",
    });

    index = await VectorStoreIndex.fromVectorStore(vectorStore);
    const retriever = index.asRetriever({
        filters: {
            filters: [{
                key: "workspaceId",
                operator: "==",
                value: "abc"
            }]
        }
    })

    const chatEngine = index.asChatEngine({
        retriever,
        chatHistory: chatHistory,
        systemPrompt: `You are a helpful assistant that can answer questions about the documents provided. 
                       You can use the documents to answer questions. 
                       Say 'I don't know' if you can't find enough information in the provided documents.`
    });
    const response = await chatEngine.chat({ message: userMessage });
    console.log(JSON.stringify(response));
    return { response: response.message.content };
}

Expected a filter to have shown up in the logs. No filter shows up in the logs and records are not filtered as expected

create or replace function match_documents (
  match_count int,
  query_embedding vector(1536),
  filter jsonb default '{}'
)
returns table (
  id uuid,
  content text,
  metadata jsonb,
  similarity float
)
language plpgsql stable
as $$
begin
  -- This line will print the received filter 
  raise warning 'match_documents received filter: %', filter;

  return query
    select
      d.id,
      d.content,
      d.metadata,
      1 - (d.embedding <=> query_embedding) as similarity
    from document as d
    where d.metadata @> filter
    order by d.embedding <=> query_embedding
    limit match_count;
end;
$$;

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. macOS, Linux]
• JS Runtime / Framework / Bundler (select all applicable)
• Node.js
• Deno
• Bun
• Next.js
• ESBuild
• Rollup
• Webpack
• Turbopack
• Vite
• Waku
• Edge Runtime
• AWS Lambda
• Cloudflare Worker
• Others (please elaborate on this)
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

I would have expected filters be handled in SupabaseVectorStore, but that doesn't seem to be the case.

[marcusschiesser]

Yes, filters are not implemented for supabase. Want to send a PR? QdrantVectorStore impements filters so I think it's a good start

[jerinthomascarmel]

I just thought of play around the Supabase Vectore store. But i am unable to add some Text Node to the database.

Here is the code that i tried to run .

import 'dotenv/config';                                  

import { Settings } from 'llamaindex';                     
import { OpenAI } from '@llamaindex/openai';
import { OpenAIEmbedding } from '@llamaindex/openai';


Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY! });
Settings.embedModel = new OpenAIEmbedding({ apiKey: process.env.OPENAI_API_KEY! });

import { createClient } from '@supabase/supabase-js';        
import { SupabaseVectorStore } from '@llamaindex/supabase';
import { storageContextFromDefaults, VectorStoreIndex } from 'llamaindex';
import { TextNode } from '@llamaindex/core/schema';

const supabaseClient = createClient(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_KEY!,
);

const vectorStore = new SupabaseVectorStore({
  client: supabaseClient,
  table: 'supabase_document',                     
});


const documents = [
    new TextNode({
        text: "hellow",
        metadata: {
            "author":"author",
            "theme":"this is the theme"
        }
    }),
    new TextNode({
        text: "The Godfather",
        metadata: {
            director: "Francis Ford Coppola",
            theme: "Mafia",
        },
    }),
    new TextNode({
        text: "Inception",
        metadata: {
            director: "Christopher Nolan",
        },
    }),
];

async function main() {
    const storageContext = await storageContextFromDefaults({ vectorStore });
    const index = await VectorStoreIndex.fromDocuments(documents, { storageContext });
}

main();

error that i got :
throw new Error(Error inserting documents: ${JSON.stringify(error, null, 2)});
^

Error: Error inserting documents: {}
at SupabaseVectorStore.add (D:\open_source\llamaindex\supabasedemo js\node_modules@llamaindex\supabase\dist\index.cjs:7525:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async addNodesToVectorStores (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:254:28)
at async VectorStoreIndex.insertNodes (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2528:9)
at async VectorStoreIndex.buildIndexFromNodes (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2422:9)
at async VectorStoreIndex.init (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2372:13)
at async VectorStoreIndex.fromDocuments (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2450:20)

Node.js v20.18.1
PS D:\open_source\llamaindex\supabasedemo js>

I get to know that python based llamaindex doesn't have the supabase client , we can directly give the posgres connection url . But this option is not in the case of Typescript LLammaindex .

Do i missing something in that code , can you give how to solve this problem .

[marcusschiesser]

can you try using new Document instead of new TextNode (see the supabase example examples/storage/supabase/index.ts)

[jerinthomascarmel]

Thanks for the suggestion! I replaced new TextNode with new Document (following the example in examples/storage/supabase/index.ts) and switched to using the OpenAI embeddings and LLMs.
While adding Document nodes to the vector store, I hadn’t created the underlying table in Supabase first. When configuring the vector store, it asks for the table schema, which I don’t yet know—I had assumed the library would automatically infer and create it for me.

After creating an empty table named "document", I got the following error when inserting:

Error: Error inserting documents: {
  "code": "PGRST204",
  "details": null,
  "hint": null,
  "message": "Could not find the 'content' column of 'document' in the schema cache"
}
    at SupabaseVectorStore.add (D:\open_source\llamaindex\supabasedemo js\node_modules\@llamaindex\supabase\dist\index.cjs:7523:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async addNodesToVectorStores (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:254:28)
    at async VectorStoreIndex.insertNodes (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2528:9)
    at async VectorStoreIndex.buildIndexFromNodes (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2422:9)
    at async VectorStoreIndex.init (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2372:13)
    at async VectorStoreIndex.fromDocuments (D:\open_source\llamaindex\supabasedemo js\node_modules\llamaindex\indices\dist\index.cjs:2450:20)
PS D:\open_source\llamaindex\supabasedemo js> 

Point me toward how to automate schema creation? Thanks in advance, and sorry for the basic question!