[BUG]: Weaviate: Indexes seem to be queries with underscores removed?

[bbrala]

How are you running VectorAdmin?

Docker (local)

What happened?

Error when trying to sync.

Are there known steps to reproduce?

Connected to weaviate with existing named classes that do not use camelcase, then you get following messages consistently when trying to sync.

vector-admin | ClientError: Cannot query field "Bp10153C51068DocumentChunksV1" on type "GetObjectsObj". Did you mean "Bp101_53c51068_documentChunks__v1", "Bp131_13b35c33_documentChunks", "Bp160_15dede47_documentChunks", "Bp100_e19743d9_documentChunks__v1", or "Bp48_8ae7d6f8_documentChunks__v1"?: {"response":{"errors":[{"locations":[{"column":6,"line":1}],"message":"Cannot query field \"Bp10153C51068DocumentChunksV1\" on type \"GetObjectsObj\". Did you mean \"Bp101_53c51068_documentChunks__v1\", \"Bp131_13b35c33_documentChunks\", \"Bp160_15dede47_documentChunks\", \"Bp100_e19743d9_documentChunks__v1\", or \"Bp48_8ae7d6f8_documentChunks__v1\"?","path":null}],"status":200,"headers":{}},"request":{"query":"{Get{Bp10153C51068DocumentChunksV1(limit:1){_additional { vector }}}}"}}

And

vector-admin | Weaviate:namespaceCountWithClient Cannot query field "Bp111F0880665DocumentChunksV1" on type "AggregateObjectsObj". Did you mean "Bp126_0dcea379_documentChunks", "Bp138_15a0c090_documentChunks__v1", "Bp58_ee2680db_documentChunks__v1", "Bp111_f0880665_documentChunks__v1", or "Bp125_414b683f_documentChunks"?: {"response":{"errors":[{"locations":[{"column":12,"line":1}],"message":"Cannot query field \"Bp111F0880665DocumentChunksV1\" on type \"AggregateObjectsObj\". Did you mean \"Bp126_0dcea379_documentChunks\", \"Bp138_15a0c090_documentChunks__v1\", \"Bp58_ee2680db_documentChunks__v1\", \"Bp111_f0880665_documentChunks__v1\", or \"Bp125_414b683f_documentChunks\"?","path":null}],"status":200,"headers":{}},"request":{"query":"{Aggregate{Bp111F0880665DocumentChunksV1{meta { count }}}}"}}

And more of those.

[bbrala]

After changing all this.camelCase things to seem to sync.

Very ugly hack:
embeddings-fix.patch

[timothycarambat]

Was this namespace created by AnythingLLM or was it already existing? We shouldn't have issues with referring to spaces created via AnythingLLM since in theory, the rules of namespace should be consistent from creation/update/read/delete

[bbrala]

No it was an existing one. Was under the inpression it would allow some interfacing to our existing weaviate database.

[timothycarambat]

Ah, so technically speaking, if the workspace and the namespace in Weaviate have the same slug value - it would work when it returns vector search results, but they would never show in citations and the embeddings created out of AnythingLLM need to have a text metadata property to even be injected. This kind of use-case is not explicitly supported because of the litany of possible issues from someone embedding their data in any way they deem fit and it not being compatible with the way AnythingLLM expects and manages vectors and their associated metadata.

[bbrala]

Ok so vector admin is technically AnythingLVM-admin?

Unfortunate. Thanks.

[timothycarambat]

No, they are two totally separate repos. If it was just for anythingLLM it would be in this repo

[bbrala]

Ok, but you do not aim to support existing databases then. Having fixed rules on how things are named seems kinda counter intuitive.

I could contribute a setting of some sort to enable/disable the forces camelcase if you like.

[timothycarambat]

I just realized we have been talking on the vector-admin repo which explains all the confusion. This whole time I thought this was AnythingLLM's repo where I get 99% of issues/notifications.

This is a bug for VA, lol. Yes underscores should not matter in this context

[bbrala]

Hehe, thanks :)

[HassanAITBRIK]

Any update about this issue ?