Bug Report: Hardcoded schema expectation breaks compatibility with standard Vespa deployments

[parisni]

Environment

• Vispana Version: v0.0.1-SNAPSHOT (latest Docker image vispana/vispana:latest)
• Vespa Version: 8.521.17
• Deployment: Kubernetes cluster

Problem Description

Vispana fails to load with a 404 error when connecting to a standard Vespa cluster due to a hardcoded assumption about schema file names.

Error Details

Error Message:

404 Not Found: {"error-code":"NOT_FOUND","message":"Session 4 does not contain a file at path '/schemas/cluster-controllers.sd'"}

Full Stack Trace:

org.springframework.web.client.HttpClientErrorException$NotFound: 404 Not Found: "{"error-code":"NOT_FOUND","message":"Session 4 does not contain a file at path '/schemas/cluster-controllers.sd'"}"
	at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:112)
	at com.vispana.vespa.state.helpers.Request.requestGet(Request.java:10)
	at com.vispana.vespa.state.assemblers.ContentAssembler.lambda$fetchSchemaContent$2(ContentAssembler.java:87)
	at com.vispana.vespa.state.assemblers.ContentAssembler.fetchSchemaContent(ContentAssembler.java:97)
	at com.vispana.vespa.state.assemblers.ContentAssembler.lambda$assemble$0(ContentAssembler.java:71)
	at com.vispana.vespa.state.assemblers.ContentAssembler.assemble(ContentAssembler.java:76)
	at com.vispana.vespa.state.VespaStateClient.lambda$vespaState$8(VespaStateClient.java:47)

Root Cause

The issue is in ConfigNodesAssembler.java line 20:

private final static String CLUSTER_CONTROLLERS_SCHEMA = "cluster-controllers";

Workaround Attempted

We tried creating a dummy cluster-controllers.sd schema file, but Vespa doesn't allow hyphens in schema names (requires underscores), creating an incompatibility between Vispana's expectation (cluster-controllers) and Vespa's requirements (cluster_controllers).

[broknloop]

Would you mind trying a docker pull or preferably running Vispana with a defined version1.6.8? (e.g., docker run -p 4000:4000 --network vespanet vispana/vispana:1.6.8

I don't see a hardcoded schema in main from the code you just shared: https://github.com/spotify/vispana/blob/main/src/main/java/com/vispana/vespa/state/assemblers/ConfigNodesAssembler.java#L20

[parisni]

Thanks for your support. Just tried with 1.6.8, same stack trace. Agreed there is no cluster-controllers.sd hardcoded in the source. No idea how comes the stacktrace has this schema

[parisni]

Turns out it's a schema.sd problem. I don't know right now the exact problem i am facing with the schema, but using an other schema and vispana works fine with the exact same instance.

[francoisWeber]

Hi, bellow is a schema that provokes the problem (@parisni please validate). Let me mention that Vespa is happy with this config file and I'm able to feed/query a Vespa having this configuration. The *.sd:

schema ad {
    document ad {
        field list_id type string {
            indexing: summary | index
        }
        field subject type string {
            indexing: summary | index
            index: enable-bm25
        }
        field body type string {
            indexing: summary | index
            index: enable-bm25
        }
        field category_id type int {
            indexing: attribute | summary
        }
        field price_cents type int {
            indexing: attribute | summary
        }
        field embedding type tensor<float>(x[768]) {
            indexing: attribute | index
            attribute {
                distance-metric: angular
            }
            index {
                hnsw {
                    max-links-per-node: 16
                    neighbors-to-explore-at-insert: 200
                }
            }
        }
    }
    fieldset default {
        fields: subject, body
    }
    rank-profile default {
        first-phase {
            expression {
                bm25(subject) + bm25(body)
            }
        }
    }
    rank-profile vector {
        first-phase {
            expression {
                closeness(field, embedding)
            }
        }
    }
    rank-profile hybrid inherits default {
        first-phase {
            expression {
                0.5 * bm25(subject) + 0.5 * bm25(body) + 0.8 * closeness(field, embedding)
            }
        }
    }
}

[parisni]

Turns out the schema name leads to the problem.

• ad.sd -> vispana broken
• ad_schema.sd -> vispana happy