Skip to content

Invalid index_id with ingest v2 returns 429 #5721

Merged
rdettai merged 3 commits intomainfrom
fix-invalid-index-id-code
Apr 10, 2025
Merged

Invalid index_id with ingest v2 returns 429 #5721
rdettai merged 3 commits intomainfrom
fix-invalid-index-id-code

Conversation

@rdettai
Copy link
Collaborator

@rdettai rdettai commented Mar 25, 2025

Description of the issue

When using index templates, specifying and index name that matches the pattern but has illegal characters results in a 429 response code instead of a 400.

Description of the problem:

  • the index_id validity is not validated until the metastore is called
  • calling GetOrCreateOpenShards with an invalid index_id on the metastore fails with MetastoreError::JsonDeserializeError
  • a failure to call the metastore is logged but not repported to the ingest workbench
  • because the router is not populated, the workbench fails with "shard not found errors" for the failing index but also other targetted indexes batched with it

Proposed solution

The index id is now validated in quickwit-serve, before calling the ingest router (ES bulk and native APIs)

How was this PR tested?

Added unit and integration (python) tests

@rdettai rdettai self-assigned this Mar 25, 2025
@rdettai rdettai requested a review from guilload March 25, 2025 15:30
@rdettai rdettai changed the title Failure to create an index with ingest v2 returns 429 Invalid index_id with ingest v2 returns 429 Mar 25, 2025
)
})?;

// Validate index id early because propagating back the right error (400)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Validate index id early because propagating back the right error (400)
// Validate index ID early because propagating back the right error (400)

// from deeper ingest layers is harder
if validate_identifier("", &index_id).is_err() {
return Err(IngestServiceError::BadRequest(
"invalid index_id".to_string(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"invalid index_id".to_string(),
"invalid index ID".to_string(),

@rdettai rdettai force-pushed the fix-invalid-index-id-code branch from ab3a96d to 0106709 Compare April 10, 2025 08:45
@rdettai rdettai enabled auto-merge (squash) April 10, 2025 08:45
@rdettai rdettai merged commit aec968e into main Apr 10, 2025
8 checks passed
@rdettai rdettai deleted the fix-invalid-index-id-code branch April 10, 2025 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants