Skip to content

Commit

Permalink
allow chunk_overlap with zero value (#457) #none
Browse files Browse the repository at this point in the history
Co-authored-by: Tadashi <[email protected]>
  • Loading branch information
pvl and taprosoft authored Nov 5, 2024
1 parent d127fec commit 908452c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion libs/ktem/ktem/index/file/pipelines.py
Original file line number Diff line number Diff line change
Expand Up @@ -742,7 +742,7 @@ def route(self, file_path: str | Path) -> IndexPipeline:
loader=reader,
splitter=TokenSplitter(
chunk_size=chunk_size or 1024,
chunk_overlap=chunk_overlap or 256,
chunk_overlap=chunk_overlap if chunk_overlap is not None else 256,
separator="\n\n",
backup_separators=["\n", ".", "\u200B"],
),
Expand Down

0 comments on commit 908452c

Please sign in to comment.