New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement text chunking processor with fixed token length and delimiter algorithm #607

Merged

model-collapse merged 189 commits into opensearch-project:main from yuye-aws:feature/documentChunkingProcessor

Mar 18, 2024

Merged

feat: implement text chunking processor with fixed token length and delimiter algorithm #607

bug fix: only update runtime max chunk limit when enabled

Codecov / codecov/patch succeeded Mar 18, 2024 in 0s

97.89% of diff hit (target 82.62%)

View this Pull Request on Codecov

97.89% of diff hit (target 82.62%)

Annotations

Check warning on line 159 in src/main/java/org/opensearch/neuralsearch/processor/TextChunkingProcessor.java

codecov / codecov/patch

src/main/java/org/opensearch/neuralsearch/processor/TextChunkingProcessor.java#L158-L159

Added lines #L158 - L159 were not covered by tests

View more details on Codecov