Documentation for match_only_text field #6041

rishabhmaurya · 2024-01-05T00:18:28Z

Description

Document the usage of match_only_text field introduced as part of opensearch-project/OpenSearch#11039

Issues Resolved

Closes #5427

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

rishabhmaurya · 2024-01-05T00:22:46Z

@andrross @harshavamsi @msfroh please let me know if I missed anything which you think could be relevant to the users

msfroh

Looks good! I think it covers the essentials of the field type. Just added a couple of minor comments/questions.

msfroh · 2024-01-08T19:37:58Z

_field-types/supported-field-types/match-only-text.md

+
+## Parameters
+
+While `match_only_text` supports most parameters available for text fields, modifying most of them can actually be counterproductive. This field type thrives on simplicity and efficiency, minimizing data stored in the index to optimize storage costs. Therefore, sticking with the default settings is generally the best approach. Any adjustments beyond analyzer settings could reintroduce overhead and negate the efficiency benefits of `match_only_text`.


Do the various parameters work? Like, could you theoretically coerce a match_only_text field to behave more like a text field? (If so, I don't think we should support those parameters. Could we just ignore them, maybe with a deprecation warning?)

All unsupported params or unsupported values for a param will result in an error. But there are other parameters like stored, term_vectors which are still supported but shouldn't be enabled as it defeats the purpose.
I like your idea to be more precise here, so I will list down the ones which are supported.

done, thanks

msfroh · 2024-01-08T19:38:25Z

_field-types/supported-field-types/match-only-text.md

+
+## Migrating from text field
+
+Reindex API can be used to migrate from text field to `match_only_text` by updating the right mapping in the target index.


Maybe add an example of this?

andrross · 2024-01-08T22:50:18Z

_field-types/supported-field-types/match-only-text.md

+ - Supports most query types, but not interval/span queries.
+
+
+Choosing `match_only_text` means prioritizing efficient full-text search over complex ranking and positional queries, while also optimizing storage costs. It excels when you need to quickly find documents containing specific terms without the overhead of storing frequencies and positions, leading to significantly smaller indexes. This translates to lower storage costs, especially for large datasets. However, it's not the best choice for ranking results based on relevance or for queries that hinge on term proximity or order, like interval/span queries. While it does support phrase queries, their performance isn't as efficient as with text field type. So, if pinpointing exact phrases or their locations within documents is essential, consider using text field type.


While it does support phrase queries, their performance isn't as efficient as with text field type. So, if pinpointing exact phrases or their locations within documents is essential, consider using text field type.

Does this maybe downplay it too much? match_only_text supports phrase queries but performance will likely be very very bad, right? I'd maybe tweak the last sentence:

"So, if pinpointing exact phrases or their locations within documents is essential, use the text field type instead."

Makes sense! In addition to what you're suggesting, maybe I can add a line on how phrase queries work here and how bad/optimal they could depending on query and workload.

kolchfa-aws · 2024-01-12T21:06:44Z

@rishabhmaurya Let me know when you've made the changes and are ready for me to review this documentation. Thanks a lot!

Signed-off-by: Rishabh Maurya <[email protected]>

rishabhmaurya · 2024-01-23T04:15:15Z

@kolchfa-aws I somehow missed this one and was long overdue. Have addressed the comments now.

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2024-01-26T21:15:30Z

@rishabhmaurya Thanks for putting up the PR! I reviewed and pushed my changed in this commit. Could you review to make sure it preserves technical accuracy? Then we'll move to editorial review. Thanks!

rishabhmaurya · 2024-01-30T18:02:48Z

@kolchfa-aws looks good to me, thanks for putting this up.

natebower

@kolchfa-aws Please see my comments and changes and let me know if you have any questions. Thanks!

natebower · 2024-01-30T18:11:10Z

_field-types/supported-field-types/match-only-text.md

+
+A `match_only_text` field is different from a `text` field in the following ways:
+
+ - Omits storing positions, frequencies, and norms, reducing storage.


"reducing storage requirements"?

_field-types/supported-field-types/match-only-text.md

natebower · 2024-01-30T18:43:02Z

_field-types/supported-field-types/string.md

@@ -18,4 +18,5 @@ Field data type | Description
 :--- | :---  
 [`keyword`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/keyword/) | A string that is not analyzed. Useful for exact-value search.
 [`text`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/text/) | A string that is analyzed. Useful for full-text search.
+[`match_only_text`]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/match-only-text/) | A space-optimized version of a text field.


Should "text" be in code font?

_field-types/supported-field-types/text.md

natebower · 2024-01-30T18:45:14Z

_field-types/supported-field-types/text.md

@@ -12,12 +12,15 @@ redirect_from:

 # Text field type

-A text field type contains a string that is analyzed. It is used for full-text search because it allows partial matches. Searches with multiple terms can match some but not all of them. Depending on the analyzer, results can be case insensitive, stemmed, stopwords removed, synonyms applied, etc.
+A text field type contains a string that is analyzed. It is used for full-text search because it allows partial matches. Searches with multiple terms can match some but not all of them. Depending on the analyzer, results can be case insensitive, stemmed, stopwords removed, synonyms applied, and so on.


Should "text" be in code font?

_field-types/supported-field-types/text.md

natebower · 2024-01-30T18:47:25Z

_field-types/supported-field-types/text.md



 If you need to use a field for exact-value search, map it as a [`keyword`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/keyword/) instead.
 {: .note }

+The [`match_only_text`]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/match-only-text/) field is a space-optimized version of the text field. If you don't need to query phrases or use positional queries, map the field as `match_only_text` instead of `text`. Positional queries are queries for which the position of the term in the phrase matters, such as interval or span queries. 


Should "text" be in code font?

_field-types/supported-field-types/index.md

_field-types/supported-field-types/match-only-text.md

_field-types/supported-field-types/string.md

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

rishabhmaurya marked this pull request as ready for review January 5, 2024 00:21

rishabhmaurya requested review from hdhalter, kolchfa-aws, Naarcha-AWS, vagimeli, AMoo-Miki and natebower as code owners January 5, 2024 00:21

rishabhmaurya self-assigned this Jan 5, 2024

kolchfa-aws self-assigned this Jan 5, 2024

rishabhmaurya force-pushed the match-only-text-doc branch from 9e5eb02 to 5a350a5 Compare January 5, 2024 18:38

hdhalter added v2.12.0 release-notes PR: Include this PR in the automated release notes 3 - Tech review PR: Tech review in progress labels Jan 8, 2024

rishabhmaurya force-pushed the match-only-text-doc branch from 5a350a5 to c2cabaa Compare January 8, 2024 19:11

msfroh reviewed Jan 8, 2024

View reviewed changes

andrross reviewed Jan 8, 2024

View reviewed changes

rishabhmaurya added 2 commits January 22, 2024 20:13

Documentation for match_only_text field

7a40253

Signed-off-by: Rishabh Maurya <[email protected]>

Address pr comments

526ef40

Signed-off-by: Rishabh Maurya <[email protected]>

rishabhmaurya force-pushed the match-only-text-doc branch from c2cabaa to 526ef40 Compare January 23, 2024 04:13

rishabhmaurya requested a review from dlvenable as a code owner January 23, 2024 04:13

Doc review

a1b3ecf

Signed-off-by: Fanit Kolchina <[email protected]>

natebower reviewed Jan 30, 2024

View reviewed changes

kolchfa-aws reviewed Jan 30, 2024

View reviewed changes

_field-types/supported-field-types/index.md Outdated Show resolved Hide resolved

kolchfa-aws reviewed Jan 30, 2024

View reviewed changes

_field-types/supported-field-types/match-only-text.md Outdated Show resolved Hide resolved

kolchfa-aws reviewed Jan 30, 2024

View reviewed changes

_field-types/supported-field-types/match-only-text.md Outdated Show resolved Hide resolved

kolchfa-aws reviewed Jan 30, 2024

View reviewed changes

_field-types/supported-field-types/string.md Outdated Show resolved Hide resolved

kolchfa-aws and others added 2 commits January 30, 2024 13:56

Apply suggestions from code review

b36324a

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

Apply suggestions from code review

9f7e3bd

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws added 6 - Done but waiting to merge PR: The work is done and ready to merge and removed 3 - Tech review PR: Tech review in progress labels Jan 30, 2024

kolchfa-aws approved these changes Feb 1, 2024

View reviewed changes

hdhalter merged commit de8eb4e into opensearch-project:main Feb 1, 2024
4 checks passed

hdhalter added 3 - Done Issue is done/complete and removed 6 - Done but waiting to merge PR: The work is done and ready to merge labels Feb 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation for match_only_text field #6041

Documentation for match_only_text field #6041

rishabhmaurya commented Jan 5, 2024 •

edited

Loading

rishabhmaurya commented Jan 5, 2024

msfroh left a comment

msfroh Jan 8, 2024

rishabhmaurya Jan 9, 2024

rishabhmaurya Jan 23, 2024

msfroh Jan 8, 2024

rishabhmaurya Jan 9, 2024

andrross Jan 8, 2024

rishabhmaurya Jan 9, 2024

rishabhmaurya Jan 23, 2024

kolchfa-aws commented Jan 12, 2024

rishabhmaurya commented Jan 23, 2024

kolchfa-aws commented Jan 26, 2024

rishabhmaurya commented Jan 30, 2024

natebower left a comment

natebower Jan 30, 2024

natebower Jan 30, 2024

natebower Jan 30, 2024

natebower Jan 30, 2024


		## Parameters

		While `match_only_text` supports most parameters available for text fields, modifying most of them can actually be counterproductive. This field type thrives on simplicity and efficiency, minimizing data stored in the index to optimize storage costs. Therefore, sticking with the default settings is generally the best approach. Any adjustments beyond analyzer settings could reintroduce overhead and negate the efficiency benefits of `match_only_text`.


		## Migrating from text field

		Reindex API can be used to migrate from text field to `match_only_text` by updating the right mapping in the target index.

		- Supports most query types, but not interval/span queries.


		Choosing `match_only_text` means prioritizing efficient full-text search over complex ranking and positional queries, while also optimizing storage costs. It excels when you need to quickly find documents containing specific terms without the overhead of storing frequencies and positions, leading to significantly smaller indexes. This translates to lower storage costs, especially for large datasets. However, it's not the best choice for ranking results based on relevance or for queries that hinge on term proximity or order, like interval/span queries. While it does support phrase queries, their performance isn't as efficient as with text field type. So, if pinpointing exact phrases or their locations within documents is essential, consider using text field type.


		A `match_only_text` field is different from a `text` field in the following ways:

		- Omits storing positions, frequencies, and norms, reducing storage.

Documentation for match_only_text field #6041

Documentation for match_only_text field #6041

Conversation

rishabhmaurya commented Jan 5, 2024 • edited Loading

Description

Issues Resolved

Checklist

rishabhmaurya commented Jan 5, 2024

msfroh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kolchfa-aws commented Jan 12, 2024

rishabhmaurya commented Jan 23, 2024

kolchfa-aws commented Jan 26, 2024

rishabhmaurya commented Jan 30, 2024

natebower left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rishabhmaurya commented Jan 5, 2024 •

edited

Loading