Add Full-text Search support #407

jiechic · 2020-10-15T03:21:12Z

Add Fts3 & Fts4 support

codecov · 2020-10-15T03:24:47Z

Codecov Report

Merging #407 (6f38765) into develop (cfc9e5b) will increase coverage by 0.22%.
The diff coverage is 89.28%.

@@             Coverage Diff             @@
##           develop     #407      +/-   ##
===========================================
+ Coverage    83.40%   83.63%   +0.22%     
===========================================
  Files           68       69       +1     
  Lines         1687     1741      +54     
===========================================
+ Hits          1407     1456      +49     
- Misses         280      285       +5

Flag	Coverage Δ
floor	`83.63% <89.28%> (+0.22%)`	⬆️
floor_generator	`83.56% <89.28%> (+0.26%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
floor_generator/lib/value_object/entity.dart	`95.65% <85.71%> (+0.53%)`	⬆️
floor_generator/lib/value_object/fts.dart	`88.00% <88.00%> (ø)`
...loor_generator/lib/processor/entity_processor.dart	`82.20% <91.66%> (+1.63%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cfc9e5b...6f38765. Read the comment docs.

jiechic · 2020-10-15T06:26:09Z

resolve issue #397

vitusortner · 2020-10-15T19:34:39Z

Thanks for your contribution 🙏 I'll look into reviewing it ASAP.

vitusortner · 2020-11-22T19:35:25Z

floor_generator/lib/value_object/entity.dart

+    } else {
+      final ftsType = fts.type ?? '';
+      final ftsTokenizer = fts.tokenizer ?? '';
+      return 'CREATE VIRTUAL TABLE IF NOT EXISTS `$name` USING $ftsType(${databaseDefinition.join(', ')}, tokenize=$ftsTokenizer)$withoutRowidClause';


❓ Does FTS work without a rowid?

Suggested change

return 'CREATE VIRTUAL TABLE IF NOT EXISTS `$name` USING $ftsType(${databaseDefinition.join(', ')}, tokenize=$ftsTokenizer)$withoutRowidClause';

return 'CREATE VIRTUAL TABLE IF NOT EXISTS `$name` USING $ftsType(${databaseDefinition.join(', ')}, tokenize=$ftsTokenizer)';

ths，next commit will change

vitusortner · 2020-11-22T19:41:28Z

floor_annotation/lib/src/fts.dart

@@ -0,0 +1,34 @@
+class Fts {


💡 FTS4 has some additional features that FTS3 doesn't support. Such as setting an external content table to be used by the FTS table. It might make sense to create two separate types: FTS3 and FTS4 as it's done in Room (see links below). We don't have to support all of these additional features initially but having two classes might allow us to be more flexible in the future.
Having two FTS classes also requires some changes to Entity because it would either need to understand both the FTS3 and FTS4types or we list FTS out of the Entity annotation as done in Room. What are your thoughts about this?

https://developer.android.com/reference/androidx/room/Fts3
https://developer.android.com/reference/androidx/room/Fts4

at last time ，i didn't known how to get a new annotation on classElement
i change to code like room now

vitusortner · 2020-11-22T19:45:07Z

floor_annotation/lib/src/fts.dart

+abstract class FtsType {
+  ///FTS3 table in the database
+  static const fts3 = 'fts3';
+
+  ///FTS4 table in the database
+  static const fts4 = 'fts4';
+}
+
+abstract class FtsTokenizer {
+  ///The name of the default
+
+  ///The name of the default tokenizer used on FTS tables
+  static const simple = 'simple';
+
+  ///The name of the tokenizer based on the Porter Stemming Algorithm
+  static const porter = 'porter';
+
+  ///The name of a tokenizer implemented by the ICU library.
+  static const icu = 'icu';
+}


💡 We could use enums here instead of strings to benefit from type safety. We've done this with OnConflictStrategy as you can see here https://github.com/vitusortner/floor/blob/c5925af0f8ab319cd6e8436143bc4c7aace9b2d1/floor_annotation/lib/src/on_conflict_strategy.dart and here https://github.com/vitusortner/floor/blob/c5925af0f8ab319cd6e8436143bc4c7aace9b2d1/floor_generator/lib/processor/update_method_processor.dart#L72.

yes,i will fix

After thinking about it, I decided string instead of strings enum,
I don’t know if you will open the bottom-level implementation of database similar to room in the future,like this:
Room.databaseBuilder(...) .openHelperFactory(factory) .build();
in other openHelperFactory may be use custom tokenizer (in android room use wcdb factory can use mmicu tokenizer)

So I decided to use strings to give users better input and provide candidates on ftsTokenizer

example/lib/database.dart

floor_annotation/pubspec.lock

jiechic · 2020-11-23T09:09:18Z

why ci/floor-generator fail false but my local is success

jiechic · 2020-11-24T07:43:16Z

i change commit for fix for support some you sugguest，now is ok？

jiechic · 2020-12-31T01:10:50Z

any update for review？

vitusortner

I was only able to partly review the PR for now. I'll review the rest ASAP.

floor_generator/test/value_object/fts_test.dart

floor_generator/lib/misc/fts_action.dart

fix test case name

jiechic · 2020-12-31T13:19:05Z

alread check and fix，please review again

jiechic · 2021-01-05T09:13:40Z

any update for review again？

vitusortner

Sorry for the late reply and thanks for your contribution. This is looking really good. I'll extend the project's documentation so that your contribution is easily discoverable.

jiechic force-pushed the develop branch 2 times, most recently from b608074 to d541aad Compare October 15, 2020 06:14

jiechic mentioned this pull request Oct 15, 2020

Which version is planned to support fts #397

Closed

vitusortner linked an issue Oct 15, 2020 that may be closed by this pull request

Which version is planned to support fts #397

Closed

vitusortner self-requested a review October 15, 2020 19:34

jiechic force-pushed the develop branch 3 times, most recently from 4aeea98 to b210d2e Compare October 21, 2020 05:43

vitusortner requested changes Nov 22, 2020

View reviewed changes

jiechic force-pushed the develop branch 6 times, most recently from 42a7aa7 to 9640a8f Compare November 24, 2020 07:35

jiechic requested a review from vitusortner November 25, 2020 06:37

vitusortner reviewed Dec 31, 2020

View reviewed changes

floor_generator/test/value_object/fts_test.dart Outdated Show resolved Hide resolved

floor_generator/lib/misc/fts_action.dart Outdated Show resolved Hide resolved

jiechic added 4 commits December 31, 2020 21:07

add fts annotation support

fe3ccc5

add fts generator support

cdb74cc

add fts integration

62dec9c

remove unused class

ff4e0a8

fix test case name

jiechic force-pushed the develop branch from 9640a8f to ff4e0a8 Compare December 31, 2020 13:13

jiechic requested a review from vitusortner December 31, 2020 13:18

Merge branch 'develop' into develop

6f38765

vitusortner approved these changes Jan 11, 2021

View reviewed changes

vitusortner merged commit 362625d into pinchbv:develop Jan 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Full-text Search support #407

Add Full-text Search support #407

jiechic commented Oct 15, 2020

codecov bot commented Oct 15, 2020 •

edited

Loading

jiechic commented Oct 15, 2020

vitusortner commented Oct 15, 2020

vitusortner Nov 22, 2020

jiechic Nov 23, 2020

vitusortner Nov 22, 2020

jiechic Nov 23, 2020

vitusortner Nov 22, 2020

jiechic Nov 23, 2020

jiechic Nov 24, 2020

jiechic commented Nov 23, 2020

jiechic commented Nov 24, 2020 •

edited

Loading

jiechic commented Dec 31, 2020

vitusortner left a comment

jiechic commented Dec 31, 2020

jiechic commented Jan 5, 2021

vitusortner left a comment

	return 'CREATE VIRTUAL TABLE IF NOT EXISTS `$name` USING $ftsType(${databaseDefinition.join(', ')}, tokenize=$ftsTokenizer)$withoutRowidClause';
	return 'CREATE VIRTUAL TABLE IF NOT EXISTS `$name` USING $ftsType(${databaseDefinition.join(', ')}, tokenize=$ftsTokenizer)';

Add Full-text Search support #407

Add Full-text Search support #407

Conversation

jiechic commented Oct 15, 2020

codecov bot commented Oct 15, 2020 • edited Loading

Codecov Report

jiechic commented Oct 15, 2020

vitusortner commented Oct 15, 2020

vitusortner Nov 22, 2020

Choose a reason for hiding this comment

jiechic Nov 23, 2020

Choose a reason for hiding this comment

vitusortner Nov 22, 2020

Choose a reason for hiding this comment

jiechic Nov 23, 2020

Choose a reason for hiding this comment

vitusortner Nov 22, 2020

Choose a reason for hiding this comment

jiechic Nov 23, 2020

Choose a reason for hiding this comment

jiechic Nov 24, 2020

Choose a reason for hiding this comment

jiechic commented Nov 23, 2020

jiechic commented Nov 24, 2020 • edited Loading

jiechic commented Dec 31, 2020

vitusortner left a comment

Choose a reason for hiding this comment

jiechic commented Dec 31, 2020

jiechic commented Jan 5, 2021

vitusortner left a comment

Choose a reason for hiding this comment

codecov bot commented Oct 15, 2020 •

edited

Loading

jiechic commented Nov 24, 2020 •

edited

Loading