Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FMWK-202 Add documentation page for Indexed annotation #612

Merged
merged 8 commits into from
Aug 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ writing a repository style data access layer.

== Documentation

* The https://aerospike.github.io/spring-data-aerospike[User Guide]
* The https://aerospike.github.io/spring-data-aerospike[Documentation Reference]
* Java code documentation on https://www.javadoc.io/doc/com.aerospike/spring-data-aerospike[javadoc.io]
* https://docs.aerospike.com/[Aerospike documentation]

Expand Down
1 change: 1 addition & 0 deletions src/main/asciidoc/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ include::spring-data-commons-docs/object-mapping.adoc[]
include::reference/aerospike-object-mapping.adoc[]
include::reference/template.adoc[]
include::reference/secondary-indexes.adoc[]
include::reference/indexed-annotation.adoc[]
include::reference/caching.adoc[]
include::spring-data-commons-docs/dependencies.adoc[]
include::spring-data-commons-docs/auditing.adoc[]
Expand Down
150 changes: 150 additions & 0 deletions src/main/asciidoc/reference/indexed-annotation.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
[[indexed-annotation]]
= Indexed Annotation

The `@Indexed` annotation allows to create secondary index on a specific field of a Java object.
For the details on secondary indexes in Aerospike see <<secondary-indexes, Secondary Indexes>>.

The annotation allows to specify the following parameters:

[width="100%",cols="<14%,<24%,<18%,<26%",options="header",]
|===
|parameter |index type |mandatory |example
|name |index name |yes |"friend_address_keys_idx"
|type |index type |yes |IndexType.STRING
|bin |indexed bin type |no |"friend"
|collectionType |index type |no |IndexCollectionType.MAPKEYS
|ctx |context (path to the indexed elements) |no |"address"
|===

Here is an example of creating a complex secondary index for fields of a person's friend address.

[source,java]
----
@Data
@AllArgsConstructor
public class Address {

private String street;
private Integer apartment;
private String zipCode;
private String city;
}

@Data
@NoArgsConstructor
@Setter
public class Friend {

String name;
Address address;
}

@Data
@Document
@AllArgsConstructor
@NoArgsConstructor
@Setter
public class Person {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "friend_address_keys_idx",
collectionType = IndexCollectionType.MAPKEYS, ctx = "address")
Friend friend;
}

@Test
void test() {
Friend carter = new Friend();
carter.setAddress(new Address("Street", 14, "1234567890", "City"));

Person dave = new Person();
dave.setFriend(carter);
repository.save(dave);
}
----

A `Person` object in this example has a field called "friend" (`Friend` object).
A `Friend` object has a field called "address" (`Address` object).
So when "friend" field is set to a `Friend` with existing `Address`, we have a person (`dave` in the example above) with a friend (`carter`) who has
a particular address.

`Address` object on its own has certain fields: `street`, `apartment`, `zipCode`, `city`.

NOTE: In Aerospike DB a POJO (such as `Address`) is represented by a Map, so the fields of POJO become map keys.

Thus, if we want to index by `Address` object fields, we set `collectionType` to `IndexCollectionType.MAPKEYS`.

`Ctx` parameter represents context, or path to the necessary element in the specified bin ("friend") - which is "address", because we want to index by fields of friend's address.

== Secondary Index Context DSL

Secondary index context (`ctx` parameter in `@Indexed` annotation) represents path to a necessary element in hierarchy. It uses infix notation.

The document path is described as dot-separated context elements (e.g., "a.b.[2].c") written as a string. A path is made of singular path elements and ends with one (a leaf element) or more elements (leaves) - for example, "a.b.[2].c.[0:3]".

[width="100%",cols="<14%,<24%,<18%",options="header",]
|===
|Path Element |Matches |Notes
|`"a"` |Map key “a” |Single element by key
|`"1"` or `'1'` |Map key (numeric string) “1” |
|`1` |Map key (integer) 1 |
|`\{1\}` |Map index 1 |
|`{=1}` |Map value (integer) 1 |
|`{=bb}` |Map value “bb” |Also {="bb"}
|`{="1"}` or `{='1'}` |Map value (string) “1” |
|`{#1}` |Map rank 1 |
|`[1]` |List index 1 |
|`[=1]` |List value 1 |
|`[#1]` |List rank 1 |
|===

=== Example

Let's consider a Map bin example:

[source,text]
----
{
1: a,
2: b,
4: d,
"5": e,
a: {
55: ee,
"66": ff,
aa: {
aaa: 111,
bbb: 222,
ccc: 333,
},
bb: {
bba: 221,
bbc: 223
},
cc: [ 22, 33, 44, 55, 43, 32, 44 ],
dd: [ {e: 5, f:6}, {z:26, y:25}, {8: h, "9": j} ]
}
}
----

So the following will be true:

[width="100%",cols="<24%,<30%,<18%",options="header",]
|===
|Path |CTX |Matched Value
|a.aa.aaa |[mapKey("a"), mapKey("aa"), mapKey("aaa")] |111
|a.55 |[mapKey("a"), mapKey(55)] |ee
|a."66" |[mapKey("a"), mapKey("66")] |ff
|a.aa.\{2\} |[mapKey("a"), mapKey("aa"),mapIndex(2)] |333
|a.aa.{=222} |[mapKey("a"), mapKey("aa"),mapValue(222)] |222
|a.bb.{#-1} |[mapKey("a"), mapKey("bb"),mapRank(-1)] |223
|a.cc.[0] |[mapKey("a"), mapKey("cc"),listIndex(0)] |22
|a.cc.[#1] |[mapKey("a"), mapKey("cc"),listRank(1)] |32
|a.cc.[=44] |[mapKey("a"), mapKey("cc"),listValue(44)] |[44, 44]
|a.dd.[0].e |[mapKey("a"), mapKey("dd"),listIndex(0), mapKey("e")] |5
|a.dd.[2].8 |[mapKey("a"), mapKey("dd"),listIndex(2), mapKey(8)] |h
|a.dd.[-1]."9" |[mapKey("a"), mapKey("dd"),listIndex(-1), mapKey("9")] |j
|a.dd.[1].{#0} |[mapKey("a"), mapKey("dd"),listIndex(1), mapRank(0)] |y
|===

7 changes: 5 additions & 2 deletions src/main/asciidoc/reference/secondary-indexes.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[[secondary.indexes]]
[[secondary-indexes]]
= Secondary indexes

A secondary index (SI) is a data structure that locates all the records in a namespace, or a set within it, based on a bin value in the record.
Expand Down Expand Up @@ -73,7 +73,7 @@ public class AerospikeIndexConfiguration {
== Creating Secondary Index using @Indexed annotation

You can use `@Indexed` annotation on the field where the index is required.
Here is an example of the `Person` object:
Here is an example of the `Person` object getting indexed by `lastName`:

[source,java]
----
Expand All @@ -91,6 +91,9 @@ public class Person {
}
----

The annotation allows to specify also bin name, collectionType and ctx (context) if needed.
For the details on using `@Indexed` annotation see <<indexed-annotation, Indexed Annotation>>.

== Matching the Secondary Index

NOTE: In Aerospike, secondary indexes are case-sensitive, they match the exact queries.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,67 @@ public class IndexedAnnotationTests extends BaseBlockingIntegrationTests {

@Test
void usingIndexedAnnotationWithCtx() {
class TestFriend {

String name;
Address address;
}

class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "address") // CTX.mapKey(Value.get("address"))
TestFriend friend;
}
indexRefresher.refreshIndexes();

assertThat(
additionalAerospikeTestOperations.getIndexes(template.getSetName(TestPerson.class)).stream()
.filter(index -> index.getName()
.equals("test_person_friend_address_keys_index")
&&
CTX.toBase64(index.getCTX()).equals(CTX.toBase64(new CTX[]{CTX.mapKey(Value.get("address"))}))
)
.count()
).isEqualTo(1L);
assertThat(indexesCache.hasIndexFor(new IndexedField(namespace, template.getSetName(TestPerson.class),
"friend"))).isTrue();

additionalAerospikeTestOperations.dropIndexIfExists(IndexedPerson.class,
"test_person_friend_address_keys_index");
indexRefresher.refreshIndexes();

assertThat(
additionalAerospikeTestOperations.getIndexes(template.getSetName(TestPerson.class)).stream()
.filter(index -> index.getName()
.equals("test_person_friend_address_keys_index")
&&
CTX.toBase64(index.getCTX()).equals(CTX.toBase64(new CTX[]{CTX.mapKey(Value.get("address"))}))
)
.count()
).isZero();
assertThat(indexesCache.hasIndexFor(new IndexedField(namespace, template.getSetName(TestPerson.class),
"friend"))).isFalse();
}

@Test
void usingIndexedAnnotationWithBinNameAndCtx() {
class TestFriend {

String name;
Address address;
}

class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
collectionType = IndexCollectionType.MAPKEYS, ctx = "address") // CTX.mapKey(Value.get("address"))
TestFriend test;
}
indexRefresher.refreshIndexes();

assertThat(
Expand Down Expand Up @@ -63,15 +116,21 @@ class TestPerson {

@Test
void usingIndexedAnnotationWithComplexCtxSingleQuotes() {
class TestFriend {

String name;
Address address;
}

class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "ab.cd.'10'.{#5}.{='1'}.[-1].[#100].[=20]")
// CTX.mapKey(Value.get("ab")), CTX.mapKey(Value.get("cd")), CTX.mapKey(Value.get("10")), CTX.mapRank(5),
// CTX.mapValue(Value.get("1")), CTX.listIndex(-1), CTX.listRank(100), CTX.listValue(Value.get(20))
Address address;
TestFriend friend;
}
indexRefresher.refreshIndexes();

Expand All @@ -97,14 +156,20 @@ class TestPerson {

@Test
void usingIndexedAnnotationWithCtxDoubleQuotes() {
class TestFriend {

String name;
Address address;
}

class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "\"10\".{=\"1\"}")
// CTX.mapKey(Value.get("10")), CTX.mapValue(Value.get("1"))
Address address;
TestFriend friend;
}
indexRefresher.refreshIndexes();

Expand Down Expand Up @@ -132,9 +197,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_many_dots_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_many_dots_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "ab....cd..")
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand All @@ -149,9 +214,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_too_small_context_length_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_too_small_context_length_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "ab.[]")
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand All @@ -166,9 +231,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_wrong_closing_bracket_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_wrong_closing_bracket_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "ab.{cd]")
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand All @@ -183,9 +248,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "{#address}") // rank must be integer
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand All @@ -200,9 +265,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "[#address]") // rank must be integer
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand All @@ -217,9 +282,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "{address}") // index must be integer
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand All @@ -234,9 +299,9 @@ class TestPerson {

@Id
String id;
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index", bin = "friend",
@Indexed(type = IndexType.STRING, name = "test_person_friend_address_keys_index",
collectionType = IndexCollectionType.MAPKEYS, ctx = "[address]") // index must be integer
Address address;
String someField;
}
indexRefresher.refreshIndexes();

Expand Down