feat: Feast/IKV online store documentation (#4146)

* feat: Feast/IKV online store documentation Signed-off-by: Pushkar Gupta <[email protected]> * functionality matric Signed-off-by: Pushkar Gupta <[email protected]> * more changes Signed-off-by: Pushkar Gupta <[email protected]> * mount dir Signed-off-by: Pushkar Gupta <[email protected]> --------- Signed-off-by: Pushkar Gupta <[email protected]>
feast-dev · Apr 25, 2024 · 73601e4 · 73601e4
1 parent 3fdb716
commit 73601e4
Show file tree

Hide file tree

Showing 6 changed files with 99 additions and 19 deletions.
diff --git a/README.md b/README.md
@@ -175,6 +175,7 @@ The list below contains the functionality that contributors are planning to deve
   * [x] [Bigtable](https://docs.feast.dev/reference/online-stores/bigtable)
   * [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
   * [x] [Dragonfly](https://docs.feast.dev/reference/online-stores/dragonfly)
+  * [x] [IKV - Inlined Key Value Store](https://docs.feast.dev/reference/online-stores/ikv)
   * [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
   * [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/online-stores/postgres)
   * [x] [Cassandra / AstraDB (contrib plugin)](https://docs.feast.dev/reference/online-stores/cassandra)

diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
@@ -91,6 +91,7 @@
   * [Snowflake](reference/online-stores/snowflake.md)
   * [Redis](reference/online-stores/redis.md)
   * [Dragonfly](reference/online-stores/dragonfly.md)
+  * [IKV](reference/online-stores/ikv.md)
   * [Datastore](reference/online-stores/datastore.md)
   * [DynamoDB](reference/online-stores/dynamodb.md)
   * [Bigtable](reference/online-stores/bigtable.md)

diff --git a/docs/reference/online-stores/README.md b/docs/reference/online-stores/README.md
@@ -22,6 +22,10 @@ Please see [Online Store](../../getting-started/architecture-and-components/onli
 [dragonfly.md](dragonfly.md)
 {% endcontent-ref %}
 
+{% content-ref url="ikv.md" %}
+[ikv.md](ikv.md)
+{% endcontent-ref %}
+
 {% content-ref url="datastore.md" %}
 [datastore.md](datastore.md)
 {% endcontent-ref %}

diff --git a/docs/reference/online-stores/ikv.md b/docs/reference/online-stores/ikv.md
@@ -0,0 +1,73 @@
+# IKV (Inlined Key-Value Store) online store
+
+## Description
+
+[IKV](https://github.com/inlinedio/ikv-store) is a fully-managed embedded key-value store, primarily designed for storing ML features. Most key-value stores (think Redis or Cassandra) need a remote database cluster, whereas IKV allows you to utilize your existing application infrastructure to store data (cost efficient) and access it without any network calls (better performance). 
+
+For provisioning API keys for using it as an online-store in Feast, go to [https://inlined.io](https://inlined.io) or email onboarding[at]inlined.io
+
+## Getting started
+Make sure you have Python and `pip` installed.
+
+Install the Feast SDK and CLI: `pip install feast`
+
+In order to use this online store, you'll need to install the IKV extra (along with the dependency needed for the offline store of choice). E.g.
+-  `pip install 'feast[gcp, ikv]'`
+-  `pip install 'feast[snowflake, ikv]'`
+-  `pip install 'feast[aws, ikv]'`
+-  `pip install 'feast[azure, ikv]'`
+
+You can get started by using any of the other templates (e.g. `feast init -t gcp` or `feast init -t snowflake` or `feast init -t aws`), and then swapping in IKV as the online store as seen below in the examples.
+
+### 1. Provision an IKV store
+Go to [https://inlined.io](https://inlined.io) or email onboarding[at]inlined.io
+
+### 2. Configure
+
+Update `my_feature_repo/feature_store.yaml` with the below contents:
+
+{% code title="feature_store.yaml" %}
+```yaml
+project: my_feature_repo
+registry: data/registry.db
+provider: local
+online_store:
+    type: ikv
+    account_id: secret
+    account_passkey: secret
+    store_name: your-store-name
+    mount_directory: /absolute/path/on/disk/for/ikv/embedded/index
+```
+{% endcode %}
+
+After provision an IKV account/store, you should the required id, passkey and store-name.
+
+Additionally you must specify a mount-directory - where IKV will pull/update (maintain) a copy of the index for online reads (IKV is an embedded database). It can be skipped only if you don't plan to read any data from this container. The mount directory path usually points to a location on local/remote disk.
+
+The full set of configuration options is available in IKVOnlineStoreConfig at `sdk/python/feast/infra/online_stores/contrib/ikv_online_store/ikv.py`
+
+## Functionality Matrix
+
+The set of functionality supported by online stores is described in detail [here](overview.md#functionality).
+Below is a matrix indicating which functionality is supported by the IKV online store.
+
+|                                                           | IKV   |
+| :-------------------------------------------------------- | :---- |
+| write feature values to the online store                  | yes   |
+| read feature values from the online store                 | yes   |
+| update infrastructure (e.g. tables) in the online store   | yes   |
+| teardown infrastructure (e.g. tables) in the online store | yes   |
+| generate a plan of infrastructure changes                 | no    |
+| support for on-demand transforms                          | yes   |
+| readable by Python SDK                                    | yes   |
+| readable by Java                                          | no    |
+| readable by Go                                            | no    |
+| support for entityless feature views                      | yes   |
+| support for concurrent writing to the same key            | yes   |
+| support for ttl (time to live) at retrieval               | no    |
+| support for deleting expired data                         | no    |
+| collocated by feature view                                | no    |
+| collocated by feature service                             | no    |
+| collocated by entity key                                  | yes   |
+
+To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).
diff --git a/docs/reference/online-stores/overview.md b/docs/reference/online-stores/overview.md
@@ -29,26 +29,26 @@ See this [issue](https://github.com/feast-dev/feast/issues/2254) for a discussio
 ## Functionality Matrix
 
 There are currently five core online store implementations: `SqliteOnlineStore`, `RedisOnlineStore`, `DynamoDBOnlineStore`, `SnowflakeOnlineStore`, and `DatastoreOnlineStore`.
-There are several additional implementations contributed by the Feast community  (`PostgreSQLOnlineStore`, `HbaseOnlineStore`, and `CassandraOnlineStore`), which are not guaranteed to be stable or to match the functionality of the core implementations.
+There are several additional implementations contributed by the Feast community  (`PostgreSQLOnlineStore`, `HbaseOnlineStore`, `CassandraOnlineStore` and `IKVOnlineStore`), which are not guaranteed to be stable or to match the functionality of the core implementations.
 Details for each specific online store, such as how to configure it in a `feature_store.yaml`, can be found [here](README.md).
 
 Below is a matrix indicating which online stores support what functionality.
 
-| | Sqlite | Redis | DynamoDB | Snowflake | Datastore | Postgres | Hbase | [[Cassandra](https://cassandra.apache.org/_/index.html) / [Astra DB](https://www.datastax.com/products/datastax-astra?utm_source=feast)] |
-| :-------------------------------------------------------- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- |
-| write feature values to the online store                  | yes | yes | yes | yes | yes | yes | yes | yes |
-| read feature values from the online store                 | yes | yes | yes | yes | yes | yes | yes | yes |
-| update infrastructure (e.g. tables) in the online store   | yes | yes | yes | yes | yes | yes | yes | yes |
-| teardown infrastructure (e.g. tables) in the online store | yes | yes | yes | yes | yes | yes | yes | yes |
-| generate a plan of infrastructure changes                 | yes | no  | no  | no  | no  | no  | no  | yes |
-| support for on-demand transforms                          | yes | yes | yes | yes | yes | yes | yes | yes |
-| readable by Python SDK                                    | yes | yes | yes | yes | yes | yes | yes | yes |
-| readable by Java                                          | no  | yes | no  | no  | no  | no  | no  | no  |
-| readable by Go                                            | yes | yes | no  | no  | no  | no  | no  | no  |
-| support for entityless feature views                      | yes | yes | yes | yes | yes | yes | yes | yes |
-| support for concurrent writing to the same key            | no  | yes | no  | no  | no  | no  | no  | no  |
-| support for ttl (time to live) at retrieval               | no  | yes | no  | no  | no  | no  | no  | no  |
-| support for deleting expired data                         | no  | yes | no  | no  | no  | no  | no  | no  |
-| collocated by feature view                                | yes | no  | yes | yes | yes | yes | yes | yes |
-| collocated by feature service                             | no  | no  | no  | no  | no  | no  | no  | no  |
-| collocated by entity key                                  | no  | yes | no  | no  | no  | no  | no  | no  |
+| | Sqlite | Redis | DynamoDB | Snowflake | Datastore | Postgres | Hbase | [[Cassandra](https://cassandra.apache.org/_/index.html) / [Astra DB](https://www.datastax.com/products/datastax-astra?utm_source=feast)] | [IKV](https://inlined.io) |
+| :-------------------------------------------------------- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- |
+| write feature values to the online store                  | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| read feature values from the online store                 | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| update infrastructure (e.g. tables) in the online store   | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| teardown infrastructure (e.g. tables) in the online store | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| generate a plan of infrastructure changes                 | yes | no  | no  | no  | no  | no  | no  | yes | no  |
+| support for on-demand transforms                          | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| readable by Python SDK                                    | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| readable by Java                                          | no  | yes | no  | no  | no  | no  | no  | no  | no  |
+| readable by Go                                            | yes | yes | no  | no  | no  | no  | no  | no  | no  |
+| support for entityless feature views                      | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| support for concurrent writing to the same key            | no  | yes | no  | no  | no  | no  | no  | no  | yes |
+| support for ttl (time to live) at retrieval               | no  | yes | no  | no  | no  | no  | no  | no  | no  |
+| support for deleting expired data                         | no  | yes | no  | no  | no  | no  | no  | no  | no  |
+| collocated by feature view                                | yes | no  | yes | yes | yes | yes | yes | yes | no  |
+| collocated by feature service                             | no  | no  | no  | no  | no  | no  | no  | no  | no  |
+| collocated by entity key                                  | no  | yes | no  | no  | no  | no  | no  | no  | yes |
diff --git a/docs/roadmap.md b/docs/roadmap.md
@@ -33,6 +33,7 @@ The list below contains the functionality that contributors are planning to deve
   * [x] [Bigtable](https://docs.feast.dev/reference/online-stores/bigtable)
   * [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
   * [x] [Dragonfly](https://docs.feast.dev/reference/online-stores/dragonfly)
+  * [x] [IKV - Inlined Key Value Store](https://docs.feast.dev/reference/online-stores/ikv)
   * [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
   * [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/online-stores/postgres)
   * [x] [Cassandra / AstraDB (contrib plugin)](https://docs.feast.dev/reference/online-stores/cassandra)