From 5c45d679b9c659da3d220ecc9de197fa684ae3ab Mon Sep 17 00:00:00 2001 From: karifabri <137208546+karifabri@users.noreply.github.com> Date: Tue, 28 May 2024 09:52:20 -0400 Subject: [PATCH] Added blog post for v0.13.1-alpha --- content/_index.html | 2 +- content/posts/releases/v0.13.1-alpha.md | 104 ++++++++++++++++++++++++ 2 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 content/posts/releases/v0.13.1-alpha.md diff --git a/content/_index.html b/content/_index.html index 4a821e92..e4444cec 100644 --- a/content/_index.html +++ b/content/_index.html @@ -8,7 +8,7 @@ diff --git a/content/posts/releases/v0.13.1-alpha.md b/content/posts/releases/v0.13.1-alpha.md new file mode 100644 index 00000000..1a052750 --- /dev/null +++ b/content/posts/releases/v0.13.1-alpha.md @@ -0,0 +1,104 @@ +--- +date: 2024-05-27 +title: 'Spice v0.13.1-alpha (May 27, 2024)' +type: blog +author: Spice AI ([@spice_ai](https://twitter.com/spice_ai)) +categories: [release] +tags: [results caching, stability, operability, dataset schema, query history] +--- + +The v0.13.1-alpha release of Spice is a minor update focused on stability, quality, and operability. Query result caching provides protection against bursts of queries and schema support for datasets has been added logical grouping. An issue where Refresh SQL predicates were not pushed down underlying data sources has been resolved along with improved Acceleration Refresh logging. + +## Highlights in v0.13.1-alpha + +- **Results Caching:** Introduced query results caching to handle bursts of requests and support caching of non-accelerated results, such as refresh data returned [on zero results](https://docs.spiceai.org/data-accelerators/data-refresh#behavior-on-zero-results). Results caching is enabled by default with a `1s` item time-to-live (TTL). [Learn more](https://docs.spiceai.org/features/caching). + +- **Query History Logging:** Recent queries are now logged in the new `spice.runtime.query_history` dataset with a default retention of 24-hours. Query history is initially enabled for HTTP queries only (not Arrow Flight queries). + +- **Dataset Schemas:** Added support for dataset schemas, allowing logical grouping of datasets by separating the schema name from the table name with a `.`. E.g. + + ```yaml + datasets: + - from: mysql:app1.identities + name: app.users + + - from: postgres:app2.purchases + name: app.purchases + ``` + + In this example, queries against `app.users` will be federated to `my_schema.my_table`, and `app.purchases` will be federated to `app2.purchases`. + +## Contributors + +@y-f-u +@Jeadie +@sgrebnov +@ewgenius +@phillipleblanc +@lukekim +@gloomweaver +@Sevenannn + +## New in this release + +- Add more type support on mysql connector by @y-f-u in https://github.com/spiceai/spiceai/pull/1449 +- Add in-memory caching support for Arrow Flight queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1450 +- Fix the table reference to use the full table reference, not just the table by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1460 +- Make `file_format` parameter required for S3/FTP/SFTP connector by @ewgenius in https://github.com/spiceai/spiceai/pull/1455 +- Add more verbose logging when acceleration refresh update is finished by @y-f-u in https://github.com/spiceai/spiceai/pull/1453 +- Fix snowflake dataset path when using federation query by @y-f-u in https://github.com/spiceai/spiceai/pull/1474 +- Update cargo to use spiceai datafusion fork by @y-f-u in https://github.com/spiceai/spiceai/pull/1475 +- Enable in-memory results caching by default by @sgrebnov in https://github.com/spiceai/spiceai/pull/1473 +- Add basic integration test for MySQL federation by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1477 +- Update results_cache config names per final spec by @sgrebnov in https://github.com/spiceai/spiceai/pull/1487 +- Add DuckDB quickstart to E2E tests by @lukekim in https://github.com/spiceai/spiceai/pull/1461 +- Add X-Cache header for http queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1472 +- Add telemetry for in-memory caching by @sgrebnov in https://github.com/spiceai/spiceai/pull/1456 +- Pin Git dependencies to a specific commit hash by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1490 +- Detect `file_format` from dataset path by @ewgenius in https://github.com/spiceai/spiceai/pull/1489 +- Add `file_format` to helm chart sample dataset by @ewgenius in https://github.com/spiceai/spiceai/pull/1493 +- Improve duckdb data connector error messages by @Sevenannn in https://github.com/spiceai/spiceai/pull/1486 +- Add `file_format` prompt for s3 and ftp datasets in Dataset Configure CLI if no extension detected by @ewgenius in https://github.com/spiceai/spiceai/pull/1494 +- Add llms to the spicepod definition and use throughout by @Jeadie in https://github.com/spiceai/spiceai/pull/1447 +- Fix duckdb acceleration converting null into default values. by @y-f-u in https://github.com/spiceai/spiceai/pull/1500 +- Separate runtime Dataset from spicepod Dataset by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1503 +- Duckdb e2e test OSX support by @y-f-u in https://github.com/spiceai/spiceai/pull/1505 +- Use TableReference for dataset name by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1506 +- Tweak Results Cache naming and output by @lukekim in https://github.com/spiceai/spiceai/pull/1509 +- Fix refresh_sql not properly passing down filters by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1510 +- Allow datasets to specify a schema by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1507 +- Cache invalidation for accelerated tables by @sgrebnov in https://github.com/spiceai/spiceai/pull/1498 +- Improve spark data connector error messages by @Sevenannn in https://github.com/spiceai/spiceai/pull/1497 +- Parse postgres table schema from prepare statement to support empty tables by @ewgenius in https://github.com/spiceai/spiceai/pull/1445 +- Improve clarity of README and add FAQ by @lukekim in https://github.com/spiceai/spiceai/pull/1512 +- Use binary data transfer for ftp by @gloomweaver in https://github.com/spiceai/spiceai/pull/1517 +- Add support for time64 for SQL insertion statement by @y-f-u in https://github.com/spiceai/spiceai/pull/1519 +- Add Spice Extensions PoC by @ewgenius in https://github.com/spiceai/spiceai/pull/1476 +- Add results cache metrics, pod and quantile filters to Grafana dashboard by @sgrebnov in https://github.com/spiceai/spiceai/pull/1513 +- Add unit tests for results caching utils by @sgrebnov in https://github.com/spiceai/spiceai/pull/1514 +- Add E2E tests for results caching by @sgrebnov in https://github.com/spiceai/spiceai/pull/1515 +- Pass table_reference full string into spark_session table so it can query across schemas or catalogs by @y-f-u in https://github.com/spiceai/spiceai/pull/1521 +- Trace on debug level for tables in `runtime` schema by @ewgenius in https://github.com/spiceai/spiceai/pull/1524 +- Update SparkSessionBuilder::remote and update spark fork hash by @Sevenannn in https://github.com/spiceai/spiceai/pull/1495 +- Fix federation push-down for datasets with schemas by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1526 +- Store history of queries in 'spice.runtime.query_history' by @Jeadie in https://github.com/spiceai/spiceai/pull/1501 +- Disable cache for system queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1528 +- Register runtime tables with runtime schema by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1532 +- Fix acknowledgments workflow to include all cargo features by @Jeadie in https://github.com/spiceai/spiceai/pull/1531 + +**Full Changelog**: https://github.com/spiceai/spiceai/compare/v0.13.0-alpha...v0.13.1-alpha + +## Resources + +- [Getting started with Spice.ai](https://docs.spiceai.org/getting-started/) +- [Documentation](https://docs.spiceai.org/) + +## Community + +Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved. + +- Twitter: [@spice_ai](https://twitter.com/spice_ai) +- Discord: [https://discord.gg/kZnTfneP5u](https://discord.gg/kZnTfneP5u) +- Telegram: [Spice AI Discussion](https://t.me/spiceaichat) +- Reddit: [https://www.reddit.com/r/spiceai](https://www.reddit.com/r/spiceai) +- Email: [hey@spice.ai](mailto:hey@spice.ai)