Feast v0.9.1

Changelog

v0.9.1 (2021-01-29)

Full Changelog

Implemented enhancements:

• Add telemetry to Python SDK #1289 (jklegar)

Fixed bugs:

• Fix kafka download url #1298 (jklegar)
• disable telemetry in docker-compose test and job_service #1297 (jklegar)

Feast v0.9.0

Full Changelog

Implemented enhancements:

• Enable user to provide spark job template as input for jobservice deployment #1285 (khorshuheng)
• Add feature table name filter to jobs list api #1282 (terryyylim)
• Report observed value for aggregated checks in pre-ingestion feature validation #1278 (pyalex)
• Add docs page for Azure setup #1276 (jklegar)
• Azure example terraform #1274 (jklegar)

Fixed bugs:

• make EMR jar uploader work the same as k8s one #1284 (oavdeev)
• Don't error when azure vars not set #1277 (jklegar)
• Prevent ingestion job config parser from unwanted fieldMapping transformation #1261 (pyalex)
• Features are not being ingested due to max age overflow #1209 (pyalex)
• Feature Table is not being update when only max_age was changed #1208 (pyalex)
• Truncate staging timestamps in entities dataset to ms #1207 (pyalex)
• Bump terraform rds module version #1204 (oavdeev)

Merged pull requests:

• Use date partitioning column in FileSource #1293 (pyalex)
• Add EMR CI/CD entrypoint script #1290 (oavdeev)
• Online serving optimizations #1286 (pyalex)
• Make third party grpc packages recognizable as python module #1283 (khorshuheng)
• Report observed values in feature validation as Gauge #1280 (pyalex)
• Keep same amount of partitions after repartitioning in IngestionJob #1279 (pyalex)
• Add request feature counter metric #1272 (terryyylim)
• Use SEND_INTERRUPT to cancel EMR jobs #1271 (oavdeev)
• Fix historical test flakiness #1270 (jklegar)
• Allow https url for spark ingestion jar #1266 (jklegar)
• Add project name to feature validation metric #1264 (pyalex)
• Use dataproc console url instead of gcs for log uri #1263 (khorshuheng)
• Make nodes priority (for redis cluster) configurable in Serving #1260 (pyalex)
• Enhance job api to return associated feature table and start time #1259 (khorshuheng)
• Reporting metrics from validation UDF #1256 (pyalex)
• Allow use the same timestamp column for both created & even timestamp in Historical Retrieval #1255 (pyalex)
• Apply grpc tracing interceptor on Feast SDK #1243 (khorshuheng)
• Apply grpc tracing interceptor on online serving #1242 (khorshuheng)
• Python UDF in Ingestion being used for feature validation #1234 (pyalex)
• Add spark k8s operator launcher #1225 (oavdeev)
• Add deadletter/read-from-source metrics to batch and stream ingestion #1223 (terryyylim)
• Implement AbstractStagingClient for azure blob storage #1218 (jklegar)
• Configurable materialization destination for view in BigQuerySource #1201 (pyalex)
• Update Feast Core list features method #1176 (terryyylim)
• S3 endpoint configuration #1169 #1172 (mike0sv)
• Increase kafka consumer waiting time in e2e tests #1268 (pyalex)
• E2E tests support for jobservice's control loop #1267 (pyalex)
• Optimize memory footprint for Spark Ingestion Job #1265 (pyalex)
• Fix historical test for azure #1262 (jklegar)
• Change azure https to wasbs and add azure creds to spark #1258 (jklegar)
• Docs, fixes and scripts to run e2e tests in minikube #1254 (oavdeev)
• Fix azure blob storage access in e2e tests #1253 (jklegar)
• Update python version requirements to 3.7 for Dataproc launcher #1251 (pyalex)
• Fix build-ingestion-py-dependencies script #1250 (pyalex)
• Add datadog(statsd) client to python package for IngestionJob #1249 (pyalex)
• Add prow job for azure e2e test #1244 (jklegar)
• Azure e2e test #1241 (jklegar)
• Add Feast Serving histogram metrics #1240 (terryyylim)
• CI should work on python 3.6 #1237 (pyalex)
• Integration test for k8s spark operator support #1236 (oavdeev)
• Add prow config for spark k8s operator integration testing #1235 (oavdeev-tt)
• Upgrading spark to 3.0.1 #1227 (pyalex)
• Support TFRecord as one of the output formats for historical feature retrieval #1222 (khorshuheng)
• Remove stage_dataframe from the launcher interface #1220 (oavdeev)
• Refactor staging client uploader and use it in EMR launcher #1219 (oavdeev)
• Remove unused EMR code #1217 (oavdeev)
• Remove job id from ingested row counter metric #1216 (terryyylim)
• Quickstart link fixed #1213 (szczeles)
• Delete v1 concepts #1194 (terryyylim)
• Dont write defaults to config #1188 (mike0sv)
• Refactor tests which utilizes feature sets #1186 (terryyylim)
• Refactor configurable options and add sphinx docs #1174 (terryyylim)
• Remove unnecessary Google Auth dependency #1170 (woop)

Feast v0.8.4

Bug fixes
Spark job jar URL handling for EMR spark launcher

Feast v0.8.3

Fixed bugs

Spark job jar URL handling for EMR spark launcher

Feast v0.8.2

Implemented enhancements:

• Configurable materialization destination for view in BigQuerySource #1201 (pyalex)

Fixed bugs:

• Fix tag order for release workflow #1205 (terryyylim)
• Fix Feature Table not updated on new feature addition #1197 (khorshuheng)

Merged pull requests:

• Suppress kafka logs in Ingestion Job #1206 (pyalex)
• Add project name to metrics labels in Ingestion Job #1202 (pyalex)

Feast v0.8.1

Implemented enhancements:

• Expires Redis Keys based on Feature Table Max Age #1161 (khorshuheng)
• Jobservice control loop (based on #1140) #1156 (oavdeev)

Fixed bugs:

• Lazy metrics initialization (to correct pick up in executor) #1195 (pyalex)
• Add missing third_party folder #1185 (terryyylim)
• Fix missing name variable instantiation #1166 (terryyylim)

Merged pull requests:

• Bump ssh-agent version #1175 (terryyylim)
• Refactor configurable options and add sphinx docs #1174 (terryyylim)
• Stabilize flaky e2e tests #1173 (pyalex)
• Fix connection resets in CI for Maven #1164 (woop)
• Add dataproc executor resource config #1160 (terryyylim)
• Fix github workflow deprecating env variable #1158 (terryyylim)
• Ensure consistency of github workflow #1157 (terryyylim)

Feast v0.8.0

Implemented enhancements:

• Implement JobService API calls & connect it to SDK #1129 (tsotnet)
• Allow user to specify custom secrets to be mounted on Feast Serving and Feast Core pods #1127 (khorshuheng)
• Allow spark expressions in field mapping during Ingestion #1122 (pyalex)
• Update rest endpoints #1121 (terryyylim)
• Add feature table deletion #1114 (terryyylim)
• Add historical retrieval via job service #1107 (oavdeev)
• Implement list job and get job methods for Dataproc launcher #1106 (khorshuheng)
• Allow entities and features to be updated #1105 (terryyylim)
• Add get_by_id and list_jobs interface to the launcher interface and implement it for EMR #1095 (oavdeev)
• Support redis ssl in feast-serving #1092 (oavdeev)
• Add helm charts for feast jobservice #1081 (tsotnet)
• Terraform cleanup: tags, formatting, better defaults #1080 (oavdeev)
• Update docker-compose for Feast 0.8 #1078 (khorshuheng)
• Configure jupyter env on AWS via terraform #1077 (oavdeev)
• Implement EMR job cancelling #1075 (oavdeev)
• Streaming Ingestion Job supports AVRO format as input #1072 (pyalex)
• Accept Pandas dataframe as input for historical feature retrieval #1071 (khorshuheng)
• Add EMR streaming job launcher #1065 (oavdeev)
• EMR launcher #1061 (oavdeev)
• Add AWS managed kafka config to the example terraform #1058 (oavdeev)
• Feast SDK integration for historical feature retrieval using Spark #1054 (khorshuheng)
• Update GetOnlineFeatures method in sdks #1052 (terryyylim)
• "Start Offline-to-online ingestion" method in Python SDK #1051 (pyalex)
• Adding support for custom grpc dial options in Go SDK #1043 (ankurs)
• CLI command to start/stop/list streaming ingestion job on emr #1040 (oavdeev)
• Update serving service to handle new redis encoding #1038 (terryyylim)
• terraform config for aws #1033 (oavdeev)
• Streaming Ingestion Pipeline with Spark #1027 (pyalex)
• Run offline-to-online ingestion job on EMR #1026 (oavdeev)
• Add redis SSL support to the offline-to-online ingestion job #1025 (oavdeev)
• Dataproc and Standalone Cluster Spark Job launcher #1022 (khorshuheng)
• Pyspark job for feature batch retrieval #1021 (khorshuheng)
• Batch Ingestion Job rewritten on Spark #1020 (pyalex)
• Add Feature Tables API to Core & Python SDK #1019 (mrzzy)
• Introduce Entity as higher-level concept #1014 (terryyylim)

Fixed bugs:

• Fix stencil client serialization issue #1147 (pyalex)
• Deadletter path is being incorrectly joined #1144 (pyalex)
• In Historical Retrieval (SDK) use project from client context #1138 (pyalex)
• Pass project from context to get entities #1137 (pyalex)
• JobService is in crashloop after installing helm chart #1135 (pyalex)
• Fix env var names for jupyter terraform config #1085 (oavdeev)
• Fix java class name validation #1084 (oavdeev)
• Multiple tiny AWS related fixes #1083 (oavdeev)

Merged pull requests:

• Make created_timestamp property optional in KafkaSource #1146 (pyalex)
• In Streaming E2E Test filter kafka consumers by group id prefix #1145 (pyalex)
• Limit concurrency on e2e test runs to 1 #1142 (oavdeev)
• Update prow trigger for AWS #1139 (oavdeev)
• e2e test fixes to make them work on AWS #1132 (oavdeev)
• Add feature table name & job id to deadletter destination #1143 (pyalex)
• Drop hardcoded FEAST_CORE_URL env from JobService helm chart #1136 (pyalex)
• Add Prow to AWS codebuild trigger #1133 (oavdeev)
• Optional IngestionJob parameters passed by Spark Launcher #1130 (pyalex)
• Wait for job to be ready before cancelling #1126 (khorshuheng)
• Ability to run e2e tests in non-default project #1125 (pyalex)
• Ensure job is completed when ingesting to BQ #1123 (terryyylim)
• Add end-to-end Prow Job launcher for AWS tests #1118 (woop)
• Add confluent kafka installation to minimal notebook #1116 (woop)
• Scaffolding for integration tests #1113 (khorshuheng)
• Add serving integration test for updated feature type #1112 (terryyylim)
• In Historical Retrieval from BQ join between source & entities is performed inside BQ #1110 (pyalex)
• Make created timestamp column optional #1108 (khorshuheng)
• Add validation when feature type is changed #1102 (terryyylim)
• E2E flow in prow is working #1101 (pyalex)
• Return e2e back to prow runner #1100 (pyalex)
• Add gh workflow for dockerhub #1098 (terryyylim)
• Make demo notebook work on AWS #1097 (oavdeev)
• Update Kubernetes setup for Feast 0.8 #1096 (khorshuheng)
• Refactored end-to-end tests fully orchestrated by pytest #1094 (pyalex)
• D...

Feast v0.7.1

Full Changelog

Fixed bugs:

• Provide stable jobName in RowMetrics labels #1028 (pyalex)

Feast v0.7.0

Full Changelog

Breaking changes:

• Add request response logging via fluentd #961 (terryyylim)
• Run JobCoontroller as separate application #951 (pyalex)
• Output Subject Claim as Identity in Logging interceptor #946 (mrzzy)
• Use JobManager's backend as persistent storage and source of truth #903 (pyalex)
• Fix invalid characters for project, featureset, entity and features creation #976 (terryyylim)

Implemented enhancements:

• Add redis key prefix as an option to Redis cluster #975 (khorshuheng)
• Authentication Support for Java & Go SDKs #971 (mrzzy)
• Add configurable prefix to Consumer Group in IngestionJob's Kafka reader #969 (terryyylim)
• Configurable kafka consumer in IngestionJob #959 (pyalex)
• Restart Ingestion Job on code version update #949 (pyalex)
• Add REST endpoints for Feast UI #878 (SwampertX)
• Upgrade Feast dependencies #876 (pyalex)

Fixed bugs:

• Fix Java & Go SDK TLS support #986 (mrzzy)
• Fix Python SDK setuptools not supporting tags required for Go SDK to be versioned. #983 (mrzzy)
• Fix Python native types multiple entities online retrieval #977 (terryyylim)
• Prevent historical retrieval from failing on dash in project / featureSet name #970 (pyalex)
• Fetch Job's labels from dataflow #968 (pyalex)
• Fetch Job's Created Datetime from Dataflow #966 (pyalex)
• Fix flaky tests #953 (pyalex)
• Prevent field duplications on schema merge in BigQuery sink #945 (pyalex)
• Fix Audit Message Logging Interceptor Race Condition #938 (mrzzy)
• Bypass authentication for metric endpoints on Serving. #936 (mrzzy)
• Fix grpc security variables name and missing exec qualifier in docker.dev #935 (jmelinav)
• Remove extra line that duplicates statistics list #934 (terryyylim)
• Fix empty array when retrieving stats data #930 (terryyylim)
• Allow unauthenticated access when Authorization is disabled and to Health Probe #927 (mrzzy)
• Impute default project if empty before authorization is called #926 (jmelinav)
• Fix Github Actions CI load-test job failing due inability to install Feast Python SDK. #914 (mrzzy)
• Fix Online Serving unable to retrieve feature data after Feature Set update. #908 (mrzzy)
• Fix unit tests not running in feast.core package. #883 (mrzzy)
• Exclude dependencies signatures from IngestionJob package #879 (pyalex)
• Prevent race condition in BQ sink jobId generation #877 (pyalex)
• Add IngestionId & EventTimestamp to FeatureRowBatch to calculate lag metric correctly #874 (pyalex)
• Fix typo for fluentd request response map key #989 (terryyylim)
• Reduce polling interval for docker-compose test and fix flaky e2e test #982 (terryyylim)
• Fix rate-limiting issue on github actions for master branch #974 (terryyylim)
• Fix docker-compose test #973 (terryyylim)
• Fix Helm chart requirements lock and version linting #925 (woop)
• Fix Github Actions failures due to possible rate limiting. #972 (mrzzy)
• Fix docker image building for PR commits #907 (woop)
• Fix Github Actions versioned image push #994(mrzzy)
• Fix Go SDK extra colon in metadata header for Authentication #1001(mrzzy)
• Fix lint version not pulling tags. #999(mrzzy)
• Call fallback only when theres missing keys #1009 (pyalex)

Merged pull requests:

• Add cryptography to python ci-requirements #988 (pyalex)
• Allow maps in environment variables in helm charts #987 (pyalex)
• Speed up Github Actions Docker builds #980 (mrzzy)
• Use setup.py develop instead of pip install -e #967 (pyalex)
• Peg black version #963 (terryyylim)
• Remove FeatureRow compaction in BQ sink #960 (pyalex)
• Get job controller deployment for docker compose back #958 (pyalex)
• Revert job controller deployment for docker compose #957 (woop)
• JobCoordinator use public API to communicate with Core #943 (pyalex)
• Allow Logging Interceptor to be toggled by Message Logging Enabled Flag #940 (mrzzy)
• Clean up Feast CI, docker compose, and notebooks #916 (woop)
• Allow use of Kubernetes for Github Actions #910 (woop)
• Wait for docker images to be ready for e2e dataflow test #909 (woop)
• Add docker image building to GitHub Actions and consolidate workflows #898 (woop)
• Add load test GitHub Action #897 (woop)
• Typo in feature sets example. #894 (ashwinath)
• Add auth integration tests #892 (woop)
• Integration Test for Job Coordinator #886 (pyalex)
• BQ sink produces sample of successful inserts #875 (pyalex)
• Add Branch and RC Awareness to Version Lint & Fix Semver Regex #998 (mrzzy)

Feast v0.6.2

Full Changelog

Implemented enhancements:

• Redis sink flushes only rows that have more recent eventTimestamp #913 (pyalex)
• Dataflow runner options: disk type & streaming engine #906 (pyalex)
• Add Structured Audit Logging #891 (mrzzy)
• Add Authentication and Authorization for feast serving #865 (jmelinav)
• Throw more informative exception when write_triggering_frequency_seconds is missing #917 (pyalex)
• Add caching to authorization #884 (jmelinav)
• Add Auth header #885 (AnujaVane)

Fixed bugs:

• Fix Online Serving unable to retrieve feature data after Feature Set update. #908 (mrzzy)
• Fix Python SDK ingestion for featureset name that exist in multiple projects #868 (terryyylim)
• Backport delay in Redis acknowledgement of spec #915 (woop)
• Allow unauthenticated access when Authorization is disabled and to Health Probe #927 (mrzzy)

Merged pull requests:

• Upgrade Feast dependencies #876 (pyalex)