v0.20.0

0.20.0 (2022-04-14)

Highlights

We are delighted to announce the release of Feast 0.20, which introduces many new features and enhancements:

• High performance Python feature serving (through embedding Go and optimized DynamoDB batch gets)
• Many connector improvements and bug fixes (DynamoDB, Snowflake, Spark, Trino)
  • Note: Trino has been officially bundled into Feast. You can now run this with pip install feast[trino]!
• Graduated alpha features (python feature server + push features)
• Feast API changes
• [Experimental] Feast UI as an importable npm module

Detailed changelog:

Bug Fixes

• Add inlined data sources to the top level registry (#2456) (356788a)
• Add new value types to types.ts for web ui (#2463) (ad5694e)
• Add PushSource proto and Python class (#2428) (9a4bd63)
• Add spark to lambda dockerfile (#2480) (514666f)
• Added private_key auth for Snowflake (#2508) (c42c9b0)
• Added Redshift and Spark typecheck to data_source event_timestamp_col inference (#2389) (04dea73)
• Building of go extension fails (#2448) (7d1efd5)
• Bump the number of versions bumps expected to 27 (#2549) (ecc9938)
• Create init files for the proto-generated python dirs (#2410) (e17028d)
• Don't prevent apply from running given duplicate empty names in data sources. Also fix repeated apply of Spark data source. (#2415) (b95f441)
• Dynamodb deduplicate batch write request by partition keys (#2515) (70d4a13)
• Ensure that init files exist in proto dirs (#2433) (9b94f7b)
• Fix DataSource constructor to unbreak custom data sources (#2492) (712653e)
• Fix default feast apply path without any extras (#2373) (6ba7fc7)
• Fix definitions.py with new definition (#2541) (eefc34a)
• Fix entity row to use join key instead of name (#2521) (c22fa2c)
• Fix Java Master (#2499) (e083458)
• Fix registry proto (#2435) (ea6a9b2)
• Fix some inconsistencies in the docs and comments in the code (#2444) (ad008bf)
• Fix spark docs (#2382) (d4a606a)
• Fix Spark template to work correctly on feast init -t spark (#2393) (ae133fd)
• Fix the feature repo fixture used by java tests (#2469) (32e925e)
• Fix unhashable Snowflake and Redshift sources (cd8f1c9)
• Fixed bug in passing config file params to snowflake python connector (#2503) (34f2b59)
• Fixing Spark template to include source name (#2381) (a985f1d)
• Make name a keyword arg for the Entity class (#2467) (43847de)
• Making a name for data sources not a breaking change (#2379) (71d7ae2)
• Minor link fix in CONTRIBUTING.md (#2481) (2917e27)
• Preserve ordering of features in _get_column_names (#2457) (495b435)
• Relax click python requirement to >=7 (#2450) (f202f92)
• Remove date partition column field from datasources that don't s… (#2478) (ce35835)
• Remove docker step from unit test workflow (#2535) (6f22f22)
• Remove spark from the AWS Lambda dockerfile (#2498) (6abae16)
• Request data api update (#2488) (0c9e5b7)
• Schema update (#2509) (cf7bbc2)
• Simplify DataSource.from_proto logic (#2424) (6bda4d2)
• Snowflake api update (#2487) (1181a9e)
• Support passing batch source to streaming sources for backfills (#2523) (90db1d1)
• Timestamp update (#2486) (bf23111)
• Typos in Feast UI error message (#2432) (e14369d)
• Update feature view APIs to prefer keyword args (#2472) (7c19cf7)
• Update file api (#2470) (83a11c6)
• Update Makefile to cd into python dir before running commands (#2437) (ca32155)
• Update redshift api (#2479) (4fa73a9)
• Update some fields optional in UI parser (#2380) (cff7ac3)
• Use a single version of jackson libraries and upgrade to 2.12.6.1 (#2473) (5be1cc6)
• Use dateutil parser to parse materialization times ([#2464](https://github....

v0.19.4

0.19.4 (2022-04-06)

Bug Fixes

• Add spark to lambda dockerfile (#2480) (ba22c28)
• Don't prevent apply from running given duplicate empty names in data sources. Also fix repeated apply of Spark data source. (#2415) (88e01a2)
• Fix DataSource constructor to unbreak custom data sources (#2492) (2115bd0)

v0.19.3

0.19.3 (2022-03-09)

Bug Fixes

• Closes threadpool resources upon datastore online_write_batch completion (#2386) (7a93152)
• Fix spark docs (#2382) (599dbbb)
• Fix Spark template to work correctly on feast init -t spark (#2393) (d877d27)
• Fixing Spark template to include source name (#2381) (e7a3b3f)
• Making a name for data sources not a breaking change (#2379) (993b8cc)
• Use the correct dockerhub image tag when building feature servers (#2372) (a6211cf)

v0.19.2

0.19.2 (2022-03-06)

Bug Fixes

• Fix default feast apply path without any extras (#2373) (73e4dde)

v0.19.1

0.19.1 (2022-03-05)

Bug Fixes

• Fix unhashable Snowflake and Redshift sources (6f8a3b0)

v0.19.0

0.19.0 (2022-03-05)

Bug Fixes

• Added additional value types to UI parser and removed references to registry-bq.json (#2361) (d202d51)
• Fix Redshift bug that stops waiting on statements after 5 minutes (#2363) (74f887f)
• Method _should_use_plan only returns true for local sqlite provider (#2344) (fdb5f21)
• Remove redis service to prevent more conflicts and add redis node to master_only (#2354) (993616f)
• Rollback Redis-py to Redis-py-cluster (#2347) (1ba86fb)
• Update github workflow to prevent redis from overlapping ports. (#2350) (c2a6c6c)

Features

• Add owner field to Entity and rename labels to tags (412d625)
• Allow all snowflake python connector connection methods to be available to Feast (#2356) (ec7385c)
• Allowing password based authentication and SSL for Redis in Java feature server (0af8adb)
• Event timestamps response (#2355) (5481caf)
• Feast Spark Offline Store (#2349) (98b8d8d)
• Initial merge of Web UI logic (#2352) (ce3bc59)
• Key ttl setting for redis online store (#2341) (236a108)
• Metadata changes & making data sources top level objects to power Feast UI (#2336) (43da230)

Feast v0.18.1

Full Changelog: v0.18.0...v0.18.1

Fixed bugs:

• ODFVs raise a PerformanceWarning for very large sets of features #2293
• Don't require snowflake to always be installed #2309 (judahrand)
• podAnnotations Values in the feature-server chart #2304 (tpvasconcelos)
• Fixing the Java helm charts and adding a demo tutorial on how to use them #2298 (adchia)
• avoid using transactions on OSS Redis #2296 (DvirDukhan)
• Include infra objects in registry dump and fix Infra's from_proto #2295 (adchia)
• Expose snowflake credentials for unit testing #2288 (sfc-gh-madkins)
• Fix flaky tests (test_online_store_cleanup & test_feature_get_online_features_types_match) #2276 (pyalex)

Merged pull requests:

• Remove old flag warning with the python feature server #2300 (adchia)
• Use an OFFLINE schema for Snowflake offline store tests #2291 (sfc-gh-madkins)
• fix typos in markdown files #2289 (charliec443)
• Add -SNAPSHOT suffix to pom.xml version #2286 (tsotnet)
• Update CONTRIBUTING.md #2282 (adchia)

Feast v0.18.0

Overview

Today, we released Feast 0.18, with some major developments:

• Snowflake offline store support has been merged into the main repo
• Introduced saved datasets, which allows persisting data frames retrieved from offline stores
• The first milestone of Data Quality Monitoring project has been implemented. This enables defining expectation suites (using Great Expectations) and running them against training datasets
• Python feature server graduated from the alpha status
• A significant performance improvements have been achieved in both Python & Java feature servers

✨ New Features:

• Tutorial on validation of historical features #2277 (pyalex)
• Feast plan clean up #2256 (felixwang9817)
• Return UNIX_TIMESTAMP as Python datetime #2244 (judahrand)
• Validating historical features against reference dataset with "great expectations" profiler #2243 (pyalex)
• Implement feature_store._apply_diffs to handle registry and infra diffs #2238 (felixwang9817)
• Compare Python objects instead of proto objects #2227 (felixwang9817)
• Modify feature_store.plan to produce an InfraDiff #2211 (felixwang9817)
• Implement diff_infra_protos method for feast plan #2204 (felixwang9817)
• Persisting results of historical retrieval #2197 (pyalex)
• Merge feast-snowflake plugin into main repo with documentation #2193 (sfc-gh-madkins)
• Add InfraDiff class for feast plan #2190 (felixwang9817)
• Use FeatureViewProjection instead of FeatureView in ODFV #2186 (judahrand)

🔴 Fixed bugs:

• Set created_timestamp and last_updated_timestamp fields #2266 (judahrand)
• Use datetime.utcnow() to avoid timezone issues #2265 (judahrand)
• Fix Redis key serialization in java feature server #2264 (pyalex)
• modify registry.db s3 object initialization to work in S3 subdirectory with Java Feast Server #2259 (NalinGHub)
• Add snowflake environment variables to allow testing on snowflake infra #2258 (sfc-gh-madkins)
• Correct inconsistent dependency #2255 (judahrand)
• Fix for historical field mappings #2252 (michelle-rascati-sp)
• Add backticks to left_table_query_string #2250 (dmille)
• Fix inference of BigQuery ARRAY types. #2245 (judahrand)
• Fix Redshift data creator #2242 (felixwang9817)
• Delete entity key from Redis only when all attached feature views are gone #2240 (pyalex)
• Tests for transformation service integration in java feature server #2236 (pyalex)
• Feature server helm chart produces invalid YAML #2234 (pyalex)
• Docker build fails for java feature server #2230 (pyalex)
• Fix ValueType.UNIX_TIMESTAMP conversions #2219 (judahrand)
• Add on demand feature views deletion #2203 (corentinmarek)
• Compare only specs in integration tests #2200 (felixwang9817)
• Bump log4j-core from 2.17.0 to 2.17.1 in /java #2189 (dependabot[bot])
• Support multiple application properties files (incl from classpath) #2187 (pyalex)
• Avoid requesting features from OnlineStore twice #2185 (judahrand)
• Speed up Datastore deletes by batch deletions with multithreading #2182 (ptoman-pa)
• Fixes large payload runtime exception in Datastore (issue 1633) #2181 (ptoman-pa)

🔨 Merged pull requests:

• Add link to community plugin for Spark offline store #2279 (adchia)
• Fix broken links on documentation #2278 (adchia)
• Publish alternative python package with FEAST_USAGE=False by default #2275 (pyalex)
• Unify all helm charts versions #2274 (pyalex)
• Fix / update helm chart workflows to push the feast python server #2273 (adchia)
• Update Feast Serving documentation with ways to run and debug locally #2272 (adchia)
• Fix Snowflake docs #2270 (felixwang9817)
• Update local-feature-server.md #2269 (tsotnet)
• Update docs to include Snowflake/DQM and removing unused docs from old versions of Feast #2268 (adchia)
• Graduate Python feature server #2263 (felixwang9817)
• Fix benchmark tests at HEAD by passing in Snowflake secrets #2262 (adchia)
• Refactor pa_to_feast_value_type #2246 (judahrand)
• Allow using pandas.StringDtype to support on-demand features with STRING type #2229 (pyalex)
• Bump jackson-databind from 2.10.1 to 2.10.5.1 in /java/common #2228 (dependabot[bot])
• Split apply total parse repo #2226 (mickey-liu)
• Publish renamed java packages to maven central (via Sonatype) #2225 (pyalex)
• Make online store nullable #2224 (mirayyuce)
• Optimize _populate_result_rows_from_feature_view #2223 (judahrand)
• Update to newer redis-py #2221 (judahrand)
• Adding a local feature server test #2217 (adchia)
• replace GetOnlineFeaturesResponse with GetOnlineFeaturesResponseV2 in… #2214 (tsotnet)
• Updates to click==8.* #2210 (diogommartins)
• Bump protobuf-java from 3.12.2 to 3.16.1 in /java #2208 (dependabot[bot])
• Add default priority for bug reports #2207 (adchia)
• Modify issue templates to automatically attach labels #2205 (adchia)
• Python FeatureServer optimization #2202 (judahrand)
• Refactor all importer logic to belong in feast.importer #2199 (felixwang9817)
• Refactor `Onlin...

Feast v0.17.0

Overview

Today, we released Feast 0.17, which includes:

• an initial cut at feast plan (See RFC-030)
• many optimizations for materialization / feature serving in both python + java feature servers, especially with Redis as an online store.
• a simplified Java server (without Spring Boot boilerplate)
• a helm chart for deploying the python feature server (as an alternative to deploying in AWS Lambda)
• other bug fixes, including type conversion bugs and log4j patches

✨ New Features:

• Add feast-python-server helm chart #2177 (michelle-rascati-sp)
• Add a feast plan command, and have CLI output differentiates between created, deleted and unchanged objects #2147 (achals)
• Refactor tag methods to infer created, deleted, and kept repo objects #2142 (achals)
• Pre compute the timestamp range for feature views #2103 (judahrand)

🔴 Fixed bugs:

• Fix issues with java docker building #2178 (achals)
• unpin boto dependency in setup #2168 (fengyu05)
• Fix issue with numpy datetimes in feast_value_type_to_pandas_type #2167 (achals)
• Fix BYTES and BYTES_LIST type conversion #2158 (judahrand)
• Use correct name when deleting dynamo table #2154 (pyalex)
• Bump log4j-core from 2.15.0 to 2.16.0 in /java #2146 (dependabot[bot])
• Bump log4j-api from 2.15.0 to 2.16.0 in /java #2145 (dependabot[bot])
• Respect full_feature_names for ODFVs #2144 (judahrand)
• Cache dynamodb client and resource in DynamoDB online store implement… #2138 (felixwang9817)
• Bump log4j-api from 2.13.2 to 2.15.0 in /java #2133 (dependabot[bot])
• Fix release workflow to use the new GCP action #2132 (adchia)
• Remove spring-boot from the feast serving application #2127 (achals)
• Fix Makefile to properly create the ECR_VERSION #2123 (adchia)

🔨 Merged pull requests:

• Update roadmap to include Snowflake + Trino. Also fix docs + update FAQ #2175 (adchia)
• Convert python values into proto values in bulk #2172 (pyalex)
• Push docker image after build in GH workflow #2171 (pyalex)
• Improve serialization performance #2165 (judahrand)
• Improve online deserialization latency #2164 (judahrand)
• Add a unit test for the tag_proto_objects method #2163 (achals)
• Bump log4j-core from 2.16.0 to 2.17.0 in /java #2161 (dependabot[bot])
• [Java Feature Server] Use hgetall in redis connector when number of retrieved fields is big enough #2159 (pyalex)
• Do not run benchmarks on pull requests #2155 (felixwang9817)
• Ensure that universal CLI test tears down infrastructure #2151 (felixwang9817)
• Remove underscores from ECR docker versions #2139 (achals)
• Run PR integration tests only on python 3.7 #2137 (achals)
• Bump log4j-core from 2.13.2 to 2.15.0 in /java #2134 (dependabot[bot])
• Updating lambda docker image to feature-server-python-aws #2130 (adchia)
• Fix README to reflect new integration test suites #2124 (adchia)
• Change the feast serve endpoint to be sync rather than async. #2119 (nossrannug)
• Remove argument feature_refs #2115 (judahrand)
• Fix leaking dynamodb tables in integration tests #2104 (pyalex)
• Remove untested and undocumented interfaces #2084 (judahrand)
• Update creating-a-custom-provider.md #2070 (ChaitanyaKN)
• • Add SqliteTable as an InfraObject #2157 (felixwang9817)
• Compute property-level diffs for repo objects #2156 (achals)
• Add DatastoreTable infra object #2140 (felixwang9817)
• Dynamodb infra object #2131 (felixwang9817)
• Add Infra and InfraObjects classes #2125 (felixwang9817)

Feast v0.16.1

Changelog

v0.16.1 (2021-12-10)

Full Changelog

This was a quick patch fix to patch in the log4j vulnerability fixes.

Fixed bugs:

• Bump log4j-api from 2.13.2 to 2.15.0 in /java #2133 (dependabot[bot])
• Fix release workflow to use the new GCP action #2132 (adchia)
• Fix Makefile to properly create the ECR_VERSION #2123 (adchia)

Merged pull requests:

• Updating lambda docker image to feature-server-python-aws #2130 (adchia)
• Fix README to reflect new integration test suites #2124 (adchia)
• Remove argument feature_refs #2115 (judahrand)