Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.3 backport: Add feature and feature set labels #737

Merged

Conversation

ches
Copy link
Member

@ches ches commented May 23, 2020

What this PR does / why we need it:

Backports #536 to v0.3, courtesy of @suwik's work.

We don't typically backport features, but I'm proposing such in this case for practical reasons:

  • It's a small, useful feature addition and its scope is confined mostly to registry metadata.
  • We already use this backport internally with v0.3.
  • Sticky matter of logistics on our (Agoda's) side: since we use this, it means we internally publish a v0.3-based version of datatypes-java with the labels fields added to the protos. Our Feast SDK is hosted on github.com pending being open sourced, and its GitHub Actions CI build cannot reach our behind-the-firewall Artifactory to resolve the private dependency version. We will live on v0.3 a little while longer, so I'd like to not have to relegate "v0.3 with labels" to an SDK branch where CI isn't usable. We could alternatively publish the datatypes to Maven Central under our own group ID, but it's rather roundabout, so I wanted to try reception in mainline Feast before turning to that.

This is not just housekeeping, but my note about maintenance on #734 stands.

The largest contributor to the size of this PR is Python code generated from protobufs. In v0.3 the generated code is still tracked in Git, and it seems like it was stale before this change, because running the codegen produced a lot more changes than just the label additions.

Does this PR introduce a user-facing change?:

Added key/value labels to feature and feature set specs for Feast v0.3, as in #536 for v0.5

Comment on lines 418 to 426
List<FeatureSpec> appliedFeatureSpecs =
new ArrayList<>(appliedFeatureSetSpec.getFeaturesList());
appliedFeatureSpecs.sort(Comparator.comparing(FeatureSpec::getName));

List<Map<String, String>> featureSpecsLabels =
featureSpecs.stream().map(e -> e.getLabelsMap()).collect(Collectors.toList());
assertThat(appliedEntitySpecs, equalTo(entitySpecs));
assertThat(appliedFeatureSpecs, equalTo(featureSpecs));
assertThat(featureSpecsLabels, equalTo(featureLabels));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed this when raising this PR: I believe that the featureSpecsLabels here should be assigned from appliedFeatureSpecs.stream() instead of featureSpecs.stream(), otherwise we are asserting on test setup code instead of covering actual roundtrip through the feature set being applied. I've asked @suwik to double-check my reasoning.

I've made that change locally and the test passes, so I believe it proves there is no functional issue with feature-level labels, we just aren't covering what we want to with the test. I'll include the update in this PR if @suwik corroborates, but the same change should be forward-ported to master, so I want to track that.

@ches
Copy link
Member Author

ches commented May 23, 2020

publish-docker-images will have the same problem as #734 for v0.3-branch, looking into that for both now.

Edit: I've tacked on the fix to #734, assuming that one is quick to review, then I'll rebase this one to get checks passing.

@ches ches force-pushed the v0.3-backport-536-labels branch from ec8cfe2 to 2a9d672 Compare May 23, 2020 11:09
@woop
Copy link
Member

woop commented May 23, 2020

@ches do you want this merged or do you want to wait for @suwik? I have heavy fingers and they tend to fall on /lgtm if I don't see problems with a PR.

It appears that this test was only exercising the test setup code, not
covering that labels work when applied.

This change should be forward-ported to master.
@ches
Copy link
Member Author

ches commented May 23, 2020

😄

I went ahead and pushed the change that I think makes sense, so we can forward-port the commit. I'm happy to go ahead with it so I can get a few more things done by Monday toward an 0.3.8 release we can use.

@woop
Copy link
Member

woop commented May 23, 2020

/lgtm

@woop
Copy link
Member

woop commented May 23, 2020

/approve

@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ches, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@feast-ci-bot feast-ci-bot merged commit ffe7ec8 into feast-dev:v0.3-branch May 23, 2020
@ches ches deleted the v0.3-backport-536-labels branch May 23, 2020 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants