Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: clean up loading of UDFs/UDTFs/UDAFs #3726

Merged
merged 1 commit into from
Nov 5, 2019

Conversation

purplefox
Copy link
Contributor

Description

This PR is a copy of #3702 which has been closed due to a corrupt Jenkins workspace.

Please note this is a stacked PR. Please don't review changes from other PRs in the stack.

This PR cleans up the loading of functions. The code has been simplified by replacing grungy stream logic with good old fashioned for loops. I've also extracted a couple of other things out to new functions to aid clarity.

Testing done

Non functional change

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@purplefox purplefox requested a review from a team as a code owner November 4, 2019 09:09
@purplefox purplefox requested a review from agavra November 4, 2019 09:17
private <T> Map<String, T> createMap(Function<Integer, T> valueSupplier) {
Map<String, T> map = new HashMap<>();
for (int i = 0; i < ENTRIES; i++) {
map.put(UUID.randomUUID().toString(), valueSupplier.apply(i));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test for null keys and null values. I see that you have an assertion for values to not be null but since the keys are optional schema, they could be null right? Also, there is not check in the actual class for null keys and values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vpapavas - please put these comments on #3724 this PR is stacked on top of that one

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooops sorry, will add them there

private Range rangeUdf = new Range();

@Test
public void shouldComputeIntRange() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it an overkill to test for negative start and end?

],
"inputs": [
{
"topic": "test_topic", "key": 1, "value": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add cases where key and value are null

Copy link
Member

@vpapavas vpapavas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @purplefox! It LGTM with one comment on handling null keys/values in the entries udf.

Copy link
Contributor

@agavra agavra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 thanks @purplefox! I like this patch :) (only reviewed the second commit, please merge the first one separately)

@agavra agavra requested a review from a team November 4, 2019 18:25
@purplefox purplefox force-pushed the func_load_refactor_8_new2 branch from 724c0a5 to efa27bd Compare November 5, 2019 12:32
@purplefox purplefox merged commit ee93755 into confluentinc:master Nov 5, 2019
purplefox added a commit that referenced this pull request Nov 5, 2019
This PR cleans up some of the function loading code, mainly replacing some overly complex streams based logic with simpler direct iteration.
@purplefox purplefox deleted the func_load_refactor_8_new2 branch January 26, 2020 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants