Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(remap): Make del function only take one arg #5633

Merged
merged 29 commits into from
Jan 7, 2021

Conversation

lucperkins
Copy link
Contributor

@lucperkins lucperkins commented Dec 20, 2020

At the moment, the del function can be used to delete any number of object fields. This capability, however, is inconsistent with an emerging core tenet of TRL, which is that functions should return a single value. If del is called on two fields and only one exists, it's not clear what the appropriate response should be. Furthermore, it's been decided that del should return the value of the field deleted rather than null, in order to "bubble up" the value of the field upon deletion and thus enable expressions like this:

.ts = to_timestamp(del(.timestamp))

Closes #5605

@lucperkins lucperkins marked this pull request as draft December 20, 2020 05:35
@lucperkins
Copy link
Contributor Author

lucperkins commented Dec 21, 2020

@JeanMertz @FungusHumungus There's a bit more work that needs to be done here to have del return the value, but I'm not sure about the desired behavior. What's the desired behavior for del(.non_existent_field)?

I'll finish this up when we get the desired end state sorted.

@binarylogic
Copy link
Contributor

@lucperkins does this close an issue?

@lucperkins
Copy link
Contributor Author

@binarylogic It might address #5566, but I'm going to hold off on pushing any further on this until there's consensus around expected behavior.

@jamtur01
Copy link
Contributor

Also linked to #5605.

@jamtur01
Copy link
Contributor

To move this forward: @binarylogic @lucperkins For this release we just need to answer: should calling del on a non-existing field error or return null?

@binarylogic
Copy link
Contributor

For now, it should return null until we finalize the error handling story.

@lucperkins lucperkins marked this pull request as ready for review December 29, 2020 19:12
@StephenWakely
Copy link
Contributor

It looks like this is always returning Null. It would be really handy if we could address #5150 here as well so that del returns the value of the field being deleted - only returning Null if the field doesn't exist.

I'm sure I saw somewhere you added a remove_and_get function to Object that handled this case, but I can't seem to find it now.

@lucperkins
Copy link
Contributor Author

lucperkins commented Dec 29, 2020

@FungusHumungus You're not just seeing things 😄 I did have that. I think I misunderstood the outcome of the discussion above and I now see that del should return null only if the field doesn't exist. I'll revert.

@lucperkins
Copy link
Contributor Author

lucperkins commented Dec 29, 2020

@FungusHumungus Okay, I've reverted that behavior. I've left remove_and_get unimplemented for now in LogEvent and Metric. As far as I can tell those implementations aren't currently necessary for VRL, but if that's incorrect, let me know and I'll dig in and supply them, though I may need a bit of help.

@jamtur01 jamtur01 added domain: vrl Anything related to the Vector Remap Language transform: remap Anything `remap` transform related labels Jan 1, 2021
Copy link
Contributor

@JeanMertz JeanMertz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, but isn't quite what we want just yet. I left some notes on what I think we should do before we merge this PR.

docs/reference/remap/append.cue Outdated Show resolved Hide resolved
docs/reference/remap/del.cue Outdated Show resolved Hide resolved
lib/remap-functions/src/del.rs Outdated Show resolved Hide resolved
lib/remap-functions/src/del.rs Outdated Show resolved Hide resolved
lib/remap-functions/src/del.rs Outdated Show resolved Hide resolved
Luc Perkins added 2 commits January 4, 2021 11:19
Signed-off-by: Luc Perkins <[email protected]>
docs/reference/remap/append.cue Outdated Show resolved Hide resolved
lib/remap-lang/src/object.rs Outdated Show resolved Hide resolved
Luc Perkins and others added 4 commits January 5, 2021 10:15
Signed-off-by: Luc Perkins <[email protected]>
Signed-off-by: Jean Mertz <[email protected]>
Signed-off-by: Jean Mertz <[email protected]>
Signed-off-by: Jean Mertz <[email protected]>
@JeanMertz
Copy link
Contributor

I updated the PR based on the latest consensus in #5633 (comment). This should be good for a final round of reviews before we merge it.

Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Just a few small suggestions / questions.

docs/reference/remap.cue Outdated Show resolved Hide resolved
docs/reference/remap/del.cue Show resolved Hide resolved
lib/remap-functions/src/del.rs Outdated Show resolved Hide resolved
lib/remap-functions/src/del.rs Show resolved Hide resolved
Signed-off-by: Jean Mertz <[email protected]>
Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

"user_id": 1
}
source: #"""
.user.id = if exists(.user_id) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL we support ifs as expressions 😄

Luc Perkins and others added 3 commits January 6, 2021 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: vrl Anything related to the Vector Remap Language transform: remap Anything `remap` transform related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

remove multiple arguments from del function
6 participants