Remap issues when converting between an Integer and a Float

[StephenWakely]

Remap has two numeric types Integer and Float. When performing arithmetic Rust requires both types to be the same, so Remap often needs to cast the types to ensure they are the same. Casting a number between a float and an integer doesn't always result in the same value.

For example:

Casting 36,028,797,018,963,968 as i64 to f64 results in 36,028,797,018,963,970. (Note the last two digits differ.) The converse is also true - 36,028,797,018,963,970 as f64 to i64 goes back to 36,028,797,018,963,968.

Casting 4,052,555,153,018,976,267 as i64 to f 64 results in 4,052,555,153,018,976,000. Casting that number back to i64 results in 4,052,555,153,018,976,256.

This can result in inaccurrate calculations at large values.

It can also produce surprising results. For example, in Remap:

1 * (1 / 0) == 9,223,372,036,854,775,807

In the above, for 1 / 0 both numbers are cast to a float. The result is infinity. When multiplying by 1 - since 1 is an integer, both numbers are cast to i64.

infinity as i64 is 9,223,372,036,854,775,807 - (111111111111111111111111111111111111111111111111111111111111111).

[binarylogic]

@FungusHumungus this appears to be very edge-casey, correct? And what would the solution be if we wanted to solve this?

[StephenWakely]

@binarylogic It means our arithmetic functions fall over at large numbers (integers greater than 2^52 - I think).

I'm not really sure what the solution is. Am currently reading through the chapter in Hackers Delight on it to see if I can get any inspiration. I think it would be worth asking the rest of the team next week to see if anyone has any ideas..

It might be that it's not really something anyone would ever really need to worry about, so perhaps just documenting it will suffice.

[JeanMertz]

@FungusHumungus I just realised the changes in #5663 didn't mark division as a fallible operation, which it should because we don't know at compile-time if the rhs value will be zero or not.

[JeanMertz]

Here's a test-case that passes, which it shouldn't because the error isn't handled at compile-time:

# object: {}
# result:
#
# error: program aborted
#   ┌─ :2:1
#   │
# 2 │ 1 / 0
#   │ ^ value error
#   │
#   = see language documentation at: https://vector.dev/docs/reference/vrl/

1 / 0

[JeanMertz]

Here's another one that passes, again it shouldn't because I should be able to capture the error (this is because of the incorrect type def):

# object: {}
# result:
#
# error: unneeded error assignment
#   ┌─ :2:1
#   │
# 2 │ ok, err = 1 / 0
#   │ ^^^^^^^   - because this expression cannot fail
#   │ │          
#   │ this error assignment is unneeded
#   │
#   = hint: assign to "ok", without assigning to "err"
#   = see language documentation at: https://vector.dev/docs/reference/vrl/

ok, err = 1 / 0

[StephenWakely]

I just realised the changes in #5663 didn't mark division as a fallible operation, which it should because we don't know at compile-time if the rhs value will be zero or not.

Good spot. Have submitted a PR.