make ifelse generic

[gwater]

As suggested on discourse, I'd like to propose exporting ifelse() as a normal function rather than a builtin.

We have a usecase in NumberIntervals.jl for extending ifelse() to do some three-value logic which is currently impossible for builtins.

CC @yuyichao @oschulz

[vchuravy]

It used to be a generic function #27068 and eschnett/SIMD.jl#28

Maybe we can separate Core.ifelse from Base.ifelse or introduce select.

[yuyichao]

AFAIK we don't even need select or Core.ifelse. The only benefit of ifelse is the eager evaluation of the argument which does not come from the builtin. I've seen LLVM makes the transformation either way (select <-> br) and in the off chance that LLVM does not make the right call it should be very easy to do our own pattern matching for simple cases.

[oschulz]

Sounds promising. :-) I think being able to specialize ifelse could make for some powerful idioms (e.g. the three-valued logic application @gwater and I had in mind).

[oschulz]

@gwater and me come to the conclusion that it's probably best to use missing as a result for comparison of arithmetic intervals, in cases where the intervals overlap.

@yuyichao, if ifelse were to become a simple function again, would it be acceptable to add

ifelse(::Missing, a, b) = missing

to Base?

[StefanKarpinski]

What if a === b? Do you still want the value to be missing?

[gwater]

Assuming isequal() never returns missing, this definition seems sensible:

ifelse(::Missing, a, b) = ifelse(a === b, a, missing)

It might actually recover some cases after an inconclusive result (which would currently throw a TypeError).

[oschulz]

I agree, that seems elegant.

[oschulz]

Just curious, @KristofferC, why don't you like missing propagating through ifelse? Do you expect this to cause trouble?

[StefanKarpinski]

I'll explain for him. When we agreed to put missing in Base, there was a deal struck: we'll put this special type in Base to help support data science, but we will not add method for missing to every function—that will be up to the end user to deal with missing propagation outside of a handful of really core operations. Since that deal was struck, the data science side has not kept up their side of the deal. Every other day, it seems, there's a request for some random function in Base to support missing. Are we really supposed to add methods for every goddamned function in the language? Where does it stop?

[oschulz]

Ah, right, I understand. @gwater and I were discussing whether we should use Missing or a custom type for this interval comparison three-valued logic. @StefanKarpinski , would you say that as a guideline, custom types would be preferable to Missing in such more specialized cases?

[StefanKarpinski]

I don't know, and I understand the desire to use missing, I'm just explaining the objection.

[ararslan]

FWIW I also object to 1. making ifelse generic, and 2. defining ifelse(::Missing, a, b), but for slightly different reasons. Extending ifelse with random things introduces "truthiness" and "falsiness"; an if condition should only ever evaluate to a Bool, otherwise it should be an error. If your condition can evaluate to missing, you should check for that before conditioning. Propagating missing into conditionals is way too far.

[mbauman]

Ignoring the use-case for now, I'd say that I'd very much like for us to never export intrinsics/Core.Builtins. There are several reasons for this:

• They don't work with normal reflection tools (e.g.,@which and friends). ArgumentError: argument is not a generic function is a bizarre error message, especially when…
• This language is built on generic functions and multiple dispatch! Why should we expose users to non-generic functions when it's so easy to throw a simple wrapper around these intrinsics?
• And, sure, maybe ifelse shouldn't be extended with multiple dispatch, but we've never been a language with strong guardrails. We let folks do things they probably shouldn't do all the time. Even if someone wants to add truthiness to ifelse, they'll still be limited by the fact that normal if statements will never be truthy.

[JeffBezanson]

I'm not sure which side of this I'm on. I'm generally against adding methods for missing. On the other hand, potentially supporting missing is another reason to have the ifelse function instead of just if and ?. For example, it can be seen as

ifelse(c, a, b) = (c & a) | (!c & b)

but with better 3VL support such that c | !c is true even when c is missing.

[oschulz]

I understand the reluctance to use missing for too many things. It certainly doesn't have to be missing, in our case, we could also use a custom type.

The specific use case that motivated this is the comparison of arithmetic intervals. They can be interpreted as numbers with an uncertainty, so comparisons do make sense. But when the intervals overlap, neither true nor false is a good result (e.g. when using intervals for optimization problems, with user code that wasn't specifically written to handle intervals).

So the idea is to use three-valued logic: true, false and third true_or_false value. After some discussions, @gwater and I thought, why not use missing for that, since it's already well supported. But we could define a custom TrueOrFalse type, instead. Specializing ifelse for this type would be nice though, it would make it easier to write generic code that supports both normal numbers and intervals.

[JeffBezanson]

I understand the reluctance to use missing for too many things. It certainly doesn't have to be missing, in our case, we could also use a custom type.

IMO that is not the problem --- if we want to allow adding methods to ifelse, and we're going to add a method that handles an unknown boolean value as the first argument, then as far as I'm concerned that value might as well be missing. The alternative would be requiring people to handle the unknown value in a different way in their code, outside ifelse itself.

[oschulz]

and we're going to add a method that handles an unknown boolean value as the first argument, then as far as I'm concerned that value might as well be missing

If it's not missing, it could be done outside of Base though (in this case, in NumberIntervals.jl), if you guys prefer to keep Base lean.

[PallHaraldsson]

"Unordered" for floats seems relevant here (no language implements?).

IEEE 754 floating-point data comparisons can have four possible outcomes -
= E Equal
< L Less than
"> G Greater than" # had to put in quotes to not screw up formatting.
? U Unordered

[..] that possibility was removed from 2019.
Language standards are required to define 22 boolean predicates.

I found that and quoted with link plus more on IEEE-754-2019 and potentially sqrt(-0.0) at: https://www.quora.com/Why-is-it-that-mathematicians-say-positive-zero-zero-and-negative-zero-In-the-book-I-am-reading-Calculus-for-Dummies-the-guy-used-negative-zeros-for-some-things-like-in-the-left-rectangle-rule/answer/Alon-Amit/comment/105579523

[nalimilan]

I agree we shouldn't add definitions for Missing to an arbitrarily growing set of functions (we need a more general mechanism like f?(x) for that). Though as @JeffBezanson noted, three-valued logic is quite special. Actually it's one of the only properties of Missing which really has to be supported deep inside Base (e.g. in == for arrays), so ifelse could deserve a special treatment. It would make sense to have ifelse follow 3VL, since it's to & and | what if/? are to && and ||: it doesn't short-circuit, so it can propagate missingness.

[ChrisRackauckas]

For symbolic libraries like ModelingToolkit, this is coming up because we want to be able to have the function ifelse as part of the possible symbolic functions, but we cannot add dispatches to it. Xref: https://discourse.julialang.org/t/modelingtoolkit-jl-and-differential-equations-with-step-pulse-inputs/40272/4?u=chrisrackauckas

[ChrisRackauckas]

Bumping this again. This issue seems to be one of the main recurring issues in ModelingToolkit, Pumas, etc. @chriselrod hit this the other day as well, because https://github.com/chriselrod/LoopVectorization.jl needs to define vifelse in order to allow overloads, so the plan in Flux.jl and NNLib.jl now is to replace every instance of ifelse with vifelse so that it can get the proper overload for @avx.

But this really raises the question: should Zygote, Flux, NNLib, LoopVectorization, ModelingToolkit, Pumas, SymbolicUtils, and probably more all be standardizing around a non-Base ifelse and telling all users that code will break if ifelse is used, or should ifelse be changed?

Pinging @chriselrod @AStupidBear @DhairyaLGandhi whom I know this has bitten.

[chriselrod]

Could we go ahead and export a generic ifelse separate from Core.ifelse, leaving the decisions with respect to ifelse and missing to a separate issue?

[c42f]

But this really raises the question: should Zygote, Flux, NNLib, LoopVectorization, ModelingToolkit, Pumas, SymbolicUtils, and probably more all be standardizing around a non-Base ifelse and telling all users that code will break if ifelse is used, or should ifelse be changed?

This along with Matt's comments in #32844 (comment) seem pretty convincing to me.

I thought we should make this change so I made the obvious patch to separate builtin Core.ifelse from generic Base.ifelse, however this causes a regression in inference which I couldn't figure out quickly. The history in #27068 shows some of the special treatment that the builtin is getting in inference.