Proposal: replace Indeterminate() with Base.missing

[gwater]

I feel like we're reinventing the wheel in indetermiate.jl; and it is leading to some method creep. There is a lot of support in Base for three-value logic, especially in Boolean array methods, like all() or any(). Keeping a custom three-value logic means reimplementing all those functions.

On the other hand, the only reason for using a custom type instead of Base.missing (as far as I can tell) is that it would allow something like

ifelse(::Indeterminate, a::Real, b::Real) = NumberInterval(a, b) 

(assuming a<b). With Base.missing that would be type-piracy.

Since extending ifelse() is currently off the table anyway, I think we should switch to Base.missing as soon as possible. Am I missing something, @oschulz? (No pun intended.)

[gwater]

An alternative would be extending:

import Base: ismissing
ismissing(::Indeterminate) = true

That would, in most cases, allow Base methods to promote Indeterminate() to missing. However I'm very concerned about weird interactions when a type claims to be something it isn't. (See the whole Number/Real debate in IntevalArithmetic.jl).

[gwater]

Oh, and of course this proposal would break the exception catching concept (for TypeError). Although it might be ok to treat missing information more generally rather than specializing for missing data due to interval arithmetic?

[oschulz]

Hmm, I have to admit I'm unsure about this myself. Using Base.missing seems nice and simple. But on the other hand, it really makes things a bit unspecific, and there may be other things besides ifelse that one may want to specialize in the future. And then, semantically, it's not really "missing" information - semantically, it's actually more like three-valued logic, resp. a [false, true] interval.

[gwater]

And then, semantically, it's not really "missing" information

I would say it is missing information: we simply don't know which two numbers we are comparing (for example). And Base documentation suggests that missing was added explicitly to implement three valued logic.

Consider for example Base.==(::AbstractArray, ::AbstractArray) it would be very annoying to reimplement that for the Indeterminate type because it would be 100% the same function

[gwater]

And as long as we don't add methods to Missing, it's not type piracy

[oschulz]

Sure - but then, you did get kind of positive replies to you suggestion to make ifelse a non-builtin again. :-)

[gwater]

True, I think there are three methods we could implement without type piracy

ifelse(::Missing, ::NumberInterval, ::NumberInterval)
ifelse(::Missing, ::NumberInterval, ::Number)
ifelse(::Missing, ::Number, ::NumberInterval)

the only one that would be problematic imho, is ifelse(::Missing, ::Number, ::Number)

[oschulz]

True. Alas, ifelse(::Missing, ::Number, ::Number) may be quite relevant.

How about using a specialized type like TrueOrFalse for now. It can always be turned into an alias for Missing later on, if we find it unnecessary. However, moving in the other direction, from Missing to a specialized type, is bound to break user code.

[gwater]

True. Alas, ifelse(::Missing, ::Number, ::Number) may be quite relevant.

I'm not sure intervals are even the right result for that operation. The set of solutions isnt the interval [a, b] (for ifelse(::Missing, a::Number, b::Number), assuming a<b) it's really the set {a,b}.

[oschulz]

Hm, yes. Ok, assuming we'll go with missing - we could propose ifelse(::Missing, a, b) = missing.

[gwater]

Yes, I think that would make sense in a Base context.

I guess in the future one could also propose a generalized "set arithmetic" which is able to compose arbitrary sets of numbers, not just intervals. Then, taking in two sets of single numbers, {a} and {b}, one could reasonably return their union {a, b} (without any type piracy). It's beyond the scope here, but I think it's worth considering for a future package…

[oschulz]

Yes, I agree.

[gwater]

Ok, so I think I will move forward with #17 .

Would you like to add ifelse(::Missing, a, b) = missing to JuliaLang/julia/issues/32844 ? I think it would make sense to combine the proposal to make ifelse generic with that extending it in Base.

[oschulz]

Good idea - done.

[gwater]

ok, thanks