Better inference for _apply of iterable

[carnaval]

Simulate the iteration protocol in inference to cope with other iterables than tuples in _apply.

This solves inference for, e.g., f(x::Vector{Int}...). I prefered this way instead of adding yet another special case for arrays, but maybe it's a bit too convoluted ?

@JeffBezanson I'm not really sure about passing dummy expressions to abstract_call, is there a better way ? I also cowardly restricted it to leaf types.

[carnaval]

ref the original issue reported by Andreas :

julia> Z() = complex(randn(2)...)
Z (generic function with 1 method)

master

julia> code_typed(Z,())
1-element Array{Any,1}:
 :($(Expr(:lambda, Any[], Any[Any[Any[symbol("##dims#6190"),Tuple{Int64},0]],Any[],Any[Array{Float64,1}],Any[]], :(begin  # none, line 1:
        GenSym(0) = (Base.Random.Array)(Base.Random.Float64,2)::Array{Float64,1}
        return (top(_apply))((top(getfield))(Main,:call)::F,Main.complex,(Base.Random.randn!)(Base.Random.GLOBAL_RNG,GenSym(0))::Array{Float64,1})
    end))))

this

julia> code_typed(Z,())
1-element Array{Any,1}:
 :($(Expr(:lambda, Any[], Any[Any[Any[symbol("##dims#6674"),Tuple{Int64},0]],Any[],Any[Array{Float64,1}],Any[]], :(begin  # none, line 1:
        GenSym(0) = (Base.Random.Array)(Base.Random.Float64,2)::Array{Float64,1}
        return (top(_apply))((top(getfield))(Main,:call)::F,Main.complex,(Base.Random.randn!)(Base.Random.GLOBAL_RNG,GenSym(0))::Array{Float64,1})::Complex{Float64}
    end::Complex{Float64}))))

[vtjnash]

is it worthwhile to simulate the actual function (https://github.com/JuliaLang/julia/blob/master/base/essentials.jl#L119)? also perhaps worth noting that vectors of only pointer arrays are handled by a special case in https://github.com/JuliaLang/julia/blob/master/src/builtins.c#L430-L432, where the assumption is that nobody will try to override iteration of an array.

[carnaval]

I don't think append_any would give any interesting result since it always output an Any array. It only does the appending part, not the unpacking (done implicitly by the builtin).

[carnaval]

ping @JeffBezanson the only thing I'm not sure about is the perf impact, what is the canonical way to see that ? Benchmark bootstrap ?

[JeffBezanson]

Looks ok to me. We can also add this much simpler patch to handle the specific issue here:

diff --git a/base/inference.jl b/base/inference.jl
index 4961b11..7668b4f 100644
--- a/base/inference.jl
+++ b/base/inference.jl
@@ -781,6 +781,8 @@ function precise_container_types(args, types, vtypes, sv)
             end
         elseif ti<:Tuple && (i==n || !isvatuple(ti))
             result[i] = ti.parameters
+        elseif ti<:AbstractArray && i==n
+            result[i] = Any[Vararg{eltype(ti)}]
         else
             return nothing
         end

We can also delete the block of code at

julia/base/inference.jl

Line 805 in 89ca810

if is(af,tuple) && length(aargtypes)==1

Yes, I would time bootstrap and also look at test suite times.

[martinholters]

Can we resurrect this to fix #20518? Looks like there was no opposition to this and the simple patch above that instead made it into master apparently has some shortcomings.

FWIW, I've tried something very similar (but also allowing non-leaf state types) on top of #20343 and it seems to work quite nicely. touch base/sysimg.jl; time make also yields about the same.

[JeffBezanson]

@martinholters Is your version of this available in #20343? We might as well use yours.

[martinholters]

Not yet, but I will include it there and update the PR shortly.

[Keno]

A version of this was included in #20343.