Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve foldl's stability on nested Iterators. #45789

Merged
merged 5 commits into from
Jun 28, 2022

Conversation

N5N3
Copy link
Member

@N5N3 N5N3 commented Jun 23, 2022

Fix #45748.
This PR splits _xfadjoint into _xfadjoint_unwrap and _xfadjoint_wrap thus helps compiler to realize that this recursion is convergent.
Test added.

N5N3 added 2 commits June 23, 2022 16:01
1. Make `Fix1(f, Int)` stable
2. split `_xfadjoint` into `_xfadjoint_unwrap` and `_xfadjoint_wrap`
@N5N3 N5N3 added the compiler:inference Type inference label Jun 23, 2022
@N5N3 N5N3 requested a review from tkf June 23, 2022 08:34
@@ -1078,8 +1078,7 @@ struct Fix1{F,T} <: Function
f::F
x::T

Fix1(f::F, x::T) where {F,T} = new{F,T}(f, x)
Fix1(f::Type{F}, x::T) where {F,T} = new{Type{F},T}(f, x)
Fix1(f, x) = new{Core.Typeof(f),Core.Typeof(x)}(f, x)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This usage of Core.Typeof is not correct, because Typeof doesn't always return legal types:

julia> struct Fix1{F,T} <: Function
           f::F
           x::T

           Fix1(f, x) = new{Core.Typeof(f),Core.Typeof(x)}(f, x)
       end

julia> Fix1(==, Vector{Core._typevar(:T, Union{}, Any)})
ERROR: TypeError: in new, expected DataType, got Type{Fix1{typeof(==), Type{Array{T, 1}}}}
Stacktrace:
 [1] Fix1(f::Function, x::Type)
   @ Main ./REPL[3]:5
 [2] top-level scope
   @ REPL[4]:1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use it for f? (I see ComposedFunction(outer, inner) = new{Core.Typeof(outer),Core.Typeof(inner)}(outer, inner) above)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's possible to define methods on incomplete types with free typevars, but it will change the kind of error thrown to a TypeError during construction instead of a MethodError when actually called, so it's not great either

Copy link
Member Author

@N5N3 N5N3 Jun 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried f(::Type{T}) where {T} = T, but f(Vector{Core._typevar(:T, Union{}, Any)}) throws a ERROR: UndefVarError: T not defined
So we need something like:

_typeof(x) = typeof(x)
_typeof(::Type{T}) where {T} = @isdefined(T) ? Type{T} : DataType

?
If this is acceptable, should we also modifies

Returns(value) = new{Core.Typeof(value)}(value)

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anything we should talk at this point, or is this PR ready for merge?

Copy link
Member Author

@N5N3 N5N3 Jun 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with incomplete type. If core devs are happy with the change in a929310, I think it’s ready to merge.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged, thanks

@N5N3 N5N3 force-pushed the recursion_inference branch from 29e506f to 8e9d35f Compare June 25, 2022 06:57
@oschulz
Copy link
Contributor

oschulz commented Jun 25, 2022

Closes #45715

@vtjnash vtjnash merged commit d58289c into JuliaLang:master Jun 28, 2022
@oschulz
Copy link
Contributor

oschulz commented Jun 28, 2022

@vtjnash - since this doesn't change behavior, could it be backported to v1.9 v1.8?

@oschulz
Copy link
Contributor

oschulz commented Jun 28, 2022

could it be backported to v1.9 v1.8?

Ah, I meant Julia v1.8, of course.

@N5N3 N5N3 deleted the recursion_inference branch June 29, 2022 01:45
pcjentsch pushed a commit to pcjentsch/julia that referenced this pull request Aug 18, 2022
* Make `Fix1(f, Int)` inference-stable
* split `_xfadjoint` into `_xfadjoint_unwrap` and `_xfadjoint_wrap`
* Improve `(c::ComposedFunction)(x...)`'s inferability
* and fuse it in `Base._xfadjoint`.
* define a `Typeof` operator that will partly work around internal type-system bugs

Closes JuliaLang#45715
@simsurace
Copy link
Contributor

It would be really great if this could be backported to 1.8.

@KristofferC KristofferC added the backport 1.8 Change should be backported to release-1.8 label Aug 30, 2022
KristofferC pushed a commit that referenced this pull request Aug 30, 2022
* Make `Fix1(f, Int)` inference-stable
* split `_xfadjoint` into `_xfadjoint_unwrap` and `_xfadjoint_wrap`
* Improve `(c::ComposedFunction)(x...)`'s inferability
* and fuse it in `Base._xfadjoint`.
* define a `Typeof` operator that will partly work around internal type-system bugs

Closes #45715

(cherry picked from commit d58289c)
@KristofferC KristofferC removed the backport 1.8 Change should be backported to release-1.8 label Sep 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:inference Type inference
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inference suboptimality when using nested generators
7 participants