Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensions: Bind invocations and member accesses #77172

Merged
merged 13 commits into from
Feb 19, 2025

Conversation

jcouv
Copy link
Member

@jcouv jcouv commented Feb 12, 2025

This is mostly re-using logic from the roles branch.
The core change/method is ResolveExtension (in Binder_Expression.cs) and the various callers.
It's used when resolving method groups (binding invocations, MakeMemberAccessValue, ...) and binding member accesses (BindMemberAccessWithBoundLeft and BindInstanceMemberAccess)
The MethodGroupResolution which it returns can now also represent a non-method result (ie. we found a property instead of methods).

Function type scenarios are out-of-scope for this PR.

Relates to test plan #76130


Note that we don't need to resolve to a non-method extension member during conversions.
Here are the relevant sections of the spec that justify this understanding:

The Member Lookup has concept of a member being "invoked":

"If the simple_name or member_access occurs as the primary_expression of an invocation_expression, the member is said to be invoked."
"Next, if the member is invoked, all non-invocable members are removed from the set."

But in Method group conversions, although treat the conversion as an invocation:

A single method M is selected corresponding to a method invocation of the form E(A) ... [using arguments constructed from the parameter types of delegate type]

The method group conversion only kicks in when we have already determined that we have a method group:

"An implicit conversion exists from a method group (§12.2) to a compatible delegate type (§20.4). If D is a delegate type, and E is an expression that is classified as a method group, ..."

In short, a conversion to a delegate type does not cause the expression to be treated as "invoked", although we process it somewhat like an invocation.

Relates to test plan #66722

@jcouv jcouv self-assigned this Feb 12, 2025
@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Compilers untriaged Issues and PRs which have not yet been triaged by a lead labels Feb 12, 2025
@jcouv jcouv added Feature - Extension Everything The extension everything feature and removed Area-Compilers untriaged Issues and PRs which have not yet been triaged by a lead labels Feb 12, 2025
@@ -928,6 +928,8 @@ protected BoundCall MakeQueryInvocation(CSharpSyntaxNode node, BoundExpression r
Debug.Assert(receiver.Type is object || ultimateReceiver.Type is null);
if ((object?)ultimateReceiver.Type == null)
{
Debug.Assert(ultimateReceiver.Kind != BoundKind.MethodGroup || ultimateReceiver.HasAnyErrors);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 See previous discussion on this change #73239 (comment)

@jcouv jcouv force-pushed the extensions-lookup branch 3 times, most recently from ac436b7 to ae9a409 Compare February 12, 2025 23:03
@jcouv jcouv force-pushed the extensions-lookup branch from ae9a409 to 245479d Compare February 12, 2025 23:46
@jcouv jcouv marked this pull request as ready for review February 13, 2025 01:22
@jcouv jcouv requested a review from a team as a code owner February 13, 2025 01:22
@@ -1975,7 +1975,6 @@ protected override void AfterMembersCompletedChecks(BindingDiagnosticBag diagnos
// The type 'Microsoft.CodeAnalysis.EmbeddedAttribute' must be non-generic, internal, sealed, non-static, have a parameterless constructor, inherit from System.Attribute, and be able to be applied to any type.
diagnostics.Add(ErrorCode.ERR_EmbeddedAttributeMustFollowPattern, GetFirstLocation());
}

Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like there are meaningful changes in this file. Consider reverting it to its previous state. #Closed

MutableTypeMap? substitution = null;
bool result = CanUnifyHelper(parameterType, type, onlySubstituteInLHS: true, ref substitution);
#if DEBUG
if (result && (parameterType is not null && type is not null))
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(parameterType is not null && type is not null)

It looks like we asserted these conditions earlier. Is there a specific reason to check them? #Closed

#endif

PooledHashSet<NamedTypeSymbol>? visited = null;
var baseType = type;
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var baseType = type;

A couple of comments:

  1. It looks like we are not handling scenario when type is a generic type parameter.
  2. It is not clear to me why we are not using the same approach that ReducedExtensionMethodSymbol.Create uses to check compatibility and a general type argument inference algorithm like in ReducedExtensionMethodSymbol.InferExtensionMethodTypeArguments. We have a parameter and we have a type for the corresponding argument. What is the reason to use different approach? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR re-uses the algorithm from the roles branch as we'd discussed. But in email I suggested that we use method type inference here instead of type unification, which is what ReducedExtensionMethodSymbol does. I was planning to do that in a subsequent PR, but am happy to include that change here.

Yes, this PR doesn't handle type parameter receivers (it's explicitly blocked in places, marked with PROTOTYPE). I probably can revisit that after change to type inference.

if (lookupResult.IsMultiViable)
{
return BindMemberOfType(node, right, rightName, rightArity, indexed, boundLeft, typeArgumentsSyntax, typeArguments, lookupResult, BoundMethodGroupFlags.None, diagnostics: diagnostics);
return BindMemberOfType(node, right, rightName, rightArity, indexed, boundLeft, typeArgumentsSyntax, typeArguments, lookupResult, BoundMethodGroupFlags.SearchExtensions, diagnostics: diagnostics);
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BoundMethodGroupFlags.SearchExtensions

Should this be conditional? Perhaps, based on options? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flag should be set unconditionally, unless we can identify a type member access scenario where we shouldn't resolve extensions.

In short:

Previously, we'd only set the flag in instance member access scenarios. It would generally indicate that the member group came from an instance member access, with a couple of limited exceptions (see below).
Now, we also want to set this flag for method groups resulting for a type member access. The previous exceptions are not applicable and we haven't found new exceptions that would require using this flag conditionally here.
Setting this flag on type member access enables invocation scenarios (ResolveMethodGroup should call ResolveExtensions, see methodGroup.SearchExtensions check) and natural function type scenarios (GetUniqueSignatureFromMethodGroup should inspect extensions in a future PR).

Note: although LookupOptions does have an IncludeExtensionMethods flag, I don't think it's relevant here (it's only used for lookup APIs in the semantic model). The options chosen in BindMemberAccessWithBoundLeft don't need to use it.

My thought process:

This flag gets stored into the BoundMethodGroup in cases where one is returned:

  • in BindMemberOfType, we either return a method group with this flag (when a method group is found or no result), or a non-method result such as a type, property access, event access, field access (which don't use this flag)
  • in MakeBoundMethodGroupAndCheckOmittedTypeArguments always returns a method group with this flag

The flag indicates that subsequent usage of the method group should look at extensions, because it came from a member access where we want to allow extension resolution.

  1. if we don't have a member access, we don't use this flag. For example, we don't use it when binding an identifier (see ConstructBoundMemberGroupAndReportOmittedTypeArguments call in BindIdentifier).
  2. for a type receiver, we previously didn't set this flag
  3. for the remaining cases, we generally set the flag for a member access with two exceptions: in BindMemberAccessWithBoundLeftTypeOrInstance if !searchExtensionsIfNecessary (only used for binding interpolation handlers, which purposefully exclude extension resolution) andleftIsBaseReference(no extension resolution onbase` access).

We want to set the flag for case 2 and have not yet identified any exceptions for that scenario.

}

return MakeBoundMethodGroupAndCheckOmittedTypeArguments(boundLeft, rightName, typeArguments, lookupResult,
flags: BoundMethodGroupFlags.SearchExtensions, node, typeArgumentsSyntax, diagnostics);
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BoundMethodGroupFlags.SearchExtensions

Similar question. #Closed

// Note: we're resolving without arguments, which means we're not treating the member access as invoked
var resolution = this.ResolveExtension(
syntax, name, analyzedArguments: null, receiver, typeArgumentsOpt, options: OverloadResolution.Options.None,
returnRefKind: default, returnType: null, withDependencies: useSiteInfo.AccumulatesDependencies);
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useSiteInfo.AccumulatesDependencies

It feels like an overkill to create useSiteInfo just to use this property. We should be able to get this information directly from diagnostics. #Closed

}
#nullable disable

private BoundExpression BindMemberAccessWithBoundLeftCore(
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BindMemberAccessWithBoundLeftCore

It is not clear what motivated the rename. Especially that "Core" suffix isn't rely informative. Is there "non-Core" version? What does that do vs. this one doesn't? #Closed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the "Instance" part of the original name inaccurate? I think we need to come up with a clearer, more descriptive name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From offline discussion, yes the issue was with "Instance". The bound left may not be an instance. I'll take a stab at improving the name

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up reverting this. I still think the name isn't great, but it's not essential for our present purpose

}
else if (firstResult.IsEmpty)
{
firstResult = extensionResult;
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

firstResult = extensionResult;

We don't need to free firstResult here? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

firstResult.IsEmpty indicates that don't have anything to free (in particular, no analyzed arguments). I'll add an assertion to clarify.

{
if (extensionResult.IsNonMethodExtensionMember(out _))
{
return extensionResult;
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return extensionResult;

We don't need to free firstResult here? #Closed

if (extensionMethodResult.HasAnyApplicableMethod)
{
firstResult.Free();
return extensionMethodResult;
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return extensionMethodResult;

Are we supposed to free actualArguments here? #Closed

Debug.Assert(extensionMemberAccess.Kind != BoundKind.MethodGroup);

extensionMemberAccess = CheckValue(extensionMemberAccess, BindValueKind.RValue, diagnostics);
BoundExpression extensionMemberInvocation = BindInvocationExpression(syntax, expression, methodName, extensionMemberAccess, analyzedArguments, diagnostics);
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

methodName

This looks suspicious, it looks like we already resolved the name. #Closed

}

Debug.Assert(!resolution.IsNonMethodExtensionMember(out _));
return resolution;
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to remove return resolution; above? Otherwise this code can be moved to the else block. #Closed

@@ -33,9 +33,7 @@ static void Main(string[] args)
string expectedOperationTree = @"
IInvalidOperation (OperationKind.Invalid, Type: ?, IsInvalid) (Syntax: 'Console.WriteLine2()')
Children(1):
IInvalidOperation (OperationKind.Invalid, Type: ?, IsInvalid) (Syntax: 'Console.WriteLine2')
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IInvalidOperation (OperationKind.Invalid, Type: ?, IsInvalid) (Syntax: 'Console.WriteLine2')

Do we understand why this node disappeared? #Closed

Copy link
Member Author

@jcouv jcouv Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is due to the change in tryBindMemberAccessWithBoundTypeLeft (Binder_Expression.cs line 7916)
See explanation: #68629 (comment)
Also, letting that scenario return null instead of a method group breaks many scenarios (for example, StaticMethodInvocation_Simple would error saying that C doesn't contain a definition for M)

@AlekseyTs
Copy link
Contributor

Done with review pass (commit 1)

@jcouv jcouv requested a review from AlekseyTs February 14, 2025 17:55
return extension.Construct(typeArguments);
}

static bool checkConstraints(Symbol symbol, ImmutableArray<TypeParameterSymbol> typeParams, ImmutableArray<TypeWithAnnotations> typeArgs,
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Symbol

NamedTypeSymbol? #Closed

return extension.Construct(typeArguments);
}

static bool checkConstraints(Symbol symbol, ImmutableArray<TypeParameterSymbol> typeParams, ImmutableArray<TypeWithAnnotations> typeArgs,
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ImmutableArray typeParams

It looks like this parameter is redundant, we can get type parameters from extension type. #Closed

@@ -184,6 +184,8 @@ internal override UseSiteInfo<AssemblySymbol> GetUseSiteInfo()
internal sealed override bool IsRecordStruct => false;
internal sealed override bool HasPossibleWellKnownCloneMethod() => false;

internal sealed override ParameterSymbol ExtensionParameter => throw ExceptionUtilities.Unreachable();
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

throw ExceptionUtilities.Unreachable()

It is probably safe to return null as we do in other similar cases. #Closed

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Feb 14, 2025

        // (3,1): error CS0012: The type 'Missing' is defined in an assembly that is not referenced. You must add a reference to assembly 'missing, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'.

I am not sure how this error is relevant to the compiled scenario. #Closed


Refers to: src/Compilers/CSharp/Test/Emit3/Semantics/ExtensionTests.cs:6039 in bc0b7ed. [](commit_id = bc0b7ed, deletion_comment = False)


static bool tryResolveExtensionMember(Binder binder, SyntaxNode expression, string memberName, AnalyzedArguments? analyzedArguments, BoundExpression left,
ImmutableArray<TypeWithAnnotations> typeArgumentsWithAnnotations, OverloadResolution.Options options, RefKind returnRefKind, in CallingConventionInfo callingConvention,
TypeSymbol? returnType, BindingDiagnosticBag diagnostics, ExtensionScope scope, out MethodGroupResolution extensionResult)
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BindingDiagnosticBag diagnostics

This approach doesn't feel right. Why would we want to attach an unrelated diagnostics to MethodGroupResolution that we create? The diagnostics probably contains errors from previous iterations of the loop above, i.e. a different extension. #Closed

@AlekseyTs
Copy link
Contributor

Done with review pass (commit 5)

lookupResult, left.Type!, memberName, arity,
basesBeingResolved: null, lookupOptions, originalBinder: binder, ref useSiteInfo);

diagnostics.Add(expression, useSiteInfo);
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diagnostics.Add(expression, useSiteInfo);

I think this is where the problem is. This is where we are dropping use-site in case diagnostics is dropped below. And the diagnostics is dropped for a good reason. I think use-site diagnostics should be collected separately on the ResolveExtension level. I.e. it should be passed in as a parameter and we shouldn't be merging it into diagnostic bags that might be dropped. #Closed

BoundExpression left,
ImmutableArray<TypeWithAnnotations> typeArgumentsWithAnnotations,
OverloadResolution.Options options,
RefKind returnRefKind,
TypeSymbol returnType,
TypeSymbol? returnType,
bool withDependencies,
Copy link
Contributor

@AlekseyTs AlekseyTs Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bool withDependencies,

Here, I think that instead of passing withDependencies, we should pass ref CompoundUseSiteInfo<AssemblySymbol> useSiteInfo. And keep passing it down and collecting use-site diagnostics into it when lookup might be failing because of them, and in other similar cases. #Closed

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Feb 14, 2025

    Assert.Equal("void N.E2.<>E__0<Container>.M()", model.GetSymbolInfo(memberAccess).Symbol.ToTestDisplayString());

This change is not expected #Pending


Refers to: src/Compilers/CSharp/Test/Emit3/Semantics/ExtensionTests.cs:6036 in bc0b7ed. [](commit_id = bc0b7ed, deletion_comment = True)

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Feb 14, 2025

    extension<T>(T t)

Consider testing without this extension. We should still get use-site error. #Closed


Refers to: src/Compilers/CSharp/Test/Emit3/Semantics/ExtensionTests.cs:6030 in 8462cf7. [](commit_id = 8462cf7, deletion_comment = False)

@AlekseyTs
Copy link
Contributor

            diagnostics.Add(expression, useSiteInfo); // PROTOTYPE test use-site info

It feels like this place shouldn't change. We are not dropping diagnostics and it is specific to the resolution that we return.


Refers to: src/Compilers/CSharp/Portable/Binder/Binder_Expressions.cs:8681 in 8462cf7. [](commit_id = 8462cf7, deletion_comment = True)

@jcouv
Copy link
Member Author

jcouv commented Feb 17, 2025

            diagnostics.Add(expression, useSiteInfo); // PROTOTYPE test use-site info

Let's discuss next week. I don't understand why we would want some use-site diagnostics attached to the result, but not others.

I think we should treat all use-site diagnostics in ResolveExtension the same.
Since the caller in an invocation (ResolveMethodGroupInternal) may choose to use the default method group result or the extension method group, and it's caller (ResolveMethodGroup) has further logic in case the method resolution has no errors, so I think we should collect all the use-site diagnostics from ResolveExtension and attach them all to the method resolution that we end up producing.


In reply to: 2660477061


Refers to: src/Compilers/CSharp/Portable/Binder/Binder_Expressions.cs:8681 in 8462cf7. [](commit_id = 8462cf7, deletion_comment = True)

@AlekseyTs
Copy link
Contributor

            diagnostics.Add(expression, useSiteInfo); // PROTOTYPE test use-site info

There is a good rational behind this argument. At the same time it looks like MethodInvocationOverloadResolution doesn't add use-site diagnostics for "bad" candidates into useSiteInfo parameter. Therefore, it appears, if we propagate it separately, we will collect only some of the info, not all of it. I guess, it is worth following up on this issue later.


In reply to: 2661981228


Refers to: src/Compilers/CSharp/Portable/Binder/Binder_Expressions.cs:8681 in 8462cf7. [](commit_id = 8462cf7, deletion_comment = True)

Copy link
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (commit 8)

@jcouv
Copy link
Member Author

jcouv commented Feb 18, 2025

@jjonescz for second review. Thanks

src/Compilers/CSharp/Portable/Binder/Binder_Conversions.cs Outdated Show resolved Hide resolved
src/Compilers/CSharp/Portable/Binder/Binder_Expressions.cs Outdated Show resolved Hide resolved
src/Compilers/CSharp/Portable/Binder/Binder_Expressions.cs Outdated Show resolved Hide resolved
src/Compilers/CSharp/Portable/Binder/Binder_Lookup.cs Outdated Show resolved Hide resolved
src/Compilers/CSharp/Portable/Binder/Binder_Lookup.cs Outdated Show resolved Hide resolved
src/Compilers/CSharp/Portable/Symbols/TypeSymbol.cs Outdated Show resolved Hide resolved
@jcouv jcouv enabled auto-merge (squash) February 19, 2025 17:23
@jcouv jcouv merged commit 9be7ce0 into dotnet:features/extensions Feb 19, 2025
28 checks passed
@jcouv jcouv deleted the extensions-lookup branch February 19, 2025 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Compilers Feature - Extension Everything The extension everything feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants