Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax parameter-argument modifier matching for ref readonly parameters #68224

Merged

Conversation

jjonescz
Copy link
Member

@jjonescz jjonescz commented May 17, 2023

Allows calling methods with ref readonly parameters. Specifically:

  • Modifies overload resolution (GetEffectiveParameterRefKind) to allow passing
    • in/ref/none arguments to ref readonly parameters and
    • ref arguments to in parameters in new C# versions.
  • Emits warnings about some ref modifier mismatches (CheckRefArguments, called after overload resolution).
  • Fixes codegen to handle ref readonly arguments. Their ref kinds are lowered to in or "strict in" for simplicity (LocalRewriter.GetEffectiveArgumentRefKinds).
  • Fixes value kind of ref readonly parameters to report correct errors (e.g., when ref readonly parameter is passed as a mutable ref).

Speclet: https://github.com/dotnet/csharplang/blob/91f850ce3c5a795021e94cd7a2b4f002a64a05a8/proposals/ref-readonly-parameters.md
Test plan: #68056

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Compilers untriaged Issues and PRs which have not yet been triaged by a lead labels May 17, 2023
@jjonescz jjonescz added the Feature - Ref Readonly Parameters `ref readonly` parameters label May 17, 2023
@jjonescz jjonescz added this to the C# 12.0 milestone May 17, 2023
@jjonescz jjonescz force-pushed the RefReadonly-03-OverloadResolution branch from 2f68fda to 8c5ffab Compare May 17, 2023 16:07
@jjonescz jjonescz force-pushed the RefReadonly-03-OverloadResolution branch from 8c5ffab to f6212ba Compare June 16, 2023 12:47
@jjonescz jjonescz changed the title Relax parameter-argument modifier matching Relax parameter-argument modifier matching for ref readonly parameters Jun 19, 2023
@jjonescz jjonescz force-pushed the RefReadonly-03-OverloadResolution branch from f6212ba to 732ccbb Compare June 21, 2023 12:45
@jjonescz jjonescz marked this pull request as ready for review June 21, 2023 13:32
@jjonescz jjonescz requested a review from a team as a code owner June 21, 2023 13:32
@jjonescz
Copy link
Member Author

@jcouv @AlekseyTs for reviews, thanks

@jjonescz jjonescz requested review from AlekseyTs and jcouv June 22, 2023 10:00
object5
c5
""").VerifyDiagnostics(
// (10,58): warning CS9502: Argument 2 should not be passed with the 'ref' keyword
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not be passed with the 'ref' keyword

Consider adjusting the wording so that it explains the effect of using 'ref' keyword and provides guidance how to fix the warning. For example, if compiler "interprets" 'ref' at this call site as 'in', we could say something like: "The 'ref' modifier for an argument corresponding to 'in' parameter is equivalent to 'in'. Consider using it instead." #Closed

""";
var verifier = CompileAndVerify(source, expectedOutput: "555");
verifier.VerifyDiagnostics(
// (7,11): warning CS9503: Argument 1 should be passed with 'ref' or 'in' keyword
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argument 1 should be passed with 'ref' or 'in' keyword

I think we should use wording that explains what is happening. Do we pass a reference to a copy in this case? If so, we should say that, and then say that, in order to avoid the copy, 'in' or 'ref' should be used. #Closed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite possibly I am misunderstood the purpose of the warning and the current wording is just fine.

Copy link
Member Author

@jjonescz jjonescz Jun 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not passing a copy, will add IL verification to demonstrate that. We are passing a reference, just as when using in parameter without any argument modifiers today. But it would be better if users used some modifier to make it clear that a reference is being passed, hence the warning.

@@ -207,6 +207,7 @@ internal static int GetWarningLevel(ErrorCode code)
switch (code)
{
case ErrorCode.WRN_AddressOfInAsync:
case ErrorCode.WRN_BadArgRef: // PROTOTYPE: Document in the markdown file.
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ErrorCode.WRN_BadArgRef

Why not use level 1 for this warning? #Closed

Copy link
Member Author

@jjonescz jjonescz Jun 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it can be reported for pre-existing code... But that would be an errorneous code anyway, so I guess it's fine for it to be level 1. Thanks.

@@ -3213,6 +3213,58 @@ private BoundExpression BindArgumentExpression(BindingDiagnosticBag diagnostics,
}

#nullable enable
private void CheckRefArguments<TMember>(
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like all calls are combined with CoerceArguments calls. Why not incorporate this check into CoerceArguments? We can change it to take AnalyzedArguments, if that is necessary. #Closed

{
if (arg >= parameters.Length)
{
// We can run out of parameters before arguments. For example: `M(__arglist(x))`.
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// We can run out of parameters before arguments. For example: M(__arglist(x)).

Perhaps we should check for that exact situation directly, i.e. skip BoundArgListOperator as an argument. #Closed

@@ -2881,6 +2881,18 @@ A catch() block after a catch (System.Exception e) block can catch non-CLS excep
<data name="ERR_BadArgRef" xml:space="preserve">
<value>Argument {0} must be passed with the '{1}' keyword</value>
</data>
<data name="WRN_BadArgRef" xml:space="preserve">
<value>Argument {0} should not be passed with the '{1}' keyword</value>
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{1}

Why use argument, it looks like we are always using "ref" here? #Closed


// Warn for `ref`/`in` or None/`ref readonly` mismatch.
var argRefKind = analyzedArguments.RefKind(arg);
if (argRefKind is RefKind.Ref or RefKind.None)
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (argRefKind is RefKind.Ref or RefKind.None)

I think the logic can be simpler and easier to follow:

if (argRefKind == RefKind.Ref)
{
    if (GetCorrespondingParameter(ref result, parameters, arg).RefKind == RefKind.In)
    {
                            // Argument {0} should not be passed with the '{1}' keyword
                            diagnostics.Add(
                                ErrorCode.WRN_BadArgRef,
                                analyzedArguments.Argument(arg).Syntax,
                                arg + 1,
                                argRefKind.ToArgumentDisplayString());
    } 
}
else if (argRefKind == RefKind.None && GetCorrespondingParameter(ref result, parameters, arg).RefKind == RefKind.RefReadOnlyParameter)
{
                            // Argument {0} should be passed with 'ref' or 'in' keyword
                            diagnostics.Add(
                                ErrorCode.WRN_ArgExpectedRefOrIn,
                                analyzedArguments.Argument(arg).Syntax,
                                arg + 1);
}
``` #Closed

@@ -2192,7 +2192,7 @@ private static bool RequiredFunctionType<TMember>(MemberResolutionResult<TMember
static bool isAcceptableRefMismatch(RefKind refKind, bool isInterpolatedStringHandlerConversion)
=> refKind switch
{
RefKind.In => true,
RefKind.In or RefKind.RefReadOnlyParameter => true,
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or RefKind.RefReadOnlyParameter

Are we testing effect of this change? #Closed

Copy link
Member Author

@jjonescz jjonescz Jun 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, RefReadonlyParameter_PlainArgument_OverloadResolution fails with "ambiguous call" error without this change. Note that the method doesn't handle only interpolated string handler parameters as its name could suggest; it's called as part of RemoveWorseMembers during overload resolution.

Copy link
Member

@jcouv jcouv Jul 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to this PR: I was thrown off by the name of this method (PreferValOverInOrRefInterpolatedHandlerParameters) but as your test demonstrates no interpolation handlers are needed to get here...
Ah, never mind, I was parsing the name wrong. It's "prefer val over ... ref interpolated handler parameter" :-P

}
}

// PROTOTYPE: Should this relaxation be also applied when `isMethodGroupConversion == true`?
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// PROTOTYPE: Should this relaxation be also applied when isMethodGroupConversion == true?

This is what the speclet says:
For the purpose of delegate/lambda/method group conversions [§10.7, §10.8], ref readonly can be interchanged with in modifier, but that results in a warning. #Closed

Copy link
Member Author

@jjonescz jjonescz Jun 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, method conversions are implemented in a follow up PR. But the relaxations are slightly different and this code is not otherwise touched, so I will remove this prototype comment now.

}

// PROTOTYPE: Should this relaxation be also applied when `isMethodGroupConversion == true`?
if (paramRefKind == RefKind.RefReadOnlyParameter && argRefKind is RefKind.None or RefKind.Ref or RefKind.In)
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if

else if? #Closed

@@ -3180,6 +3180,7 @@ internal EffectiveParameters(ImmutableArray<TypeWithAnnotations> types, Immutabl

private static RefKind GetEffectiveParameterRefKind(
ParameterSymbol parameter,
BoundExpression argument,
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BoundExpression argument

It looks like we don't really need to pass the argument. What we really need to pass is whether IDS_FeatureRefReadonlyParameters is enabled. BTW, language version is available directly from the compilation, therefore, we don't even need argument's syntax to figure out that. #Closed

@@ -1215,9 +1215,9 @@ private static bool HadLambdaConversionError(BindingDiagnosticBag diagnostics, A
}
else if (refArg != refParameter && !(refArg == RefKind.None && refParameter == RefKind.In))
{
if (refParameter == RefKind.None || refParameter == RefKind.In)
if (refParameter is RefKind.None or RefKind.In or RefKind.RefReadOnlyParameter)
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (refParameter is RefKind.None or RefKind.In or RefKind.RefReadOnlyParameter)

Something doesn't feel right here. Some of the new mismatches are allowed with a warning. Not only that, but some are allowed conditionally based on language version. However, if we get here, we end up with an error no matter what. Are we testing this code path for new mismatches? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. The mismatches are not a problem - they are handled by GetEffectiveParameterRefKind, so we don't get here normally. But if there's another problem, like invalid type, overload resolution continues here and wrong error would be reported, need to extend the condition.

@@ -956,7 +956,8 @@ internal static RefKind GetArgumentRefKind(ImmutableArray<BoundExpression> argum
argRefKind = argRefKindsOpt[i];

Debug.Assert(argRefKind == parameters[i].RefKind ||
argRefKind == RefKindExtensions.StrictIn && parameters[i].RefKind == RefKind.In,
(argRefKind == RefKindExtensions.StrictIn && parameters[i].RefKind == RefKind.In) ||
(parameters[i].RefKind == RefKind.RefReadOnlyParameter),
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameters[i].RefKind == RefKind.RefReadOnlyParameter

Let's make this assert stronger and enumerate what values of argRefKind can be combined with RefReadOnlyParameter. #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually seems this assert can be reverted to the original state, since the rewriter (GetEffectiveArgumentRefKinds) ensures that argument ref kind will be the same as parameter ref kind.

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Jun 22, 2023

    internal static RefKind GetArgumentRefKind(ImmutableArray<BoundExpression> arguments, ImmutableArray<ParameterSymbol> parameters, ImmutableArray<RefKind> argRefKindsOpt, int i)

Should EmitArgument be adjusted as well?


In reply to: 1603200812 #Closed


Refers to: src/Compilers/CSharp/Portable/CodeGen/EmitExpression.cs:948 in 732ccbb. [](commit_id = 732ccbb, deletion_comment = False)

/// - StrictIn if was originally passed as In
/// - Ref if the argument is an interpolated string literal subject to an interpolated string handler conversion. No other types
/// are patched here.
/// - In if was originally passed as None or Ref
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In if was originally passed as None or Ref

This doesn't look accurate. It looks like, if an argument is passed as ref, then its effective RefKind is ref, ref readonly, or strict in. The comment looks confusing in general because it doesn't include how parameter RefKind is involved into the process, but it makes a difference. #Closed

else if (paramRefKind == RefKind.RefReadOnlyParameter)
{
fillRefKindsBuilder(argumentRefKindsOpt, parameters, ref refKindsBuilder);
refKindsBuilder[i] = RefKind.RefReadOnlyParameter;
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RefKind.RefReadOnlyParameter

Should we have a "strict" version similar to in? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, will investigate, perhaps add it in a follow up PR. Adding a prototype comment for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my investigation, "strict in" is only used to pass strict readonly address kind here:

var unexpectedTemp = EmitAddress(argument, refKind == RefKindExtensions.StrictIn ? AddressKind.ReadOnlyStrict : AddressKind.Writeable);

which only makes a difference for field referenes in a PEVerify compat mode:

// in readonly situations where ref to a copy is not allowed, consider fields as addressable
if (addressKind == AddressKind.ReadOnlyStrict)
{
return true;
}
// ReadOnly references can always be taken unless we are in peverify compat mode
if (addressKind == AddressKind.ReadOnly && !peVerifyCompatEnabled)
{
return true;
}

Not sure if we need to support that for ref readonly parameters.

I've only looked through references to RefKindExtensions.StrictIn, not sure if I should look elsewhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are reusing RefKind.In and RefKindExtensions.StrictIn now.

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Jun 28, 2023

Implementation changes LGTM (commit 39), I still need to do a pass through tests. @jcouv, I think the PR is in a good state for a second review.

}
}
""";
CreateCompilation(source2, new[] { comp1.ToMetadataReference() }, parseOptions: TestOptions.Regular11).VerifyDiagnostics(
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestOptions.Regular11

Consider adding validation that we do not get any errors for the latest version #Closed

object5
c5
""").VerifyDiagnostics(
// (10,58): warning CS9502: The 'ref' modifier for argument 2 corresponding to 'in' parameter is equivalent to 'in'. Consider using 'in' instead.
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// (10,58): warning CS9502: The 'ref' modifier for argument 2 corresponding to 'in' parameter is equivalent to 'in'. Consider using 'in' instead.

It would be good to verify IL for a scenario like that, to ensure arguments are passed properly. #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already verified in OverloadResolutionTests.PassingArgumentsToInParameters_RefKind_Ref.

""");

verifier.VerifyDiagnostics(
// 0.cs(5,9): warning CS9502: The 'ref' modifier for argument 3 corresponding to 'in' parameter is equivalent to 'in'. Consider using 'in' instead.
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// 0.cs(5,9): warning CS9502: The 'ref' modifier for argument 3 corresponding to 'in' parameter is equivalent to 'in'. Consider using 'in' instead.

This is probably a pre-existing condition, but the warning feels very confusing. There are only two arguments at the call site, but the warning refers to argument 3. #Closed


verifier.VerifyDiagnostics(
// 0.cs(5,9): warning CS9502: The 'ref' modifier for argument 3 corresponding to 'in' parameter is equivalent to 'in'. Consider using 'in' instead.
// C.M(ref x, $"text");
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/ C.M(ref x, $"text");

What is actually going to happen if we follow the suggestion from the warning? Is it going to lead us to an error? Perhaps, we should suppress the new warning that a coming from binding the interpolation in scenarios like that. #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we can't follow the suggestion literally, as there's no argument 3 as you said... The warning refers to the invisible call new CustomHandler(). But yes, if we changed the ref to in, that would be an error. I will try to suppress the warning.

}
}
""";
CreateCompilation(source2, new[] { comp1.ToMetadataReference() }, parseOptions: TestOptions.Regular11).VerifyDiagnostics(
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestOptions.Regular11

Consider testing what happens for the latest version #Closed

@AlekseyTs
Copy link
Contributor

Done with review pass (commit 42)

// Warn for `ref`/`in` or None/`ref readonly` mismatch.
if (argRefKind == RefKind.Ref)
{
if (!interpolatedStringHandler && GetCorrespondingParameter(ref result, parameters, arg).RefKind == RefKind.In)
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interpolatedStringHandler

It feels like warnings reported in the

                        else if (argRefKind == RefKind.None &&
                            GetCorrespondingParameter(ref result, parameters, arg).RefKind == RefKind.RefReadOnlyParameter)

below should be suppressed too. They are either going to create an extra noise, or suggest changes that are likely to lead to a compilation error.
#Closed

BindingDiagnosticBag diagnostics,
BoundExpression? receiver)
BoundExpression? receiver,
bool interpolatedStringHandler = false)
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interpolatedStringHandler

There is an alternative to adding an extra parameter. At the affected call sites, we could use a dedicated BindingDiagnosticBag and filter out warnings by id in the process of merging diagnostics into the original bag. Consider if you prefer doing that instead. #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sounds better than passing a parameter and worrying if we passed it to all callers.

BindingDiagnosticBag diagnostics,
BoundExpression? receiver)
BoundExpression? receiver,
bool interpolatedStringHandler = false)
Copy link
Contributor

@AlekseyTs AlekseyTs Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interpolatedStringHandler

Consider using a name that reflects the purpose of the parameter. For example, "suppressRefReadonlyParameterFeatureWarnings". #Closed

@AlekseyTs
Copy link
Contributor

Done with review pass (commit 44)

@jjonescz
Copy link
Member Author

@jcouv for a second review, thanks

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Jun 30, 2023

            diagnostics.AddRangeAndFree(outConstructorDiagnostics);

addAndFreeConstructorDiagnostics? #Closed


Refers to: src/Compilers/CSharp/Portable/Binder/Binder_InterpolatedString.cs:645 in ed895b4. [](commit_id = ed895b4, deletion_comment = False)

@AlekseyTs
Copy link
Contributor

AlekseyTs commented Jun 30, 2023

Done with review pass (commit 45).

@jjonescz
Copy link
Member Author

jjonescz commented Jul 3, 2023

@jcouv for a second review, thanks

Copy link
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (commit 46)

@jjonescz
Copy link
Member Author

jjonescz commented Jul 7, 2023

@jcouv for a second review, thanks

@jcouv jcouv self-assigned this Jul 7, 2023
@@ -2192,7 +2192,7 @@ private static bool RequiredFunctionType<TMember>(MemberResolutionResult<TMember
static bool isAcceptableRefMismatch(RefKind refKind, bool isInterpolatedStringHandlerConversion)
=> refKind switch
{
RefKind.In => true,
RefKind.In or RefKind.RefReadOnlyParameter => true,
Copy link
Member

@jcouv jcouv Jul 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec: Should the spec mention that we'll have a preference of by-value over ref readonly overload? #Pending

Copy link
Member

@jcouv jcouv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks (iteration 47)

@jjonescz jjonescz merged commit ba7606d into dotnet:features/RefReadonly Jul 10, 2023
@jjonescz jjonescz deleted the RefReadonly-03-OverloadResolution branch July 10, 2023 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Compilers Feature - Ref Readonly Parameters `ref readonly` parameters untriaged Issues and PRs which have not yet been triaged by a lead
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants