Covariant Return Types - Draft Specification

[gafter]

Abstract

This is a draft proposed specification for covariant return types in C#. Our intent is to permit the override of a method to return a more derived return type than the method it overrides, and similarly to permit the override of a read-only property to return a more derived return type. Callers of the method or property would statically receive the more refined return type from an invocation, and overrides appearing in more derived types would be required to provide a return type at least as specific as that appearing in overrides in its base types.

This is a first draft, so it was necessarily invented from scratch. Many of the ideas introduced are tentative and may be revised or eliminated in future revisions.

---

Class Method Override

The existing constraint on class override methods

• The override method and the overridden base method have the same return type.

is modified to

• The override method must have have a return type that is convertible by an identity or implicit reference conversion to the return type of the overridden base method.

And the following additional requirements are appended to that list:

• The override method must have have a return type that is convertible by an identity or implicit reference conversion to the return type of every override of the overridden base method that is declared in a (direct or indirect) base type of the override method.
• The override method's return type must be at least as accessible as the override method (Accessibility domains).

This constraint permits an override method in a private class to have a private return type. However it requires a public override method in a public type to have a public return type.

Class Property and Indexer Override

The existing constraint on class override properties

An overriding property declaration shall specify the exact same accessibility modifiers and name as the inherited property, and there shall be an identity conversion between the type of the overriding and the inherited property. If the inherited property has only a single accessor (i.e., if the inherited property is read-only or write-only), the overriding property shall include only that accessor. If the inherited property includes both accessors (i.e., if the inherited property is read-write), the overriding property can include either a single accessor or both accessors.

is modified to

An overriding property declaration shall specify the exact same accessibility modifiers and name as the inherited property, and there shall be an identity conversion or (if the inherited property is read-only) implicit reference conversion from the type of the overriding property to the type of the inherited property. If the inherited property has only a single accessor (i.e., if the inherited property is read-only or write-only), the overriding property shall include only that accessor. If the inherited property includes both accessors (i.e., if the inherited property is read-write), the overriding property can include either a single accessor or both accessors. The overriding property's type must be at least as accessible as the overriding property (Accessibility domains).

Interface Method, Property, and Indexer Override

Adding to the kinds of members that are permitted in an interface with the addition of the DIM feature in C# 8.0, we further add support for override members along with covariant returns. These follow the rules of override members as specified for classes, with the following differences:

The following text in classes:

The method overridden by an override declaration is known as the overridden base method. For an override method M declared in a class C, the overridden base method is determined by examining each base class of C, starting with the direct base class of C and continuing with each successive direct base class, until in a given base class type at least one accessible method is located which has the same signature as M after substitution of type arguments.

is given the corresponding specification for interfaces:

The method overridden by an override declaration is known as the overridden base method. For an override method M declared in an interface I, the overridden base method is determined by examining each direct or indirect base interface of I, collecting the set of interfaces declaring an accessible method which has the same signature as M after substitution of type arguments. If this set of interfaces has a most derived type, to which there is an identity or implicit reference conversion from every type in this set, and that type contains a unique such method declaration, then that is the overridden base method.

We similarly permit override properties and indexers in interfaces as specified for classes in 15.7.6 Virtual, sealed, override, and abstract accessors.

Name Lookup

Name lookup in the presence of class override declarations currently modify the result of name lookup by imposing on the found member details from the most derived override declaration in the class hierarchy starting from the type of the identifier's qualifier (or this when there is no qualifier). For example, in 12.6.2.2 Corresponding parameters we have

For virtual methods and indexers defined in classes, the parameter list is picked from the first declaration or override of the function member found when starting with the static type of the receiver, and searching through its base classes.

to this we add

For virtual methods and indexers defined in interfaces, the parameter list is picked from the declaration or override of the function member found in the most derived type among those types containing the declaration of override of the function member. It is a compile-time error if no unique such type exists.

For the result type of a property or indexer access, the existing text

• If I identifies an instance property, then the result is a property access with an associated instance expression of E and an associated type that is the type of the property. If T is a class type, the associated type is picked from the first declaration or override of the property found when starting with T, and searching through its base classes.

is augmented with

If T is an interface type, the associated type is picked from the declaration or override of the property found in the most derived of T or its direct or indirect base interfaces. It is a compile-time error if no unique such type exists.

A similar change should be made in 12.7.7.3 Indexer access

In 12.7.6 Invocation expressions we augment the existing text

• Otherwise, the result is a value, with an associated type of the return type of the method or delegate. If the invocation is of an instance method, and the receiver is of a class type T, the associated type is picked from the first declaration or override of the method found when starting with T and searching through its base classes.

with

If the invocation is of an instance method, and the receiver is of an interface type T, the associated type is picked from the declaration or override of the method found in the most derived interface from among T and its direct and indirect base interfaces. It is a compile-time error if no unique such type exists.

Implicit Interface Implementations

This section of the specification

For purposes of interface mapping, a class member A matches an interface member B when:

• A and B are methods, and the name, type, and formal parameter lists of A and B are identical.
• A and B are properties, the name and type of A and B are identical, and A has the same accessors as B (A is permitted to have additional accessors if it is not an explicit interface member implementation).
• A and B are events, and the name and type of A and B are identical.
• A and B are indexers, the type and formal parameter lists of A and B are identical, and A has the same accessors as B (A is permitted to have additional accessors if it is not an explicit interface member implementation).

is modified as follows:

For purposes of interface mapping, a class member A matches an interface member B when:

• A and B are methods, and the name and formal parameter lists of A and B are identical, and the return type of A is convertible to the return type of B via an identity of implicit reference convertion to the return type of B.
• A and B are properties, the name of A and B are identical, A has the same accessors as B (A is permitted to have additional accessors if it is not an explicit interface member implementation), and the type of A is convertible to the return type of B via an identity conversion or, if A is a readonly property, an implicit reference conversion.
• A and B are events, and the name and type of A and B are identical.
• A and B are indexers, the formal parameter lists of A and B are identical, A has the same accessors as B (A is permitted to have additional accessors if it is not an explicit interface member implementation), and the type of A is convertible to the return type of B via an identity conversion or, if A is a readonly indexer, an implicit reference conversion.

This is technically a breaking change, as the program below prints "C1.M" today, but would print "C2.M" under the proposed revision.

using System;

interface I1 { object M(); }
class C1 : I1 { public object M() { return "C1.M"; } }
class C2 : C1, I1 { public new string M() { return "C2.M"; } }
class Program
{
    static void Main()
    {
        I1 i = new C2();
        Console.WriteLine(i.M());
    }
}

Due to this breaking change, we might consider not supporting covariant return types on implicit implementations.

Constraints on Interface Implementation

We will need a rule that an explicit interface implementation must declare a return type no less derived than the return type declared in any override in its base interfaces.

API Compatibility Implications

TBD

Open Issues

If we have the following interfaces:

interface I1 { I1 M(); }
interface I2 { I2 M(); }
interface I3: I1, I2 { override I3 M(); }

Note that in I3, the methods I1.M() and I2.M() have been “merged”. When implementing I3, it is necessary to implement them both together.

Generally, we require an explicit implementation to refer to the original method. The question is, in a class

class C : I1, I2, I3
{
    C IN.M();
}

What does that mean here? What should N be?

I suggest that we permit implementing either I1.M or I2.M (but not both), and treat that as an implementation of both.

LDM notes:

• https://github.com/dotnet/csharplang/blob/master/meetings/2020/LDM-2020-01-08.md#covariant-returns

[alrz]

we might consider not supporting covariant return types on implicit implementations.

That's about 80% of use cases I had in mind :( -- there's at least two kind of usages:

(1) when you want to override members with more specific types
(2) when you want to inherit (from interfaces) with less specific types

I think the latter usage is as important as the first one.

Could we say we don't consider shadowing members for implicit implementation with covariant return types (probably regardless of the presence of the new modifier)?

[gafter]

(2) when you want to inherit (from interfaces) with less specific types

I don't know what this means. Members from interfaces are not inherited into classes, so you must be talking about inheriting from interfaces into interfaces. Can you give an example?

[alrz]

I think this would be considered an "implicit implementation"?

class Base {
  public string M() => null;
}

interface Interface { 
  object M();
}

// implicitly implements `Interface.M` with covariant return type 
class Derived : Base, Interface {} 

object is less specific, hiding the return type from base.

[gafter]

@alrz In your example, nothing from the interface is inherited into the class, so nothing would be hidden. Because it would be a breaking change to consider Base.M to be an implementation of Interface.M (e.g. if the latter has a default implementation), I expect we would not make that work, and you would have to write

class Base {
  public string M() => null;
}

interface Interface { 
  object M();
}

class Derived : Base, Interface
{
    object Interface.M() => base.M();
}

[theunrepentantgeek]

Nitpick - I think you have a couple of typos in the section on interface mapping:

For purposes of interface mapping, a class member A matches an interface member B when:

A and B are methods, and the name and formal parameter lists of A and B are identical, and the return type of A is convertible to the return type of B via an identity of or implicit reference convertion conversion to the return type of B.

[theunrepentantgeek]

I can see a particular use for this when implementing a hierarchy of immutable types - at the moment, mutating methods (that return a near-clone of the original) declared on a base class cause a bit of grief to subclasses.

It would be nice, for example, when implementing immutable class Role and it's subclass StudentRole to be able to write:

public class Role { 
    // ...
    public virtual Role FinishingAt(DateTimeOffset instant) { ... }
    // ...
}

public class StudentRole {
    // ...
    public override StudentRole FinishingAt(DateTimeOffset instant) { ... }
    // ...
}

At the moment (C#8), the only way to achieve this result is a bit messy (the implementation of Role.FinishingAt() delegates to an internal protected method, say FinishingAtCore() that gets overridden on StudentRole, and the declaration of StudentRole.FinishingAt() needs to be declared to new).

Getting rid of all that boilerplate (which is a fertile ground for bugs) would be useful.

[alrz]

@gafter

it would be a breaking change to consider Base.M to be an implementation of Interface.M (e.g. if the latter has a default implementation)

The same program in Java calls Base.M regardless of the presence of a default impl,

class Base {
    public String M() { System.out.println("Base.M"); return null; }
}
interface Interface {
    default Object M() { System.out.println("Interface.M"); return null; }
    // or Object M(); 
}
class Derived extends Base implements Interface {}
public class Main {
    public static void main(String[] args) {
        Interface x = new Derived();
        x.M(); // prints "Base.M"
    }
}

So the implicit implementation takes precedence over default implementation.

It's shame that the fact that in C# default impl is introduced before covariant return types, is limiting the feature.

One solution is that in C# we define a lower precedence for covariant return types (vs identical return types or default implementations) when we're looking for an implicit implementation. so that both programs would call Interface.M() and therefore avoiding the breaking change.

Is that feasible in practice?

[gafter]

One solution is that in C# we define a lower precedence for covariant return types (vs identical return types or default implementations) when we're looking for an implicit implementation. so that both programs would call Interface.M() and therefore avoiding the breaking change.

Is that feasible in practice?

I don't understand the suggestion well enough to comment.

[theunrepentantgeek]

This is technically a breaking change, as the program below prints "C1.M" today, but would print "C2.M" under the proposed revision.

I know that I've written code where this would happen - but I'm wondering how much of a breaking change this would be ...

... at least in the examples I remember, the purpose of M() was the same in both C1 and C2, so this wouldn't actually have broken anything.

Is there any data to compare "number of places where a different method would be called" vs "number of places where behaviour would change"?

My intuition is that this change would fall into the same category as the change to how lambdas captured loop variables: technically breaking, but effectively benign. But, as I said, that's just my (limited) intuition.

[agocke]

Due to this breaking change, we might consider not supporting covariant return types on implicit implementations.

Avoiding implicit implementations entirely doesn't seem blocking, but it would be nice if it could work in basic cases.

Could we change the search for mapping members to consider implicit implementations with different covariant returns iff there is no other implementation (including a default one)? That would be an error today, as the interface would not be properly implemented.

[alrz]

Could we change the search for mapping members to consider implicit implementations with different covariant returns iff there is no other implementation (including a default one)?

@gafter, perhaps @agocke articulated it better. Afterall, the existence of such member is the only factor that makes it a breaking change.

However, it's not like how member lookup works today wrt to closeness: we stop if a candidate set has a winner in a closer scope, but if we want to account for covariant return types only after we didn't find an implementation under current rules, we may need to run a second lookup considering the ones with covariant return types.

[gafter]

@agocke I don’t see how that would work in binary compatibility scenarios. If a new version of the interface is released with a default implementation then the runtime would change to use that one instead of the implementation from the base class?

[agocke]

Good point. I can't see how this could work for default implementations.

[gafter]

Prioritization of runtime support for covariant return types is being tracked internally at https://devdiv.visualstudio.com/DevDiv/_queries/edit/1009909

[gafter]

Added

Open Issues

If we have the following interfaces:

interface I1 { I1 M(); }
interface I2 { I2 M(); }
interface I3: I1, I2 { override I3 M(); }

Note that in I3, the methods I1.M() and I2.M() have been “merged”. When implementing I3, it is necessary to implement them both together.

Generally, we require an explicit implementation to refer to the original method. The question is, in a class

class C : I1, I2, I3
{
    C IN.M();
}

What does that mean here? What should N be?

I suggest that we permit implementing either I1.M or I2.M (but not both), and treat that as an implementation of both.

[gafter]

I am planning to check this spec in under #3394 and then close this issue.

[gafter]

Spec is now at https://github.com/dotnet/csharplang/blob/master/proposals/covariant-returns.md

[daveyostcom]

Spec is actually now at https://github.com/dotnet/csharplang/blob/master/proposals/csharp-9.0/covariant-returns.md
This feature will allow me to say what I mean in a certain part of my code. Can't wait.

[TheBoxyBear]

So it's also possible to use with get only properties, but what would prevent this scenario from being possible?

Class A defines an abstract get only property. Class B overrides the property using a covariant type and adds a setter to it. When the property is referenced through an instance defined as A, it is get only and if the instance is defined as B, it can be set but the value must be of a type compatible with the type defined by B.