Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposed changes to ECMA 335 for ref field support #63659

Merged
merged 6 commits into from
Feb 16, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/design/specs/Ecma-335-Augments.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This is a list of additions and edits to be made in ECMA-335 specifications. It
- [Static Interface Methods](#static-interface-methods)
- [Covariant Return Types](#covariant-return-types)
- [Unsigned data conversion with overflow detection](#unsigned-data-conversion-with-overflow-detection)
- [Ref field support](#ref-fields)
- [Rules for IL rewriters](#rules-for-il-rewriters)

## Signatures
Expand Down Expand Up @@ -950,6 +951,48 @@ on the top of the stack is reinterpreted as an unsigned value before the convers
Note that integer values of less than 4 bytes are extended to int32 (not native int) on the
evaluation stack.

## Ref Fields
To improve the usefulness of ref structs, support for fields which are defined as ByRefs is needed. Currently their functionality can be approximated by Span<T> fields, but not all types can be converted into Span<T> types simply. In order to support these scenarios, support for generalizing ByRef fields, and converting TypedReference, ArgIterator and RuntimeArgumentHandle into following the normal rules of C# ref structs is desired. With this set of changes, it becomes possible to have ByRef fields of T, but support for pointers to ByRef fields or ByRefs to ByRefs is not added to the ECMA specification.

Changes to the spec. These changes are relative to the 6th edition (June 2012) of the ECMA-335 specification published by ECMA available at:

https://www.ecma-international.org/publications-and-standards/standards/ecma-335/

### I.8.2.1.1 Managed pointers and related types
- First paragraph. Replace “Managed pointer types are only allowed for local variable (§I.8.6.1.3) and parameter signatures (§I.8.6.1.4); they cannot be used for field signatures (§I.8.6.1.2), as the element type of an array (§I.8.9.1), and boxing a value of managed pointer type is disallowed (§I.8.2.4).” with “Managed pointer types are only allowed for local variable (§I.8.6.1.3), parameter signatures (§I.8.6.1.4), and instance fields of byref-like types; they cannot be used for field signatures(§I.8.6.1.2) of static fields, or of instance fields of types which are not byref-like, as the element type of an array (§I.8.9.1), and boxing a value of managed pointer type is disallowed (§I.8.2.4).”
- Add a new paragraph before the paragraph on the three special types. “Byref-like types are value types which may contain managed pointers, or pointers onto the VES stack. Byref-like types have the same restrictions as byrefs. Value types which are marked with the System.Runtime.CompilerServices.IsByRefLikeAttribute attribute are considered to be byref-like types.”

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean only a System.Runtime.CompilerServices.IsByRefLikeAttribute type defined in the runtime, or also identically named types in user assemblies? IIRC, the spec does not require the runtime to recognize any other types just by name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KalleOlaviNiemitalo We no longer attempt to update Partition IV of the ECMA specification, so we cannot refer to that as part of this set of updates, and the CoreCLR/Mono runtimes are likely to allow attributes defined in any module to satisfy this requirement, but I do not have any intention of mandating in the spec support for attributes by name only. (Parsing of attributes by name will be an implementation detail of the CoreCLR/Mono implementations). If you have suggestions for appropriate wording to clarify that, I'd appreciate it.

- Replace “Typed references have the same restrictions as byrefs.” With “Typed references are byref-like types and have the same restrictions as normal byref-like types.”
- Replace “They can be used for local variable and parameter signatures. The use of these types for fields, method return types, the element type of an array, or in boxing is not verifiable (§I.8.8). These two types are referred to as byref-like types.” With “If a function uses these types only for local variable and parameter signatures, then the use of those types is verifiable. Otherwise these types have the same restrictions as normal byref-like types.”

### I.8.6.1.1
- Add to the first paragraph a modified version of the third paragraph of I.8.6.1.3. "The typed reference ~~local variable signature~~ type signature states that the type is a structure which will contain both a managed pointer to a location and a runtime representation of the type that can be stored at that location. A typed reference signature is similar to a byref constraint(§I.8.6.1.2), but while the byref specifies the type as part of the byref constraint (and hence statically as part of the type description), a typed reference provides the type information dynamically."
- Add a second paragraph which is a slight modification of the former fourth paragraph in section I.8.6.1.3. "The typed reference signature is actually represented as a built-in value type, like the integer and floating-point types. In the Base Class Library (see Partition IV Library) the type is known as System.TypedReference and in the assembly language used in Partition II it is designated by the keyword typedref. This type shall only be used for parameters, the type of a field, return values and local variables. It shall not be boxed, nor shall it be used as the type of an element of an array.
- Move CLS Rule 14 to this section.

### I.8.6.1.2
- Move the contents of the byref constraint paragraph from Section I.8.6.1.3 to a bullet point.
“- The byref constraint states that the content of the corresponding location is a managed pointer. A managed pointer can point to a local variable, parameter, field of a compound type, or element of an array. However, when a call crosses a remoting boundary (see §I.12.5) a conforming implementation can use a copy-in/copy-out mechanism instead of a managed pointer. Thus programs shall not rely on the aliasing behavior of true pointers. A managed pointer cannot point to another managed pointer, but a managed pointer can point to a byref-like type.”
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A managed pointer cannot point to another managed pointer,

This is effectively saying that ref ref int is not supported at an IL level correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is also how I read this statement.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider that once we have ref fields then we can effectively have ref ref by having ref Span<T> or really any ref SomeTypeWithRefFields. Hence wanted to understand a bit more why ref ref is disallowed because it seems like it's effectively allowed with a bit of indirection.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW: I don't have any particular use for ref ref and no intention to add it to the language. But do want to understand why it's disallowed at the IL level when it seems like you can achieve it a slight amount of indirection.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ECMA-335 still partially describes remoting, although the .NET Runtime does not support remoting. I would prefer that the ECMA-335 augments be consistent with the remoting clauses until those clauses are deleted.

I.8.2.1.1 and I.8.6.1.3 mention a copy-in/copy-out mechanism for remoting of managed pointers. It is not clear what this should mean for ref ref int; just copying the bits of the managed pointer in and out seems too risky. So if ref ref were allowed, then perhaps the spec should explicitly say that parameters with such types cannot be used across a remoting boundary.

OTOH, the spec does not say that the serializable attribute affects remoting, nor that e.g. RuntimeArgumentHandle cannot be passed over a remoting boundary. Which means the implementation can choose which types it supports in remoting and can exclude ref ref types even without having the spec spell it out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you could only declare it indirectly via a structure. There are a bunch of weird and unfortunate costs to adding the ability to take a ref of a ref, or a pointer to a ref.

  • Mono would need to restructure its representation of type
  • We would need to update the type name generation/parsing for custom attribute types
  • A bunch of little implementation details scattered all throughout the CoreCLR runtime would need updates.
  • We would need new syntax in C#, or there may be a whole set of unspeakable types that can only appears as a result of generics or something
  • @KalleOlaviNiemitalo also describes that remoting is a thing that would be impacted. At this time, that is not a major concern, but he is unquestionably correct.

Given that, I'd like to make the first effort to implement this simply not allow that more complex feature to exist. I don't currently see a need for it in the motivation for this feature. Also, given that is just a strict restriction on behavior, if it turns out to be a capability we want to add in the future, we can do so.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono would need to restructure its representation of type

This isn't a total dealbreaker (I prototyped it during the hackathon) but it will significantly increase implementation (really... testing and validation) cost for Mono in the .NET 7 timeframe.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that, I'd like to make the first effort to implement this simply not allow that more complex feature to exist. I don't currently see a need for it in the motivation for this feature.

That makes sense. I have no known immediate uses for it. Mostly I just wanted to understand why it wasn't directly allowed when it seems to exist perfectly fine with a small amount of indirection. The rest of your comment hit on that.


### I.8.6.1.3
- Edit the second sentence of the first paragraph. "A local signature specifies the contract on a local variable allocated during the running of a method. A local signature contains a full location signature~~, plus it can specify one additional constraint:~~."
- Remove the rest of this section. Its contents have been moved into sections I.8.6.1.1 and I.8.6.1.2

### I.8.7
- In the section about type compatibility when determining a type from a signature: The byref constraint is to be referenced from section I8.6.1.2 insetad of I.8.6.1.3

### I.8.9.2
- Insert at the end of the first paragraph “An unmanaged pointer type cannot point to a managed pointer.”

Changes to signatures:
### II.23.2.10
- Remove special case for TYPEDBYREF

### II.23.2.11
- Remove special case for TYPEDBYREF

### II.23.2.12
AaronRobinsonMSFT marked this conversation as resolved.
Show resolved Hide resolved
- Add TYPEDBYREF as a form of Type

## Rules for IL Rewriters

There are apis such as `System.Runtime.CompilerServices.RuntimeHelpers.CreateSpan<T>(...)` which require that the PE file have a particular structure. In particular, that api requires that the associated RVA of a FieldDef which is used to create a span must be naturally aligned over the data type that `CreateSpan` is instantiated over. There are 2 major concerns.
Expand Down