Forward type declarations

[HertzDevil]

"Forward" type declarations in Crystal, i.e. defining empty types in one place and then reopening them later, can be used to declare the type hierarchy prior to def definitions, so that the order between overloads is properly computed (see #4897 and #10518), and they might also be useful down the road for incremental compilation. But since Crystal makes no distinction between a type declaration and a type definition, there are two shortcomings to this approach:

• If a type has an abstract def, that def must be defined, so the type declaration ceases to be a declaration. On the other hand, if the type has no abstract defs, an empty "declaration" becomes a complete type definition on its own.
• The above prevents the ability to only require parts of certain libraries, e.g. all the Big* types must be defined even when only one of them is needed, because all of those types include Comparable.

I propose the addition of a @[Declaration] annotation or an equivalent keyword. The forward declarations for the Big* types might look like below:

# src/big/decls.cr

@[Declaration]
struct BigInt < Int
  include Comparable(Int::Signed)
  include Comparable(Int::Unsigned)
  include Comparable(BigInt)
  include Comparable(Float)
end

@[Declaration]
struct BigFloat < Float
  include Comparable(Int)
  include Comparable(BigFloat)
  include Comparable(Float)
end

@[Declaration]
struct BigRational < Number
  include Comparable(BigRational)
  include Comparable(Int)
  include Comparable(Float)
end

@[Declaration]
struct BigDecimal < Number
  include Comparable(Int)
  include Comparable(Float)
  include Comparable(BigRational)
  include Comparable(BigDecimal)
end

# src/big/big_int.cr
# no extra inheritance or includes needed here, although they wouldn't hurt
# similar goes for the other big_*.cr

require "./decls"

# `BigInt`'s summary
struct BigInt
  # ...
end

# src/big.cr

require "./big/decls"
# remaining `require` statements

The presence of @[Declaration] distinguishes forward type declarations from type definitions. The semantics would be as follows:

• A declaration may only contain superclasses, includes, extends, visibility modifiers, generic type parameters, abstract, and other nested declarations, nothing else is allowed. In particular, instance / class variables and abstract defs are not declarations.
  • Forward declarations of enum types cannot define any enum values or use @[Flags].
  • Forward declarations of lib types cannot define anything at all.
  • Forward declarations of annotation types cannot define anything (their definitions can't either, for that matter).
• @[Declaration] applies recursively to types within the applied type's scope.
• There would be 2 passes of the top-level visitor; the first handles declarations, the second handles non-declarations plus macros. Thus types marked with @[Declaration] are always visible in macros and no intermediate state can be observed:

  # the following line is executed *after* `M` is declared
  {% p M %} # => M
  
  @[Declaration]
  module M
    class Foo
    end
  
    # `M`, `M::Foo`, and `M::Bar` are all forward declarations
    # the following line is executed *after* these types are declared
    {% M.constants %} # => [Foo, Bar]
  
    class Bar
    end
  end

  Since macros are not expanded for declarations, it is impossible to declare a type like class {{ "T#{`date '+%s'`}".id }}, but types like record Foo can still be declared (a record is simply a struct node and it absolutely can represent a reopened type).
• Reopening a forward declared type without @[Declaration] would count as a definition; things that must apply only to a type's first definition could go there as usual, e.g. includes as long as the client ensures no overloads are broken, and enum values.
• Forward declaring the same type more than once is allowed. This is needed in some cases to break cyclic dependencies:

  @[Declaration]
  class Foo
  end
  
  @[Declaration]
  class Bar
    include Comparable(Foo)
  end
  
  @[Declaration]
  class Foo
    include Comparable(Bar)
  end

• Since a forward declaration is not a definition, any attempts to use that type (or its metaclass) without a definition is an error:

  @[Declaration]
  class Foo
  end
  
  Foo.new # Error: 'Foo' is declared but not defined
  
  def foo(x : Foo) # okay, `Foo`'s definition is not used here
  end

  typeof(T), sizeof(T), instance_sizeof(T), offsetof(T, ...), etc. are also uses of type T's definition.
• If a type is declared but never defined, its abstract defs need not be implemented at all:

  abstract module Enumerable(T)
    abstract def each(& : T -> _) # definition
  end
  
  @[Declaration]
  abstract module Iterable(T)
    # a declaration would not have abstract defs in the first place
  end
  
  @[Declaration]
  class MyArray(T)
    include Enumerable(T) # okay
    include Iterable(T)   # okay
  end

It is undecided whether docstrings and annotations other than @[Declaration] are allowed on forward declarations.

For very large libraries like the entire standard library, it would be cumbersome to write out all forward declarations by hand, but we shouldn't need to; crystal tool hierarchy could have a new "decls" format, and we get all the declarations by running it on src/**/*.cr (or src/docs_main.cr for the standard library).

[HertzDevil]

The hypothetical decls format is essentially a variant of #3457, but user projects additionally need #3451 so that standard library types can be excluded automatically.

[stellarpower]

Related to #11764, in languages that support them, I almost always like to define my methods outside of the class definition.

For me, it makes the code much more readable - the class doesn't span pages and pages - and the class definition then serves as an important piece of documentation. I can read through the methods in the class over about one page (and if it's more than that, I probably haven't broken it up enough to be modular) and see how I am supposed to use it, and what is important and what's not. It's also maintaining the concept of interface-implementation abstraction. The class body defines the interface, the method implementation I don't need to care about most of the time. Especially when two spaces of indentation is customary, I find it very hard to get to the jist of what a class does when the methods are defined in full within the class body.

Because Crystal allows re-opening a class, I came up with a trivial macro I could use to define methods outside of the class. However, whilst this makes the code much more readable, I then lose my declarations within the class. These could of course be made as empty within the class and overwritten, but unless we add some more magic into the macro, to check if the method already exists in some form, this of course would be dangerous - because if I change the implementation's signature, the compiler will probably now pick the empty no-op overload and not the one I want, and that's potentially a subtle bug to fix. I use something similar in Python for the same reason - the class just has methods with pass, and further down in the file I write what I want them to do, but obviously python doesn't support overloads and everything is runtime, so this can't go wrong in the same way, I just have to run it a million times and fix everything when it crops up at the latest possible opportunity 🙃

So I think this feature would be useful in another case here as well. May not suit everyone, but I'd definitely make use of the ability to forward-declare a method so I can then define it later on in this or another file. I've also heard of dependency injection form things like C#, and, I don't know the details, so no idea if it'd be a good or a bad thing to consider in Crystal, but it might be possible to attach an implementations to a class or module at runtime or at least bind it late in the compilation process using something like this.