proposal: Go 2: require explicit type cast to create "nil receiver" interface

[target-san]

Summary

Go's "fat" interface pointers can contain nil data pointer and proper method table pointer. While this property is useful, it may easily become source of undesired bugs - because typed nil is automatically converted to such "half-nil" interface, even if methods on source type do not support nil receivers. Such implicit conversion must result in nil interface pointer. Although, there should still be a way to create nil-receiver interface explicitly

Motivation

Simplest code snippet which demonstrates problem. Please note that error case is not specifically tied to this problem. Any typed nil to interface conversion may cause this

// Library-local error type
type MyError struct {
    // some data fields
}

func (s *MyError) Error() string {
    // Use s and return string
}
// Library-local function 
func MyOperation() string, *MyError {
    // Either return string result or MyError error by-pinter
}
// Some func which uses library
func DoStuff() SomeComplexResult, error {
    // Bunch of computations
    if result, err := MyOperation; err != nil {
        return nil, err // typed nil to interface happens here
    }
    // Other bunch of computations
}
// Some error reporter
func TopLevel() {
    // ...
    if result, err := DoStuff(); err != nil {
        print(err.Error()) // nil dereference panic happens here
    }
}

Proposal

Change existing behavior such that implicit conversion of typed nil to interface produces nil interface pointer. Allow create "nil receiver interface" pointer using explicit syntax like nil.(MyError*).(error). This would make most expected behavior the default one, with fallback available.

Drawbacks

Changes semantics, which may result in code breakage in some places where code relies on existing conversion rules.

Alternatives

1. Introduce lint
2. Leave as-is

References

Previous proposals which touched this topic:

1. proposal: Go 2: Add typednil keyword for checking whether an interface value is a typed nil. #24635
2. proposal: spec: add kind-specific nil predeclared identifier constants #22729
3. Proposal: use a value other than nil for uninitialized interfaces #21538
4. proposal: Go 2: Ban typed nil interfaces through banning nil method receivers #27890

[gwd]

Just to clarify what you mean, take the following code

func main() {
  var me *MyError // Implements error
  var e error

  e = me  // Here e's type would still be converted to error(nil), rather than *MyError(nil)?
  if e == nil {
    // Under your proposal, this gets executed, and...
  } else {
    // ...this doesn't?
  }
}

This would work for me too. I can't really see a use of keeping track of the type of nil pointer you're carrying around; but I figured someone must want that behavior, or this would have been changed already.

Anyone know of any cases where doing this sort of automatic type conversion would be problematic?

Also, if we did this, and someone wanted the old behavior, how would you specify that?

An advantage of this way is that it silently "fixes" most code to do what people expected anyway.

A disadvantage is that if anyone was relying on the old behavior, this silently breaks it in a way that's 1) nearly impossible to detect programatically, and 2) difficult to restore to functional equivalence.

As I said, I can't immediately see a use for the behavior so the last two points don't bother me.

[zigo101]

@target-san If you implement the (*MyError).Error method like the following shows, then the call print(err.Error()) will not panic.

func (s *MyError) Error() string {
    if s == nil {
    	return "<nil *MyError>"
    }
    // Use s and return string ...
}

[target-san]

@gwd
I raised this same question on Go groups quite some time ago and was told nil receivers are actually useful in cases they don't need state. An example I remember is a dummy Read or Write implementation.

And yes, you're right in your example. Implicit conversion under this proposal will result in nil e. Explicit type cast may be used to obtain typed nil - like nil.(MyError).(error). Proposal actually has this same example. Though syntax may be different.

@go101
Thanks, I know this. But AFAIK not all methods are nil receiver safe even in standard library. My point is, it's very easy to create typed nil accidentally. And normal nil checks cannot spot it.

[zigo101]

But AFAIK not all methods are nil receiver safe even in standard library.

If this exists in a standard package, it must be a bug. Please report it.

[target-san]

@go101 Is it? I may be not that knowledgeable in Go, but most other languages assume "receiver" (this, self etc.) is not nil unless type explicitly specifies otherwise. Could you please point to some recommendation like "always check receivers for nil inside methods"?

[gwd]

@target-san Re nil receivers that don't need state: It seems like you could achieve the same effect by just having dummy state; passing back a type based on an int, for example, rather than a type with a nil pointer.

@go101 The problem with your nil-safe error type is that your check to see whether there's an error or not still thinks there was an error (and presumably will abort doing whatever it was trying to do), when the call actually succeed. Where I recently ran across this was in a web app I'd written, where it did a bunch of checking, then did something like the following:

if display == nil {
  // Render not-found template
} else {
  // Pass 'display' to the appropriate template for rendering
}

I'd changed one of the functions returning a display-compatible structure to return nil when I wanted that thing not visible to this particular user; but instead of showing the "not-found" template, it passed the typed nil pointer to the "display" template, resulting in the user seeing a template error. Nothing crashed, but it still did the wrong thing in a surprising way.

[target-san]

@gwd Yes. This can be achieved by using some stateless static object. And that would be even better solution IMO. Though we have what we have. And what we have is implicit unchecked creation of broken interface pointers.

[zigo101]

@target-san @gwd
sorry, my before conclusion is really wrong.

The correct argument should be if you put a nil pointer in an interface value then call a method of the pointer, then you do it wrongly. This has no differences to calling the method on a the nil pointer directly.

[target-san]

@go101 Unlike known type, you cannot check data pointer for nil reliably without arcane reflection incantations. Even if you do, you cannot be sure it's not a legit "nil receiver" interface without examining exact type and its implementation

[zigo101]

I think if you can't make sure whether or not an interface value is encapsulating a nil pointer, and you know calling a method of the nil pointer will panic, then you should never call the method on the interface value.

How can this situation happen?

[target-san]

That's exactly the problem I am talking about. You can get into this situation with a perfectly correct code. No arcane tricks are required. Just directly pass some result - if that result is a typed nil and destination is an interface which it supports. Check initial post and George's one for examples how you can get such "broken" pointer.

Imagine you can create 0x1 pointer of some type without Unsafe package and even without explicit cast. You check it for nil - it's not. You invoke method on it - it panics.

[gwd]

I think if you can't make sure whether or not an interface value is encapsulating a nil pointer, and you know calling a method of the nil pointer will panic, then you should never call the method on the interface value.

How can this situation happen?

EDITED

This is exactly what this issue is about. Suppose we have package Foo, which includes package Bar, and package Bar includes package Zot:

type ZotError struct {
  error string
}
func (ze *ZotError) Error() string {
  return ze.error
}

func Zot(arg int) (*ZotError) {
   // Do some stuff with arg, everything turns out fine, return success
  return nil
}

func Bar()  error {
  if normal {
    // Do some stuff
    return nil
  } else {
    // Do some stuff for an excepitonal case that's hard to test
    return Zot()
  }
}

func Foo() error {
  // Do thing 1

  // Do thing 2
  if err := Bar(); err == nil {
    // Success 
  } else {
    // Clean up
   return fmt.Errorf("Error doing Bar: %v", err)
  } 

  // Do thing 3
}

In this situation, if the "excepitonal case" in Bar() happens, Bar() will return a value of type *ZotError(nil), and the err != nil check will return false, executing the second block. Foo() will then think that Bar() failed, and try to clean up 'thing 1', even though 'thing 2' actually succeeded. This might be harmless, or it might put the system in an inconsistent state that would cause data corruption down the road.

If Foo(), Bar(), and Zot() are in separate packages, there is no way for the author of Foo() to know that Bar() might return *ZotError(nil) without inspecting the source code.

The only way for Foo() to be sure that err is actually not nil is to do this instead:

func Foo() {
  if err := Bar(); reflect.ValueOf(err).Kind() == reflect.Ptr && reflect.ValueOf(err).IsNil() {
    // Success 
  } else {
    e := err.Error()
    // Do something with the error string, dereference NULL
  } 
}

But none of the guides for how to write good Go code say this; they all recommend err != nil, which is inherently unsafe.

[randall77]

The problem with the code is here:

func Zot(arg int) (*ZotError) {
    return nil
}

It's generally bad practice to return a concrete error type. If you always return error, this issue does not happen. The Go FAQ states:

It's a good idea for functions that return errors always to use the error type in their signature (as we did above) rather than a concrete type such as *MyError, to help guarantee the error is created correctly. As an example, os.Open returns an error even though, if not nil, it's always of concrete type *os.PathError.

I'd be more amenable to building a tool that detects this FAQ item than changing the language.

[target-san]

Errors are just an example. Replace with any optional object which gets cast to interface.

[gwd]

@randall77 Indeed that is bad practice given how nil interfaces work at the moment; but it's very far from obvious why. Humans are very poor at consistently following rules; every "do this for safety" rule we can get rid of will make Go a safer language.

[gwd]

@randall77 Also, this statement isn't quite true:

If you always return error, this issue does not happen.

The following is a pattern that a reasonable developer might use:

// Tries all steps regardless of error; returns an error describing the first issue encountered
func Zot() error {
  var ze *ZotError

 // Do something
 if  error1 && ze != nil {
   ze = &ZotError{error1}
 }

  if error2 && ze != nil {
    ze = &ZotError{error2}
  }
  // and so on

  return ze
}

Here we've followed the advice to return error, but still get something of type *ZotError(nil)

[zigo101]

@gwd
The Zot function in your last comment is not implemented correctly.
You should fix it here, instead of checking if an error value encapsulates a nil pointer later.
You should correct the source, instead of remedying the result.

[target-san]

@go101
Sorry for interfering, but "go fix the source" may be rather hard in real production code. The pointer in question may be passing dozens of stack frames and quite few modules on its way. It may be inserted into some map and checked for nil significantly later. And Shroedinger conversion may happen anywhere on that path.

[gwd]

@go101 You'll have to forgive me if that kind of argument makes me a bit angry; my main job involves writing in C, which is absolutely strewn with hidden landmines of undefined behavior that can make your perfectly normal-looking program suddenly full of security holes; and when you go to the compiler people and tell them that obvious-looking code results in bizarre behavior, they tell you you're just using the language wrong, you should fix your program.

One of the things that made me so happy when I first learned Go was that there was no undefined behavior. All values are initialized by default; no conversions even from int32 to int64 without an explicit cast. The compiler even prevents you from using goto over a variable declaration, just in case something should accidentally end up uninitialized.

Go's behavior with regard to nil pointers and interfaces is a giant black mark on an otherwise very nice language. It's not only dangerous and unnecessary: it's completely out of character.

[zigo101]

There are really some interface related confusions when I started learning Go. However, when I realized that an interface value is just a box to encapsulate non-interface values which satisfy some requirements (have all the methods the interface specifies), I never get confused again.

There are two categories of interfaces, blank ones and non-blank ones. A blank interface doesn't specify any methods, they are mainly used for reflection. A non-blank interface specifies several methods, they are mainly used for polymorphism.

Go's behavior with regard to nil pointers and interfaces is a giant black mark on an otherwise very nice language. It's not only dangerous and unnecessary: it's completely out of character.

I feel your frustration comes from some bad practices in using interfaces. If you want to return a nil pointer as a non-blank interface value (aka, encapsulate a nil value in an interface value), you should expect users of the return interface value will call the method specified by the interface and declared for the pointer type, then you should guarantee that calling the method on the nil pointer will not panic. This is your responsibility. Otherwise, just don't encapsulate the nil pointer in an interface value. I think the principle is simple and clear.

BTW, Go does have some undefined behaviors, though much less than C.

And I don't very understand your proposal.

Change existing behavior such that implicit conversion of typed nil to interface produces nil interface pointer. Allow create "nil receiver interface" pointer using explicit syntax like nil.(MyError*).(error). This would make most expected behavior the default one, with fallback available.

Do you mean nil.(*MyError).(error)? Could you show an example how it is used?

The designs of main Go design elements are very consistent, though there are many inconsistencies in detailed designs and concrete implementations. Nil values are just the zero values of some kinds of types. Except this, they are not special comparing to other non-zero values, so Go doesn't treat them specially.

[target-san]

If you want to return a nil pointer as a non-blank interface value (aka, encapsulate a nil value in an interface value), you should expect users of the return interface value will call the method specified by the interface and declared for the pointer type, then you should guarantee that calling the method on the nil pointer will not panic.

You're effectively saying "expect absolutely any method to be called with nil receiver through typed nil". My point is that interface pointer with valid vptr and nil dptr can be created through implicit cast. Again, I don't see "just write code better" as an argument. Compiler should help us, not force us to decrypt some riddles.

[zigo101]

Honestly, I still don't very understand your proposal. I think I can understand it better if you can show an example how to use the nil.(*MyError).(error) syntax.

Compiler should help us, not force us to decrypt some riddles.

Sometimes, encapsulating a nil pointer in an interface value is desired, sometimes it is not. How can compilers distinguish them?

[target-san]

Honestly, I still don't very understand your proposal. I think I can understand it better if you can show an example how to use the nil.(*MyError).(error) syntax.

Example is very primitive but should illustrate idea

import "io"

type Null interface{}

func (self *Null) Read(p []byte) (n int, err error) {
    return 0, io.EOF
}

func self(*Null) Write(p []byte) (n int, err error) {
    return 0, io.EOF
}

func NewNull() io.ReadWriter {
    return nil.(*Null)(io.ReadWriter)
}

Sometimes, encapsulating a nil pointer in an interface value is desired, sometimes it is not. How can compilers distinguish them?

Cannot disagree with that. My point is, compilers should catch unsound scenarios. And implicit half-nil interface is a soundness hole to me.

[randall77]

The difficulty I see with this proposal is converting back and forth. For instance:

var x *int
var y interface{} = x
z := y.(*int)

Should the third line fail? What about

var x *int
var y interface{} = x
z := y.(*float64)

Currently the first example passes and the second one fails. If the conversion of a nil pointer to an interface makes a full nil interface, then we can't distinguish these two cases - either they both have to fail (a nil interface is convertible to no type, as it is now) or they both have to succeed (a nil interface is convertible to the nil value of every pointer type?).

Either way, I don't like the consequences. The first example seems like the identity operation that should always work, and the second example seems like it could hide type errors (although it wouldn't actually break the type system, because type fluidity only works for nil).

So although I agree your proposal solves the problem you're investigating, it's not without its drawbacks in other areas.

[target-san]

The difficulty I see with this proposal is converting back and forth. For instance:

var x *int
var y interface{} = x
z := y.(*int)

Should the third line fail?

Yes, it should fail. x is a nil. When we upcast it to interface{}, we lose all static type information, i.e. type erasure happens here.

If you want current behavior:

var x *int
var y interface{} = x.(*int).(interface{}) // or some other syntax
z := y.(*int)

What about

var x *int
var y interface{} = x
z := y.(*float64)

Fails as well.

Currently the first example passes and the second one fails. If the conversion of a nil pointer to an interface makes a full nil interface, then we can't distinguish these two cases - either they both have to fail (a nil interface is convertible to no type, as it is now) or they both have to succeed (a nil interface is convertible to the nil value of every pointer type?).

Either way, I don't like the consequences. The first example seems like the identity operation that should always work, and the second example seems like it could hide type errors (although it wouldn't actually break the type system, because type fluidity only works for nil).

So although I agree your proposal solves the problem you're investigating, it's not without its drawbacks in other areas.

Whether I'm right or you is highly dependent on one simple question. What is runtime typed nil in Golang? I mean semantically. What meaning does it carry?

I'll expand a bit on my understanding though will try to keep it short.

First, what is a pointer? It's a special type which holds location of an object of some connected type. And it has special value nil which means "does not point to anything". Although pointer has yet another property called type - which describes what kind of object may reside on the other end - or may not.

And here we have first question. What's the value on the other side when pointer is nil? The answer in most cases is "there's no value". This value is just like any other to pointer itself. But not to operations you perform over that pointer. Specifically, dereference. To me it behaves very similar to division by zero. You can have zero integer, pass it around and do many other things. But you cannot divide by it.

Second question which arises here. What's the type of variable n in n := nil? Go says it's "untyped nil" and cannot be used. It's a special default value literal for any pointer type. So far so good.

Then we come to such concept as interface. Unlike OOP languages, Go's interface is more like trait, a concept more common in FP world. We use interfaces in Go through interface pointers. Which are not the same as normal pointers. In fact, they consist of two pointers, one points to actual object and the other to method table necessary for interface to work properly. Due to Go's runtime, second pointer can be also treated as type information.

If we treat this "fat pointer" as some ordinary type, we can have this rough table of its possible states

Data	Type
nil	nil
nil	non-nil
non-nil	nil
non-nil	non-nil

First and last cases are pretty much obvious. They represent normal nil and normal interface pointer to some object.

Third case doesn't make much sense because it's a data pointer without type pointer. It cannot be created in normal Go unless one uses reflection or unsafe. It's unreachable, so let's leave it as-is.

But there's the last possible state which is nil data pointer with some type information. Remember we concluded nils are typed in Go? So far so good - except nil check tells it's not nil! Moving type information from compile time (concrete object pointer) to runtime (interface pointer) changes whole meaning of the pointer! Which to me is a huge deficiency in type system. Imagine you move some statically sized empty array to slice - and it suddenly starts to return non-zero length!

The problem is, we already have nil receiver interfaces, though used not so often. How are they used and why do they exist? I think they exist to have cheap stateless interfaces like the one I showed above. And they shouldn't be treated as nil.

Who's right is highly dependent here on the semantical meaning of runtime typed nil (RTN) in Go. Based on the answer we have several ways to resolve this strange issue - except leave it as-is of course.

If RTN is just an artifact of cheap stateless interfaces. It would mean that nil is just default value of pointer and does not carry type on its own - but pointer does. Consequently it would mean that nil receiver interfaces are also prohibited state, like nil vtable. And if one wants stateless interfaces, he can use hidden static object.

If RTN is an important part of type system, we must preserve it. But we must also properly express it as nil. So checks against nil receiver will result in "it's empty", with ability to check for its type. Then cheap stateless interfaces must go the same way as above.

A third way is the one described in this proposal. There may be more.

[randall77]

But there's the last possible state which is nil data pointer with some type information. Remember we concluded nils are typed in Go? So far so good - except nil check tells it's not nil!

I think this is the fundamental issue here. nil (and consequently, ==nil) means different things for pointers and interfaces. You're using ==nil on an interface and expecting the behavior of ==nil on the underlying pointers. There's good discussion and even a proposal to distinguish the different nils at #22729 (and I would prefer that proposal to what is proposed here). But the confusion of having syntactically similar constructs behave differently happens in many other places in Go. For instance, +1 means different things for integers and floats. [1] means different things for slices and maps. So ==nil means different things for pointers and interfaces.

I understand that there's confusion here, and that confusion can lead to bugs. But I don't think this proposal is the solution.

Interfaces with nil pointers in them aren't an aberration; they can happen all the time. For example:

package main

import (
	"container/list"
	"fmt"
)

var x, y int

func main() {
	a := []*int{&x, nil, &y}

	// encode into a list
	var l list.List
	for _, p := range a {
		l.PushBack(p)
	}

	// decode from list
	b := []*int{}
	for e := l.Front(); e != nil; e = e.Next() {
		b = append(b, e.Value.(*int))
	}

	fmt.Println(a)
	fmt.Println(b)
}

I'm just using interfaces as a container, and I expect to get back the same thing I put in. I'm not calling methods on it or anything, an interface{} is just a runtime-typed wrapper. Should I have to use l.PushBack(p.(*int).(interface{}) instead? Why do I have to do that for nils but not, say, ints?

[zigo101]

@target-san

You says this proposal is for compiler, but is it also for runtime?

var x *int
if (aRuntimeCondition) {
	x = new(int)
}
// At compile, we don't know whether x is nil.
var y interface{} = x.(*int).(interface{}) // or some other syntax
z := y.(*int) 

BTW, you can't declare methods for *Null, for its base type is an interface type.

[target-san]

@randall77

I think this is the fundamental issue here. nil (and consequently, ==nil) means different things for pointers and interfaces. You're using ==nil on an interface and expecting the behavior of ==nil on the underlying pointers.

Well, kind of yes. Going with this logic, interfaces shouldn't support nil as their value at all. As I described earlier, we in fact have compound entity which tries to mimic itself as ordinary pointer.

There's good discussion and even a proposal to distinguish the different nils at #22729 (and I would prefer that proposal to what is proposed here).

The proposal you mentioned has significant drawback, at least to my sight. It introduces a lot of complexity and different kinds of nils. While most of pointer types mentioned - maps, slices, funcs etc. - don't have such complex nature. They AFAIK either point to some object - or don't point. Their nils are statically typed. The only "black sheep" is interface pointer.

Another issue here would arise when/if generics land on table. We will have to introduce "generic nil" to be able to handle something like generic range loop over some collection of interface pointers.

I understand that there's confusion here, and that confusion can lead to bugs. But I don't think this proposal is the solution.

Interfaces with nil pointers in them aren't an aberration; they can happen all the time.

The example presented shows that runtime typed nils are significant part of type system. I have absolutely no objections against this. Let's just decide - is it nil? Or not a nil? Or should interfaces support nil comparison at all?

My short conclusion. There's a loophole in Go's type system because of attempt to "sit on two chairs", i.e. treat interface as both simple pointer and aggregate data structure at once. I'm leaning towards treating it like simple pointer and thus deciding whether it's nil based only on its data pointer.

The proposal itself happened because of certain doubts that Go core team would change behavior so significantly.

@go101

You says this proposal is for compiler, but is it also for runtime?

I think yes. Though I'm in doubts now due to Keith's feedback on typed nils whether this whole approach is right.

BTW, you can't declare methods for *Null, for its base type is an interface type.

My bad.

[zigo101]

If it is also for runtime, then the syntax is not much useful, for it can't prevent nil interfaces with concrete nil values from happening at run time.

BTW, I think using the terminology typed nil to represent non-interface nil is confusion. An interface nil is also a typed nil, its type is an interface type. Maybe, concrete nil is a better terminology for your typed nil.

[ianlancetaylor]

There's a loophole in Go's type system because of attempt to "sit on two chairs", i.e. treat interface as both simple pointer and aggregate data structure at once.

I want to be clear that in my opinion there is no loophole in Go's type system. I find that to be a perplexing statement.

What is happening here is that the builtin identifier nil is untyped and overloaded. Just as you can compare an int, int8, float64, etc., value to 0, you can compare a pointer, channel, map, slice, interface value to nil. The untyped 0 or nil takes on the type of the value on the other side of the comparison. That is not a loophole. It's the nature of how untyped constants work in Go.

People new to the language find it confusing that a pointer that is equal to nil can be stored in an interface variable and that the interface variable is then not equal to nil. I agree that that is confusing and it would be nice if there were something we could do to fix it. But it's not a loophole.

[target-san]

People new to the language find it confusing that a pointer that is equal to nil can be stored in an interface variable and that the interface variable is then not equal to nil.

The confusion here stems from the nil pointer suddenly becoming non-nil through implicit cast. That's all. Again, what would you suggest to resolve issue mentioned in opening post and this one by @gwd , taking into account this issue is long-range and cannot be always resolved by just "not doing that"?

[zigo101]

If you know the pointer type, assume its is *T, just use type assertion:

if p, _ := display.(*T); p == nil {
  // Render not-found template
} else {
  // Pass 'display' to the appropriate template for rendering
}

[target-san]

And if I don't know it? Or it causes abstraction leak? Or the set of possible types is open?

[zigo101]

If you don't know it, then you can only use its methods (through the interface proxy),
then please ensure calling the methods will not panic, even if their receivers are nil pointers.

[ianlancetaylor]

The confusion here stems from the nil pointer suddenly becoming non-nil through implicit cast.

We are talking about language semantics here, so I think it's important to be pedantic about what we are saying. A "nil pointer" is a value of pointer type that is equal to nil. In Go, any type can be implicitly converted to an interface type. When a nil pointer is implicitly converted to an interface type, it does not "become non-nil". It is still a nil pointer. The confusion that some people encounter is that the value of interface type, although it is equal to a nil pointer of the original pointer type, is not equal to nil. That is because nil is untyped and overloaded.

Again, what would you suggest to resolve issue mentioned in opening post and this one by @gwd , taking into account this issue is long-range and cannot be always resolved by just "not doing that"?

First, let me say that this since this is indeed a long-standing issue that we should take the time to find a good solution. We are not in a hurry here.

Second, I'm already on record as proposing #22729. I don't think that is the answer, but I'm hopeful that it may gesture toward the answer.

[target-san]

When a nil pointer is implicitly converted to an interface type, it does not "become non-nil".

Effectively it becomes :D. Because it no longes compares equal to nil.

First, let me say that this since this is indeed a long-standing issue that we should take the time to find a good solution. We are not in a hurry here.

You may consider making pointers non-nilable, then add "optional value" concept. Ta-da! No need to resolve half-nil values, as nil will always be "outside" the value :D. Mostly joking actually.

Second, I'm already on record as proposing #22729. I don't think that is the answer, but I'm hopeful that it may gesture toward the answer.

Read through the thread more carefully than the first time. I must say I'm more sided with Dave and the others who suggest making nil-comparison independent of runtime type pointer stored in interface. I see that these "runtime typed nils" are already a thing.

Anyway, my proposal was aimed to be "least change" by fixing only ambiguous conversion and leaving everything else as-is.

[ianlancetaylor]

My apologies, but, again, I think it's necessary to be pedantic about language issues.

Effectively it becomes :D. Because it no longes compares equal to nil.

The pointer remains equal to nil. It has been converted to a value of interface type that is not equal to nil. It is not accurate to say that the pointer has become non-nil. The pointer is still nil.

Anyway, my proposal was aimed to be "least change" by fixing only ambiguous conversion and leaving everything else as-is.

The conversion is not ambiguous. It is precisely defined. It is nil itself that could be described as ambiguous, because it is untyped.

[gwd]

The conversion is not ambiguous. It is precisely defined.

I think perhaps @target-san means "confusing" rather than "ambiguous". What happens when you assign a variable with an interface type to a pointer is, as you say, unambiguous and well-defined; but the result is still rather surprising, even to people who theoretically understand what's going on.

[zigo101]

"Confusing" comes from not well understanding what are interface values.
An interface value is just a box used to encapsulate non-interface values.
An interface value boxing nothing is called a nil interface value.
An interface value boxing any value, including nil pointer, does box something, it doesn't box nothing.
An interface value boxing something is absolutely not equal to an interface value boxing nothing.

I feel your confusion comes from not familiar with how type deduction works in Go.

When you comparing a value x with an untyped (aka, bare) nil, the nil will be deduced as the type of x, whether or not x is an interface value. That means:

• if x is an interface value, the bare nil will also be deduced as an interface value. A nil interface value boxes nothing. If x boxes a nil pointer, then it doesn't nothing, so x is not equal to the bare nil.
• if x is an non-interface value, assume its type is T, then the bare nil will also be deduced as a T value. If x is a nil pointer, then it is equal to the bare nil for sure.

The type deduction rules are straightforward and intuitive.

[target-san]

@go101

"Confusing" comes from not well understanding what are interface values.

Thank you very much but I'm well aware what are "fat pointers" and how they are usually implemented. As well as usual vtable implementations in OOP languages. And one of sources of my confusion comes from the fact that in other languages with fat pointers both pieces of such "value" are kept coherent. They're either both null or both point to something. But Go has runtime-typed nils, and here's where things get complicated.

[gwd]

@go101 We have very different attitudes on this.

In my ergonomics class, they said that it's your job as a designer to design things that are easy to use; and that 95% of the time when people misuse your product, it's due to poor design.

In my documentation class, they said that it's your job as a document writer to make the documentation understandable; 95% of the time when people are confused by the documentation it's due to poor design.

In my music conducting class, they said that it's your job as a conductor to communicate when the orchestra should start; 95% of the time when your group doesn't come in together, it's because you as a conductor sent mixed signals. (And he videoed everyone conducting, so that he could prove that it was our fault.)

I could go on and on -- I see this situation in parenting, in economics, in leadership, in school -- situations where people do the "wrong thing" because they were prompted to by the circumstances around them, and then blamed for doing the natural, obvious thing.

Sometimes there's no way to change the design, and education / exhortation to do better is the only answer. But our first resort, when we find this sort of "anti-pattern" happening, should be to see if there's a way we can re-design the system to make it better.

var x InterfaceType
var y *InterfaceInstance
y = nil
x = y
if x == nil {
  // Not executed
} else {
 // Executed
}

The code above is confusing. It is not natural or normal. It can be comprehended, but not without extra effort, and it's prone to error. Rather than spend so much effort trying to make people understand it, and dealing with the consequences of making mistakes, wouldn't it be better to redesign the language so that it doesn't point you in the wrong direction? Or at least, so that the compiler would help you avoid this sort of situation?

I mean, if "just be more careful" were an effective strategy, why bother using a typed language like Go at all?

[zigo101]

It is true that, many new Go programmers, including me when starting learning Go, with experiences of some other popular languages may view nil as the counterpart of null (or NULL) in other languages. It is wrong.

In Go, nil is not a single value, it is just a symbol to represent many many values.

Go is Go, other languages are other languages. Why do you have to let Go become other languages?

But Go has runtime-typed nils, and here's where things get complicated.

No, for every nil, whether it is typed or untyped can be determined at compile time.

@gwd
If you don't plan to adapt the rules of Go, and have to use Go as other languages. I think that is your problem, not of Go.

[zigo101]

@gwd
You last example is wrongly written. It should be written as the following to satisfy your expectation.

var x InterfaceType
var y *InterfaceInstance
y = nil
x = y
if x == (*InterfaceInstance)(nil) {
  // Not executed
} else {
 // Executed
}

otherwise, it is equivalent to

var x InterfaceType
var y *InterfaceInstance
y = nil
x = y
if x == InterfaceType(nil) {
  // Not executed
} else {
 // Executed
}

This is the last attempt to make an explanation in this thread.

[target-san]

@go101
We're perfectly aware how it works and why. What we're trying to say is that current behavior is counter-intuitive and error-prone.

I hit something similar in C++ where certain type had operator bool overload and then could be used as hashtable key - due to stupid bool-to-int implicit conversion which was causing hashes for numbers 0 and 1. Was it logical if you know the rules? Yes. Was it useful and error-proof? Definitely no.

[gwd]

@go101 You keep describing how Go currently works; we're talking about how humans respond to it, and how we can change Go to make them respond better. If I didn't know how Go actually works, then I wouldn't have been able to write that example. If you don't care about making Go more intuitive and safe to use, that's fine; but a lot of people do.

[ianlancetaylor]

Thanks for the suggestion. I don't think we've seen this one before.

A significant drawback of this proposal is that it would subtly change the behavior of existing code. The same Go code would behave differently in different versions of Go. This goes against the plan described in #28221 for language changes. We can add language features, and we can (where necessary) remove language features, but we want to avoid changing language features such that some code is valid in two different language versions but has different behavior.

[target-san]

@ianlancetaylor Well, that's a very, very strong argument against this proposal which I overlooked. I agree that "behaves exactly the same or does not compile at all" is a reasonably good way to introduce changes. I then leave it up to your decision whether to leave this proposal open or close it. Hope you will find good enough way to resolve this issue. And thanks for your patience and overall efforts on Go lang.

[ianlancetaylor]

OK, closing this specific proposal. Perhaps there is something else in this area that could work.