proposal: Go 2: chaining method calls with return type (T, error)

[songmelted]

There are many instances where chain calling methods yields code that is more readable.

For instance, a series of math operations on a custom type.

if myNewVar, err := myVar.Add(5).Multiply(2).Pow(4); err != nil {
    //handle error
}

//also allow 
//myNewVar := myVar.Add(5).Multiply(2).Pow(4)

Is far more readable than calling and error checking each method one at a time. The boilerplate error checking inhibits readability and debugging. Compare the above with the following:

myNewVar := new(myType)
var err error

if myNewVar, err = myVar.Add(5); err != nil {
    //handle error
}

if myNewVar, err = myNewVar.Multiply(2); err != nil {
    //handle error
}

if myNewVar, err = myNewVar.Pow(4); err != nil {
    //handle error
}

If there more than a few methods called with extensive error handling, it is easy to lose sight on what exactly is being accomplished.

I personally experienced this when using a library. I was performing serial transformations on a type. I had erroneously called two methods out of order. I could not find the bug until I rewrote the library to allow chain calls (by not returning errors).

This allowed me to greatly improve the readability of my code. What had taken my many hours to write incorrectly and attempt to debug, was now written correctly, the first time using only a half hour. Not only that, but I found and fixed a bug in the imported code as I rewrote the library with chain calling to better see what it was doing internally.

However, I lost my error handling along the way.

Here is a way of handling the issue using golang as is:

https://stackoverflow.com/a/50601526/11012871

It makes the error part of the struct and checks for existing errors at the beginning of each method.

I would like to see something like that baked into golang, so I can use external libraries this way without writing wrappers or otherwise modifying them.

My suggestion would be to allow a method call on a function which returns (t *myType, err error) by calling the function on "t" if the error is nil. If the error returned is not nil, stop the chain and return as is.

I would also require the that the methods in the chain have only two values returned (t *myType, err error).

I would also require that the chained methods must return the same type as the first value. That is, all methods must return the same type (t *myType, err error). (I think it would get sloppy fast if we allowed a method which returns (i *int, err error) to chain call of off (s *string, err error). But I am open to hearing good ways to handle this.)

Then you could have something like this:

type myType struct{
    \\...
}

func (t *myType) Method1(input int) (r *myType, err error){\\must return two values with second being error to allow chain calling
    \\sets r and err
    return 
}

func (t *myType) Method2(input int) (r *myType, err error){\\must return two values with second being error to allow chain calling
    \\sets r and err
    return 
}

func main() {
    \\initialize t of type *myType
    if result, err := t.Method1(5).Method2(23); err != nil {
        \\handle error
    }
    \\...
}

Method2 operates on the *myType returned by Method1.
If Method1 returns a non-nil error, the whole chain is stopped and its return value is returned.
Method2 is only called if Method1 has a nil error.

This would allow for more readable and easier to write code.

Obviously, I can write my own types as suggested in the stack overflow post. I can also write wrappers.

This, however, would allow me to use imported types this way without needless boilerplate.

Of course, I am open to other suggestions on how to allow chain calling of methods with errors.

To summarize:

If all the methods have the same two return values (t *myType, err error) then allow

myNewVar, err := myVar.Method1().Method2()

If Method1 returns a non-nil error, Method2 is not called and myNewVar and err are set to Method1's return values.

If Method1 returns a nil error, Method2 is called on the *myType returned by Method1 and myNewVar and err are set to Method2's return values.

[DeedleFake]

I think the idea is potentially valid, though I'm not sure about the specifics of this proposal.

Thinking out loud here: The issue that a lot of people had with the old check and handle proposal was that it hid things and made fine-grained error handling feel more awkward, even if you could just resort to the current way of doing it for cases where you needed it. What if an error handling proposal didn't deal at all with handling errors, and only with checking them? For example, as this proposal says, you can't chain method calls if any of the methods in the chain return more than one value, but sometimes this results in a fairly large number of if err != nil blocks and extra variables that aren't really necessary for the specific case that you're dealing with. So how about a way to chain methods that can return errors, not as a way to replace the existing error checking system, but just as a way to make dealing with those specific annoyances easier? Less of a complete overhaul, in other words.

What I'm thinking is something like this:

// Given the following:
type A
func (a A) M1() (A, error)
func (a A) M2() (A, int, error)
type B
func (a A) M3() (B, error)

// Then allow the following:
b, err := a.M1()?.M2()?.M3()

The idea here is that a ?. could be used to chain a method call off of a method that returned a value and an error, thus making such chaining explicit. The entire chain of method calls would have the return types of the final method, and if any of the method calls that had a ?. attached to them had a non-nil error, everything except for the error return would just be zero values. Each method would also have to be defined for the first return of the attached method, thus resulting in the int return from M2() in the example just simply being completely ignored.

This approach also doesn't break the fine-grained support of if err != nil, as it provides no mechanisms for actually handling errors, i.e. no automatic panicking or return. It just makes it a bit easier to deal with large numbers of calls in cases where differentiating for every call is overkill.

Hmmm... As I finish writing this, I'm not entirely sure what my opinion is of it. I'll just leave it here so that I can see what other people think, assuming that anyone responds, of course.

[beoran]

You can already do it with a slightly different chaining API:
https://play.golang.org/p/MiEhXUIdxbS

package main

import (
	"fmt"
)

type Vector2D struct {
	X float64
	Y float64
}

func (v1 Vector2D) Add(v2 Vector2D) (Vector2D, error){
	return Vector2D{v1.X + v2.X, v1.Y + v2.Y}, nil
}

func (val Vector2D) Chain(calls ...func(v2 Vector2D) (Vector2D, error)) (Vector2D, error) {
	var err error
	for _, call := range calls {
		val, err = call(val)
		if err != nil {
			return val, err
		}
	}
	return val, nil
}

type Scale float64

func (s Scale) Div(v1 Vector2D) (Vector2D, error){
	if s == 0 {
		return v1, fmt.Errorf("Division by 0")
	}
	return Vector2D{v1.X / float64(s), v1.Y / float64(s)}, nil
}

func (s Scale) Mul(v1 Vector2D) (Vector2D, error){
	return Vector2D{v1.X * float64(s), v1.Y * float64(s)}, nil
}

func main() {
	v1 := Vector2D { 7, -3}
	v2 := Vector2D { 3,  7}
	v3 := Vector2D {10,-10}
	res2, err2 := v1.Chain(v2.Add, v3.Add, Scale(3).Mul, Scale(0).Div, Scale(2).Div)
	fmt.Printf("Hello %v %v\n", res2, err2)
}

And with generics, it will be even easier, I think.

[songmelted]

EDIT most of the below post was removed by me to avoid going to far off topic with only brainstorming quality of thoughts.

@DeedleFake You really got me brainstorming with your post. I would be interested to know if this proposal could be generalized in a satisfactory way to enhance go's error handling overall.

Could you link me to a good summary of what was discussed and what people did and didn't like about it??

[songmelted]

@beoran I appreciate that you can create a new package with existing go syntax that accomplishes my goal of chaining methods and handling errors. But what I want is an easy, baked in way to chain calls from existing packages without altering them or making a bunch of boilerplate wrappers.

[songmelted]

@DeedleFake My original thought is to keep this feature limited to method calls with identical return types of type (T, error).

This way, regardless of where the chain is broken, we are returning an error and a value for T that was made to be returned with that error.

I do think it's worthwhile to consider allowing different return types, T, in the chain as long as it follows the (T, error) pattern.

It would have to return the type of the final method call even if the chain was broken earlier. Since the previous methods won't be aware of what the final type T is, they can't return a meaningful value for T. Go would need to supply a nil value of type T to return with the error.

For instance, if the following produced an error in M1(), even though M1() returns (A, error), this call would return (B, error) with B being nil.

It seems like any reduced functionality called by not returning the type A that was returned by M1 with the error should be evident to the programmer. So the programmer can weigh the benefit of this pattern for any specific use.

// Given the following:
type A
func (a A) M1() (A, error)
func (a A) M2() (A, int, error)
type B
func (a A) M3() (B, error)

// Then allow the following:
b, err := a.M1().M2().M3()

I am open to that sort of concept, but wanted to keep my initial post more narrow.

Do you think it is worth broadening this way? What drawbacks do you see there?

[DeedleFake]

@songmelted

Could you link me to a good summary of what was discussed and what people did and didn't like about it??

I'm mostly thinking of the draft design for error handling, but there were quite a few other proposals, too, such as #32437, #40821, and #39148, all of which I've basically chosen at random from the rather lengthy list.

My original thought is to keep this feature limited to method calls with identical return types of type (T, error).

It's simpler, but I'm just not sure how useful that would be. People could, and probably would, certainly write APIs specifically designed to be capable of being used that way, but in general I don't think it would really be that useful.

[ianlancetaylor]

But what I want is an easy, baked in way to chain calls from existing packages without altering them or making a bunch of boilerplate wrappers.

When designing language changes, we should primarily consider future Go code. We should not optimize for existing Go code. For the foreseeable future there will always be much much more future Go code than there is current Go code, and Go makes it easy to write tools to rewrite current Go code.

[songmelted]

@ianlancetaylor

But what I want is an easy, baked in way to chain calls from existing packages without altering them or making a bunch of boilerplate wrappers.

When designing language changes, we should primarily consider future Go code.

That is a really well stated point.

I am not as well spoken and wasn't as clear as I should have been. When I mention "existing packages", I don't mean packages which already currently exist. I meant packages that I am importing, rather than developing.

Currently, I can develop my packages to essentially accomplish my end goal of chaining method calls while still handling errors. It is a bit of boilerplate. It would be nice to have chain calling built in to succinctly use it without the boilerplate, but that is a secondary to my main issue.

My main issue is that if I would like to chain call a method defined in a package I am importing, I have to either rewrite the package with boilerplate to support it or make wrappers. Neither of these things are ideal.

Chain method calls are important to me. It can make very readable code.

If we can come up with a sensible way to allow chain method calling that doesn't rely on an imported package implementing anything special, then both of my issues are solved. I would then be able to chain call methods from other packages without wrappers or fixes. I would also be able to "implement" chain calling in my packages without any boilerplate as it would just be available.

@ianlancetaylor What is your opinion on chain calling methods in general?

[songmelted]

@DeedleFake Thanks for the references. I will do some reading.

Also, what is the inspiration for the syntax you suggested of b, err := a.M1()?.M2()?.M3() ?

Could we extend your syntax to allow for choosing which return value to chain call off of when there is more than one choice?

For instance, if we had

type A
func (a A) M1() (A, B, error)
type B
func (a A) M3() (B, error)

Could we extend your syntax to allow a.M1().M3()? We would have to specify that M3() is chaining off of the second return value from M1().

[DeedleFake]

@songmelted

The syntax is based on the ?. syntax used for nullability and/or error handling in quite a few other languages including JavaScript, Rust, and Kotlin. It actually wasn't my first thought, though. My first thought was some kind of keyword that could be prefixed to a method chain, such as check, which would alter the behavior of all method calls in the chain.

Could we extend your syntax to allow a.M1().M3()? We would have to specify that M3() is chaining off of the second return value from M1().

Probably, but I think that's really starting to get far too complicated for very little benefit, unless someone could come up with a really simple way to do it. I doubt that any simple way to do it that gives full control over how to chain the method calls exists, though.

[ianlancetaylor]

@ianlancetaylor What is your opinion on chain calling methods in general?

Speaking purely personally, I think it can be useful in limited contexts, but I think that it encourages people to restrict themselves to that specific style, which then leads to contorted code for methods that really ought to return more than a single result. I'm not at all convinced that we should have a language construct that specifically and only supports this style of programming. That's just me, though, I'm willing to listen to other opinions.

[deanveloper]

I actually really like this proposal because it actually solves the if err != nil "problem" in a way that doesn't just boil down to "make a 1-line version of the 3-liner". The issue isn't if err != nil itself, it's chained/nested calls to functions that return (T, error). What I don't like about this proposal is what @ianlancetaylor had hinted at, it doesn't solve nested calls (ie ErrFunc3(ErrFunc2(ErrFunc1())) or repeated calls (ie r.Read(slice); r.Read(slice); r.Read(slice)).

[songmelted]

@deanveloper I think we could extend the syntax suggested by @DeedleFake to capture recursion and repeated calls:

chain calls (from @DeedleFake's comment)

b, err := a.M1()?.M2()?.M3()

recursion

b, err := ErrFunc3(ErrFunc2(ErrFunc1()?)?)

repeated calls

b, err := {
    r.Read(slice)?
    r.Read(slice)?
    r.Read(slice)
}

I have no depth in developing languages. But from a user-experience point of view, I think this would be both readable and practical. In my mind, I would view the question marks as asking the question, "did this return an error?" and then returning that error or continuing to the next call if it is nil.

In each case, intermediate calls would cause the entire expression to return if the error was non-nil. In such a case, the value returned would be the return type of the final call with the error being set to the non-nil error from the intermediate call.

Can anyone with greater depth in this area weigh in? Do you have any thoughts, @ianlancetaylor?

[ianlancetaylor]

Since you ask, I don't have strong feelings here, but to me this syntax

b, err := ErrFunc3(ErrFunc2(ErrFunc1()?)?)

seems obscure--the ? is too easy to miss. And this syntax

b, err := {
    r.Read(slice)?
    r.Read(slice)?
    r.Read(slice)
}

is not like anything else in the language.

All of these ideas, including the original idea in this issue, are based on program flow in which the result of one function is passed to another function or is ignored. But my sense is that a lot of the code I write does not fit into that pattern. For example, there is a common pattern of open/do-something/close, where each of the operations can return an error, that doesn't seem to be helped by any of these ideas.

I just took a quick look at some code I wrote unrelated to the Go standard library, github.com/ianlancetaylor/demangle, to see how often I wrote something like

    if err != nil {
        return err
    }

In about 6500 lines of non-test code I wrote something like that exactly once. This could certainly be a special case. But still, while I certainly think there is something that can be improved about Go error handling, I'm not sure that the ideas here are quite getting at the heart of the problem.

And of course see also #40432.

[inliquid]

I'd like to add that there is actually a case in language where (a kind of) some similar ideas were implemented. It's pipelines in Go templates.
So if for example we have these functions:

func F1(input T) (output T, err error)
func F2(input T) (output T, err error)
func F3(input T) (output T, err error)

which process some input, we can than construct this in templates:

{{<input> | F1 | F2 | F3}}

The result in case of nil error in pipeline will be the output of F3.

So the output at each step is passed to next function and in case of error, whole pipe will be stopped:

The output of a command will be either one value or two values, the second of which has type error. If that second value is present and evaluates to non-nil, execution terminates and the error is returned to the caller of Execute. 

edit: grammar

[beoran]

@inliquid That's right. I suppose we could add ||| or similar operator which will likely be backwards compatible that works in the way you suggest. I suggest ||| since that stands out more than ?. This could actually solve quite a bit of problems with error handling in a way that Go users who use templates are already familiar with.

Then the examples become:

b, err := {{ r.Read(slice) ||| r.Read(slice) ||| r.Read(slice) }}

And for chaining, perhaps:

myNewVar, err := {{ myVar ||| .Add(5) ||| .Multiply(2) ||| .Pow(4) }}

Edit: I think we'll need {{ }} too to mark the block's beginning and end.

[deanveloper]

related: #33361

[ianlancetaylor]

The original idea in this proposal seems well suited for certain specific packages that use methods that fit the required pattern. But it does not seem like it works well for most Go code. Specific packages can already use other mechanisms, such as the panic/recover used by packages like encoding/json.

Adding this as to the language will require adding a new feature that everybody will have to learn, although it seems unlikely to be generally useful.

Therefore, this is a likely decline. Leaving open for four weeks for final comments.

[ianlancetaylor]

No further comments.