Looking up a node from a branch that is also a part of a group introduced a deadlock

[rebeccaskinner]

If something is inside of a conditional branch and is part of a group, then doing a lookup on that node without an explicit dependency introduces a deadlock.

This causes a deadlock:

switch "deadlock" {
  default {
    task.query "bar" {
      query = "echo BAR"
      group = "foo"
    }
    task.query "foo" {
      query = "echo OK - {{lookup `task.query.bar.status.stdout`}}"
    }
  }
}

but this doesn't:

switch "deadlock" {
  default {
    task.query "bar" {
      query = "echo BAR"
      group = "foo"
    }
    task.query "foo" {
      query = "echo OK - {{lookup `task.query.bar.status.stdout`}}"
      depends = ["task.query.bar"]
    }
  }
}

[rebeccaskinner]

The depends actually doesn't matter, it's just nondeterministic as to whether it will deadlock or return the following error:

rebecca@rmbp2:~/go/src/github.com/asteris-llc/converge$ ./converge plan --local deadlock.hcl
2016-10-24T15:01:33-05:00 |INFO| serving    addr=:47740 component=rpc
2016-10-24T15:01:33-05:00 |WARN| skipping module verification   component=client
2016-10-24T15:01:33-05:00 |INFO| resolving conditional macros   component=rpc function=ResolveConditionals runID=92d9dee6-aaed-4a4c-917a-8f47aa8d793b
2016-10-24T15:01:33-05:00 |INFO| got status component=client file=deadlock.hcl id=root/macro.switch.deadlock/macro.case.default/task.query.foo run=STARTED stage=PLAN
2016-10-24T15:01:33-05:00 |INFO| got status component=client file=deadlock.hcl id=root/macro.switch.deadlock/macro.case.default/task.query.bar run=STARTED stage=PLAN
2016-10-24T15:01:33-05:00 |ERROR| planning failed   component=rpc error=1 error(s) occurred:

* root/macro.switch.deadlock/macro.case.default/task.query.foo: error rendering field query: node is unresolvable function=executor.Plan location=deadlock.hcl runID=92d9dee6-aaed-4a4c-917a-8f47aa8d793b
2016-10-24T15:01:33-05:00 |FATAL| could not get responses   component=client error=error getting status response: rpc error: code = 2 desc = planning deadlock.hcl: 1 error(s) occurred:

* root/macro.switch.deadlock/macro.case.default/task.query.foo: error rendering field query: node is unresolvable file=deadlock.hcl

[rebeccaskinner]

There seem to be a few intertwined issues that are involved here:

Right now I've narrowed the deadlock down to graph.go:rootFirstWalk where we are looping infinitely because AreSiblingIDs is returning true for a node that is a sibling by name but isn't added to the graph correctly so DownEdges isn't returning it. This is nondeterministic though, so I'm going to first investigate what's causing it. There seem to be a couple of other problems too but it remains to be seen how many of them are symptoms of whatever is causing the graph to get messed up.

[rebeccaskinner]

fixed by #423