[engine] Improving performance by iteratively hydrate build file addr…

…esses within SelectTransitive (#4349)

### Problem

As described in #4283, the issue with recursively executing `SelectTransitive` is lots of calls to `store_list` creating massive intermediate lists that increases memory footprint as well as impacts perf.

### Solution

Same idea as in #4265 using `loop_fn` rewrite `SelectTransitive` to execute iteratively. One small difference is in order to use `OrderMap` and `HashSet` for state tracking and outputs are in `Value`s, those `Values` are turned into `Key`s.

ps: previous PR #4334 paved way for this optimization by splitting `SelectTransitive` from `SelectDependencies(...transitive=true)`.

### Result

All existing tests pass. Have verified there is only one `ST` node through viz.

Two follow up items
* #4358 reenable cycle detection
* #3925 Remove BFA now not only is a clean up task, but has performance benefits because the second round hydration is on `Address` which is redundant.

src/python/pants/engine/legacy/graph.py

@@ -13,7 +13,7 @@
 from pants.base.exceptions import TargetDefinitionException
 from pants.base.parse_context import ParseContext
 from pants.base.specs import SingleAddress
-from pants.build_graph.address import Address
+from pants.build_graph.address import Address, BuildFileAddress
 from pants.build_graph.address_lookup_error import AddressLookupError
 from pants.build_graph.build_graph import BuildGraph
 from pants.build_graph.remote_sources import RemoteSources
@@ -333,7 +333,7 @@ def create_legacy_graph_tasks(symbol_table_cls):
     (HydratedTargets,
      [SelectTransitive(HydratedTarget,
                        Addresses,
-                       field_types=(Address,))],
+                       field_types=(BuildFileAddress,))],
      HydratedTargets),
     (HydratedTarget,
      [Select(symbol_table_constraint),

src/python/pants/engine/subsystem/native_engine_version

@@ -1 +1 @@
-1749afda2a8990fd6371be4423dca0f37ad65379
+1c6344ab2dcdbddd692cc4afa7885e4e286f12ba

src/rust/engine/Cargo.toml

@@ -22,9 +22,9 @@ fnv = "1.0.5"
 futures = { git = "https://github.com/alexcrichton/futures-rs", branch = "master" }
 futures-cpupool = { git = "https://github.com/alexcrichton/futures-rs", branch = "master" }
 glob = "0.2.11"
-ignore = "0.1.7"
+ignore = "0.1.8"
 lazy_static = "0.2.2"
-ordermap = "0.2.7"
+ordermap = "0.2.8"
 petgraph = "0.4.4"
 # TODO: Waiting for a 0.4.11 release: https://github.com/alexcrichton/tar-rs/pull/99
 tar = { git = "https://github.com/alexcrichton/tar-rs", branch = "master" }