Macro parsing

[cmcarthur]

work in progress. this is promising -- it meaningfully speeds up compilation on a couple of my test projects. @drewbanin i'd love for you to give this a try on some of the slow-to-compile projects that we know of and see the impact.

[drewbanin]

This works really well @cmcarthur! What else do we need to do to get this merged? Is it touch-up stuff, or is it missing something more fundamental?

[beckjake]

I think this is all set, now that I handled the two identical filepaths in one project issue

[cmcarthur]

the key should be unique-per-file, instead of unique-per-node, no?

[beckjake]

I'm pretty confident unique-per-node is correct!

• models are a single node per file and their raw_sql is the full file, so it's exactly the same
• macros/operations have multiple nodes per file, and the raw_sql is the block contents their generated templates only have the named macro, and each block has a (project-unique, which is vastly more than we care about) unique name

One thing we might need to do is add in the root path/project name to the key. I hadn't considered dependent projects that happen to have an identical layout until just now. It seems that at least in some tests, original_file_path is actually the just the path under macros/.

[beckjake]

Ok, you were totally correct. It's all about dependency packages, I misunderstood the issue because I missed the difference between do_something and do_something2!

[drewbanin]

I ran dbt a bunch with some sample dbt projects. For larger, realistic projects like our internal-analytics repo, the majority of the time is spent on running parse-time queries (like already_exists()), so it mutes the benefit of this PR.

For smaller sample projects, the difference is small, but significant. After some non-rigorous timing, I found:

branch	dbt compile time
dev/guion-bluford	3.1s
macro-parsing	2.3s

So, this represents a 25% speedup, even if it's generally less than one second. Also worth restating: this effect is less apparent if a project has even a single adapter.already_exists() call that runs at parse time.

[beckjake]

Yeah, that's about what I'd expect when to comes to the already_exists check.

[drewbanin]

I don't have enough insight into the jinja internals to have any input on the code here. I did however run this on a bunch of projects, and it seemed to work just fine. LGTM!