[Tracking] Partial parsing polish

[leahwicz]

This is the list of work we want to complete to polish off partial parsing before we put it down and GA the v0.20.0 release

Todo:

• add test coverage (Add test cases for partial parsing #3449)

[gshank]

• Handle doc references
• Test macros
• Test data tests, analyses
• Test snapshots
• Test seeds -- what happens with big files?
• Test disabled/enabled
• More logging, event data, including flag that partial parsing is actually happening
• More granular handling of dbt_project.yml file (hooks, etc)
• Minimum validation of yaml file: expected keys point to lists containing dictionaries with 'name' key
• Figure out what's going on with to_dict and schema_source_file (pp_dict is supposed to be skipped but isn't)
• Make all file_id methods properties so we are consistent in how they're called

[jtcohen6]

Thanks so much for the enumeration @gshank. Let's organize these into three priority groups:

• p0: Add test cases, handle buggy behavior (i.e. errors that brick entire parse)
• p1: Fast-but-wrong edge cases (e.g. changes to docs, nested macros, vars, env vars, "volatile" Jinja context variables)
• p2: Correct-but-slower-than-strictly-necessary cases (e.g. changes to profile/target, certain behavior around vars)

[jtcohen6]

Some known p1 edge cases:

• Nested macro dependencies (gist)
• Volatile Jinja variable dependencies (gist, Jinja functions remain static in YAML files on RPC until modified #2330)

[leahwicz]

P0:

• Tests on ephemeral models
• Partial parsing: do minimal validation on schema file content
• Add test cases for partial parsing
• Update project load tracking for v0.20 parsing changes

P2:

• More granular handling of dbt_project.yml file (hooks, etc)
• Handling of vars, env variables, etc.

[jtcohen6]

Handling of vars, env variables, etc.

We should split this up into more subtasks, and I think some of those subtasks should actually be p1s: dependencies on nested macros, env vars, and volatile Jinja variables are all "fast and incorrect" today.

[jtcohen6]

I'm going to close this, given that it's the end of the cycle and we managed to hit all our P0s. Nice work team!