Improve pdp project config structure + functionality

[bdewilde]

changes

• adds a "v2" pdp project configuration schema, which differs from the previous iteration in structure and scope, plus additional validation and documentation
• adds an example template file for the v2 config
• adds a unit test to check that the schema is working correctly

Out-of-scope: Using this config in nbs. I tried that in another PR, but it got much too big.

context

This file is meant to record the full set of configuration options for a given school's pipeline in a way that's consolidated, human-readable, and not just hard-coded magic variables spread across multiple notebooks. It's especially useful when parameters must be shared and consistent across nbs; consolidating them here makes that task much easier and less error-prone.

The "v1" config was a WIP, but then a couple schools over in the private repo started using it, and I didn't want to break their shit. So, here's a "v2". In the next minor (not patch) release of this package, I'll finish the transition and make this v2 into the only option. A bit awkward, but only temporary, for the sake of a smooth transition.

questions

• What do you think of the config's structure and contents?
• Is there additional validation I should be doing, to ensure things make sense?

[vishpillai123]

This looks good! I had a few questions but overall I'm good with merging.

[vishpillai123]

Do lines 4-12 need to go under modeling.training? OR are they meant to be used for more than just modeling.training?

[bdewilde]

Yup! These get used in both preprocessing and model training.

[vishpillai123]

This is interesting using a chronology column. How is this typically used?

[bdewilde]

afaik We haven't used this configuration in our models, but we could. It's supported by AutoML (see the reference link), so included for completeness.

[vishpillai123]

We may also want to add support_threshold here as an optional parameter.

[bdewilde]

This is included with the trained model itself, see min_prob_pos_label

[vishpillai123]

would preprocessed = training dataset?

[bdewilde]

In the case of the labeled dataset, yes. For unlabeled data, it would be the dataset to be given to the model as input in order to produce predictions. (Does that have a name?)

[vishpillai123]

Oh also I wanted to ask though this is probably outside of the scope - if a school has multiple models (a retention and graduation model for example), would we want multiple configs or do we need to enable that within a config?

I'm assuming multiple configs would be easier.

[bdewilde]

Oh also I wanted to ask though this is probably outside of the scope - if a school has multiple models (a retention and graduation model for example), would we want multiple configs or do we need to enable that within a config?

I'm assuming multiple configs would be easier.

Good question! But yes, out of scope -- it hasn't ever come for a PDP school, right? :)

I'd probably follow the example of datasets, and make a top-level models dict with keys as model names and values as corresponding model configuration. I could do that as a fast follow-up, if you want.