Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/sbachmei/mic 3957 auto generate default config #39

Merged

Conversation

stevebachmeier
Copy link
Contributor

Title: Auto-generate configuration file

Description

  • Category: refactor
  • JIRA issue: MIC-3957

This does away with the default_configuration.yaml and instead generates the configuration
file realtime. Some notes:

  • The is_implemented attributes are added to make it easier for the engineers.
    When something is implemented (eg a new function or a new form), then we can
    change that to True and it will be added to the config file.
  • We've added extra keys to the ConfigTree: "column_noise" and "row_noise".
  • We've added an extra layer concept to this ConfigTree object. We now have
    a "baseline" layer (which is used to instantiate the configuration with the vast
    majority of values), a "default" layer which are default values that deviate from
    baseline (the obvious example is row omission values which are mostly
    0 by default (ie the "baseline" is 0.0) but a few forms have non-zero values, and the
    "user" layer.

Testing

All pytests pass

@stevebachmeier stevebachmeier force-pushed the feature/sbachmei/MIC-3957-auto-generate-default-config branch from a228752 to 7523550 Compare April 7, 2023 15:28
@stevebachmeier stevebachmeier marked this pull request as ready for review April 7, 2023 17:00
@mattkappel
Copy link
Contributor

mattkappel commented Apr 7, 2023

If we generate this at runtime, how will a user know what to override? It would be better if we generate this file programmatically, and commit it to the repo, imho....

@stevebachmeier
Copy link
Contributor Author

stevebachmeier commented Apr 7, 2023

If we generate this at runtime, how will a user know what to override? It would be better if we generate this file programmatically, and commit it to the repo, imho....

I'll have to defer to @rmudambi as that's extremely different than what we've been discussing. I will say that folks seemed excited to not have too ever look at this file (to the point of even offering to print it out after generating seemed to get some pushback)

@mattkappel Ok, after discussing with Rajan we decided to just keep as is since we don't yet have a requirement for how the user should interact. It is trivial to write it out as you recommend if desired.

Copy link
Collaborator

@rmudambi rmudambi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great start! I've got some comments.

src/pseudopeople/configuration.py Outdated Show resolved Hide resolved
src/pseudopeople/entity_types.py Outdated Show resolved Hide resolved
src/pseudopeople/interface.py Outdated Show resolved Hide resolved
src/pseudopeople/noise.py Outdated Show resolved Hide resolved
src/pseudopeople/noise_entities.py Outdated Show resolved Hide resolved
tests/integration/test_interface.py Outdated Show resolved Hide resolved
tests/unit/test_column_noise.py Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do need to have a strategy for testing the configuration generation code creates the correct configuration object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - I thought I'd already written that? Weird.

We should also test that each specific interface call (for each form) calls the expected functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, I misunderstood your point. Yeah, I'm not really sure how to test that offhand. Writing tests for that might be more difficult than having just stuck with the initial plan of having a default yaml.

tests/unit/test_noise_form.py Outdated Show resolved Hide resolved
tests/unit/test_noise_form.py Outdated Show resolved Hide resolved
Copy link
Contributor

@albrja albrja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating all the tests.

src/pseudopeople/configuration.py Outdated Show resolved Hide resolved
src/pseudopeople/configuration.py Outdated Show resolved Hide resolved
src/pseudopeople/configuration.py Outdated Show resolved Hide resolved
src/pseudopeople/configuration.py Outdated Show resolved Hide resolved
src/pseudopeople/configuration.py Show resolved Hide resolved
src/pseudopeople/entity_types.py Outdated Show resolved Hide resolved
src/pseudopeople/entity_types.py Outdated Show resolved Hide resolved
src/pseudopeople/entity_types.py Outdated Show resolved Hide resolved
src/pseudopeople/noise.py Outdated Show resolved Hide resolved
src/pseudopeople/noise_entities.py Outdated Show resolved Hide resolved
tests/unit/test_column_noise.py Outdated Show resolved Hide resolved
tests/unit/test_column_noise.py Outdated Show resolved Hide resolved
tests/unit/test_noise_form.py Outdated Show resolved Hide resolved
src/pseudopeople/noise.py Outdated Show resolved Hide resolved
src/pseudopeople/noise.py Show resolved Hide resolved
tests/unit/test_configuration.py Outdated Show resolved Hide resolved
tests/unit/test_configuration.py Outdated Show resolved Hide resolved
tests/unit/test_configuration.py Outdated Show resolved Hide resolved
tests/unit/test_configuration.py Outdated Show resolved Hide resolved
tests/unit/test_configuration.py Outdated Show resolved Hide resolved
assert config_token_noise_level == baseline_level.token_noise_level
else:
assert config_token_noise_level == default_token_noise_level
if noise_type.additional_parameters:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-worked this to be more inline with how we are doing the previous checks. It also looks at the DEFAULT_NOISE_VALUES now which it previously didn't.

Copy link
Collaborator

@rmudambi rmudambi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have a minor bug in your additional parameters test. Fix and merge (and release).

tests/unit/test_configuration.py Outdated Show resolved Hide resolved
@stevebachmeier stevebachmeier merged commit 43739d5 into develop Apr 11, 2023
@stevebachmeier stevebachmeier deleted the feature/sbachmei/MIC-3957-auto-generate-default-config branch April 11, 2023 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants