Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dhole clean up #91

Open
wants to merge 6 commits into
base: schemapack
Choose a base branch
from
Open

Dhole clean up #91

wants to merge 6 commits into from

Conversation

mephenor
Copy link
Member

No description provided.

@mephenor mephenor requested a review from sbilge February 11, 2025 15:07
@coveralls
Copy link

Pull Request Test Coverage Report for Build 13269764680

Details

  • 76 of 76 (100.0%) changed or added relevant lines in 21 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.02%) to 94.324%

Totals Coverage Status
Change from base Build 13196282626: 0.02%
Covered Lines: 1263
Relevant Lines: 1339

💛 - Coveralls

Copy link
Member

@sbilge sbilge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I have a couple of suggestions to fix type errors without ignoring them, and another suggestion related to the placement of 'resolve_schema_object_pathandresolve_data_object_path`

target_schema.setdefault("properties", {})[property_name] = deepcopy(source_schema)


def resolve_schema_object_path(json_schema: Mapping[str, Any], path: str) -> Any:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not really sure about the place of this function. In terms of its purpose, it partially fits in model_transform but at the same time, it is more of a helper function to modify the json schema that a path leads to. Also, grouping this with resolve_data_object_path seems more natural to me. How about we create path_utils.py and add them there?

Or maybe a better option would be creating a resolve_path module that encapsulates resolve_path.py which can be renamed as resolve_path_relations.py and maybe something like resolve_path_properties.py that includes the functions resolve_schema_object_path and resolve_data_object_path?

@@ -175,3 +172,25 @@ def get_modification_target(
if not isinstance(target, dict) or property in target:
raise EvitableTransformationError()
return target


def resolve_data_object_path(data: Mapping, path: str) -> Any:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the same suggestion/comment I made to resolve_schema_object_path

from metldata.builtin_transformations.common import SourcePath


class CopyContentSchemaPath(BaseSettings):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the reason we wanted to separate CopyContentSchema instead of using the NewContentSchemaPath was to reduce the config file fields for the empty object_path. But if this will cause typing problems requiring ignore, I think going back to how it was is preferable here, i.e. replacing CopyContentSchema with NewContentSchemaPath.

@@ -56,3 +56,13 @@ class TargetSourceInstructionProtocol(Protocol):
Instruction = TypeVar(
"Instruction", bound=TargetInstructionProtocol | SourceInstructionProtocol
)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding InstructionProtocol to Instruction bound solves the type problem.

Suggested change
Instruction = TypeVar(
"Instruction", bound=TargetInstructionProtocol | SourceInstructionProtocol | InstructionProtocol
)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate on this one?
Once I remove the type: ignores it's mypy is back to complaining

@@ -39,4 +39,4 @@ def check_model_assumptions(
assert_class_is_source(path=path, instruction=instruction)
assert_path_classes_and_relations_exist(model=schema, path=path)
assert_no_relation_target_multiplicity(model=schema, path=path)
assert_object_path_exists(model=schema, instruction=instruction)
assert_object_path_exists(model=schema, instruction=instruction) # type: ignore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my comment below about using NewContentSchemaPath in copy_content.

@@ -32,7 +32,7 @@ def copy_content(
"""Apply all transformation instructions."""
return transform_data(
data=data,
instructions_by_class=instructions_by_class,
instructions_by_class=instructions_by_class, # type: ignore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my comment above related to the Instruction type. That should solve the type problem we suppress here.

@@ -62,7 +60,7 @@ def add_content_schema_copy(
raise EvitableTransformationError() from exc

add_property_per_instruction(
cls_instruction=instruction,
cls_instruction=instruction, # type: ignore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going back to NewContentSchemaPathshould also resolve this.

@@ -9,14 +9,12 @@ copy_content:
- class_name: Sample
target_content:
property_name: referenced_filename
object_path: ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think going back to this version is better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants