Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add column/data zipping functionality #128

Merged
merged 6 commits into from
Nov 22, 2024

Conversation

ollietulloch
Copy link
Contributor

This PR adds column zipping functionality to the mapper, where 2 (or more) columns can have their string values split by a given split_char and then be zipped together in a prescribed order.

E.g. given this csv data

"title","value"
"species,colour,legs","dog,brown,4"

and this data:

    - column: title
      mappings:
      - field: zipped_field
        zip_order: 1
        split_char: ","
    - column: value
      mappings:
      - field: zipped_field
        zip_order: 2

This mapped_line would be produced:

{ "zipped_field"=>[["species", "dog"], ["colour", "brown"], ["legs", "4"]],
   :rawtext=>{"title"=>"species,colour,legs", "value"=>"dog,brown,4"}}

Copy link
Contributor

@bshand bshand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good to me. I've identified one minor typo.

I don't know whether you want to add any sort of validations (or error handling in the mapping code), to deal with incomplete zip_order entries or the absence of a split_char.

docs/zipped-field-mapping.md Outdated Show resolved Hide resolved
@ollietulloch
Copy link
Contributor Author

This all looks good to me. I've identified one minor typo.

I don't know whether you want to add any sort of validations (or error handling in the mapping code), to deal with incomplete zip_order entries or the absence of a split_char.

Yep, I thought about this a bit. I think I may just update the code to check for :zipped_values and :split_char before attempting the splitting and zipping e.g.

elsif field_data[:zipped_values].present? && field_data[:split_char].present?

@ollietulloch
Copy link
Contributor Author

This all looks good to me. I've identified one minor typo.
I don't know whether you want to add any sort of validations (or error handling in the mapping code), to deal with incomplete zip_order entries or the absence of a split_char.

Yep, I thought about this a bit. I think I may just update the code to check for :zipped_values and :split_char before attempting the splitting and zipping e.g.

elsif field_data[:zipped_values].present? && field_data[:split_char].present?

I've added 58a7b92 which ensures there is enough field config present before attempting to zip.

Copy link
Contributor

@bshand bshand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good.

@ollietulloch ollietulloch merged commit 8ae9027 into main Nov 22, 2024
19 of 20 checks passed
@ollietulloch ollietulloch deleted the feature/zip_multiple_columns branch November 22, 2024 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants