Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix various issues with import script. #339

Merged
merged 1 commit into from
May 18, 2015
Merged

Fix various issues with import script. #339

merged 1 commit into from
May 18, 2015

Conversation

monfresh
Copy link
Member

  • Improve logic that determines whether or not a location needs to be geocoded. In addition, the location importer now checks to see if an address already exists for the location. If so, it updates that same address instead of assigning address attributes, which, unbeknown to me, actually destroyed the existing address and created a new one. The side effect of destroying the address is that it triggered the reset_location_coordinates callback, which in turn triggered the geocoding since the location no longer had coordinates.
  • Invalid entries no longer prevent the entire CSV file from being saved to the database. Now, valid entries get saved, and error messages are displayed for invalid entries.
  • If a CSV file doesn't have sequential IDs, the database will now save the entry with the same ID as in the CSV file. Previously, the database would assign sequential IDs, which would cause mismatches between CSV files that refer to a particular foreign key.
  • Added a conditional for the touch_locations callback in organization.rb to improve performance during initial import of organizations.
  • The console output that lets you know which file is being imported has been moved out of the rake task and into the EntityImporter. That way, if a file is skipped (because it's not required and is missing or empty, for example), then the console won't say that it is being imported.

Closes #336.

- Improve logic that determines whether or not a location needs to be geocoded. In addition, the location importer now checks to see if an address already exists for the location. If so, it updates that same address instead of assigning address attributes, which, unbeknown to me, actually destroyed the existing address and created a new one. The side effect of destroying the address is that it triggered the `reset_location_coordinates` callback, which in turn triggered the geocoding since the location no longer had coordinates.

- Invalid entries no longer prevent the entire CSV file from being saved to the database. Now, valid entries get saved, and error messages are displayed for invalid entries.

- If a CSV file doesn't have sequential IDs, the database will now save the entry with the same ID as in the CSV file. Previously, the database would assign sequential IDs, which would cause mismatches between CSV files that refer to a particular foreign key.

- Added a conditional for the `touch_locations` callback in organization.rb to improve performance during initial import of organizations.

- The console output that lets you know which file is being imported has been moved out of the rake task and into the EntityImporter. That way, if a file is skipped (because it's not required and is missing or empty, for example), then the console won't say that it is being imported.

Closes #336.
@coveralls
Copy link

Coverage Status

Coverage increased (+0.0%) to 99.41% when pulling 02076ad on 336-import-fixes into f26a8e7 on master.

monfresh added a commit that referenced this pull request May 18, 2015
Fix various issues with import script.
@monfresh monfresh merged commit ad841f6 into master May 18, 2015
@monfresh monfresh deleted the 336-import-fixes branch May 18, 2015 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Geocoder::OverQueryLimitError while importing locations, addresses
2 participants